What happened to Stack Overflow?
Since ChatGPT, new questions have run far below what comparable sites predict. Which technologies fell most, which questions remain, and who left
The question
Stack Overflow questions have been falling since about 2014, so a decline after ChatGPT (released 30 November 2022) is not news by itself. The questions this analysis answers are:
- How much of the fall is beyond the trend that was already under way?
- Is it specific to programming, or did all of Stack Exchange fall?
- Which technologies fell most, what kind of questions remain, and who left: new askers, or the experts who answered?
The closest earlier study, by del Rio-Chanona, Laurentsyeva and Wachs (2024), found a 25% relative drop in Stack Overflow activity within six months of ChatGPT. This analysis extends the window to almost four years and adds tag-level, question-level and user-level detail.
Everything here comes from the public Stack Exchange data dump (2026-Q3 release, cut-off 30 September 2026): 27 sites, including Stack Overflow, its four versions in other languages, six other tech sites and fifteen non-programming sites. Before fitting any model on post-ChatGPT data, I wrote down the hypotheses, methods and decision rules and locked them (pre-registration). Every later change is logged there with the date and whether it was made before or after seeing results.
How big is the gap?
To say how far Stack Overflow fell because of something, you need an estimate of where it would have been otherwise. The main estimate uses a synthetic control (Abadie et al. 2010): a weighted blend of the fifteen non-programming sites (Physics, Mathematics, Science Fiction, Biology, Bicycles and others), with weights chosen so the blend tracks Stack Overflow’s monthly questions as closely as possible from 2017 to November 2022. After that, the blend shows what “Stack Overflow, if it had behaved like those sites” would look like. Before ChatGPT, the blend tracked Stack Overflow within about 6% a month on average.
| Period | Stack Overflow vs synthetic control |
|---|---|
| 2023 | −34% |
| 2024 | −61% |
| 2025 | −81% |
| 2026 (to September) | −92% |
| December 2022 – September 2026, average | −72% (95% interval −76% to −68%) |
The headline number is an average of monthly gaps. Each month counts equally, so the deep gaps of 2025–26 weigh as much as the shallower ones of 2023. Counted in questions instead, Stack Overflow received 1.41 million questions over the whole period against 2.98 million expected, −53% in total.
That is about 1.6 million fewer questions than expected (range 1.2–2.0 million). This counts only questions still in the dump. Stack Overflow deletes many questions, and the dump leaves them out. Counting every question asked, including those later deleted, the shortfall is about 3.1 million (2.4–3.7 million).
Is it real? Stack Overflow’s gap is the largest of all 16 sites when each site in turn is treated as if it were Stack Overflow: it ranks 1 of 16. That is the strongest result this test can give with 16 sites. Fake break dates in 2019, 2020 and 2021, using only pre-ChatGPT data, give gaps of +15% to +24%. Those are positive and far smaller than the real one.
Three other methods agree. Each predicts Stack Overflow’s own path from its pre-ChatGPT history alone, without the comparison sites:
- interrupted time series (a trend-plus-seasonality regression with a break at ChatGPT): −83%, or −79% with the trend fitted only on the faster post-COVID decline
- exponential smoothing (ETS) forecast: −83%
- Prophet forecast: −81%
Across 15 pre-registered robustness checks, the synthetic-control estimate ranges from −83% to −60%. The checks cover other break dates, dropping the 2023 moderator strike, a longer pre-period, weekly data, other donor sets, deleted questions included, and ending in September 2025 before Stack Overflow’s own 2025 policy changes.
Is it just programming?
No site escaped, but programming Q&A fell furthest. The comparison sites also dropped: against their own pre-2022 trends, the median non-programming site fell −31%, against Stack Overflow’s −84% measured the same way. The pre-registered test calls a decline “platform-wide” if the comparison sites fall at least 0.8 times as much as Stack Overflow on the log scale. Here the ratio is 0.21.
That cuts both ways. The fall is mostly specific to programming. But the synthetic control measures Stack Overflow’s fall beyond sites that also fell. If AI tools also pulled people away from Physics or Cooking, the 72% understates Stack Overflow’s total loss.
Stack Overflow’s Russian-language site is a useful extra check, because ChatGPT was not officially available in Russia. A difference-in-differences comparison, which tracks how far the two sites’ paths diverged, puts English Stack Overflow −21% lower relative to the Russian site (95% interval −33% to −7%). Russian users could still reach ChatGPT through VPNs, so this contrast is smaller than the true effect.
Which technologies fell most?
Each of the top 300 Stack Overflow tags of 2019–21 was compared with its own pre-2022 trend in a hierarchical model, which pulls small, noisy tags toward the average. 297 tags remain after excluding three that Stack Overflow retired or merged. Every one fell, and every interval is below zero. Data tags (pandas, dataframe) were among those that fell most, and mobile and enterprise tags (iOS, Android, Kotlin, Spring Boot) among those that fell least.
Take the exact ranking with care. The model extends each tag’s 2017–22 trend four years forward, so a tag that was growing fast before ChatGPT (Flutter, Next.js, PyTorch) gets a steeply rising “expected” line, and its fall looks bigger. In plain counts, with no trend assumed, most large tags fell between −89% and −98% from 2021 to 2025. The two rankings agree only moderately (rank correlation 0.6). On plain counts, the basic, well-documented topics (list, arrays, jQuery, MySQL) fell most. The tag lookup shows both numbers for every tag.
The “thin training data” test
A popular explanation is that chatbots answer well where they saw lots of training data. That predicts that old, well-documented technologies fall most, and that technologies newer than the chatbots’ training data fall least. The pre-registration tested both halves (H2):
- Older tags fell less, not more. The age effect is +0.41 per standard deviation of log age (95% interval +0.34 to +0.48), the opposite of the prediction. Part of this comes from older tags already declining faster before 2022.
- Tags created after ChatGPT’s training cut-off did hold up better. The 66 tags first used after September 2021 (with at least 200 questions) were −49% below a lifecycle benchmark (−55% to −44%), against −85% for the median top-300 tag. The benchmark is how earlier tags grew at the same age. Genuinely new technologies (−51%) and new versions of old ones (−48%) fared about the same.
The rule needed both halves, so H2 is not supported. Both halves are worth knowing: the “thin training data” story fits new tags, but not the comparison between old and middle-aged ones. The two measures also use different yardsticks: a lifecycle benchmark for new tags, the tag’s own trend for old ones. The same lifecycle method applied before ChatGPT gives −9%, so it is not biased much.
What kind of questions remain?
If chatbots take the easy questions, the questions left should be “harder” (H3). Comparing 2021 with 2025, with each year’s questions reweighted to the same mix of tags:
- Longer: median body 1,177 → 1,678 characters; +518 within tags. This holds even under the worst case for deleted questions (Lee bounds, +309 to +695) (Lee 2009).
- Slightly more code: +4.2 percentage points within tags, but not robust to deleted questions.
- Duplicate closures did not fall: 3.6% → 3.6%.
- The share from first-time askers did not fall: 22.5% → 22.6%.
Two of four measures moved as predicted, and the rule needed three, so H3 is not supported. The askers did change, though: the median asker’s account was 641 days old in 2021 and 2,014 in 2025.
Who left?
Newcomers
First-time askers fell about as much as all questions (−69% against the synthetic control). Their share of questions rose +18%, the opposite of the prediction (H4, part a). But fewer newcomers came back. Among people whose first post was a question, the chance of posting again within 12 months fell from 44.0% (2019–21 arrivals) to 28.5% (2023–26 arrivals). For those arriving after Staging Ground launched in June 2024, it was 22.6%. Staging Ground is the review step that new askers’ questions now go through.
The experts who answer
The “core” answerers are the top 1% of answerers in 2019–21. Two years later, 62.7% of them were still answering, and three years later 48.3%. The same group defined in each earlier window kept 69.3% to 75.6% and 61.2% to 68.3%. On the comparison sites, core answerers kept answering at about their usual rate (ratio 1.04 vs Stack Overflow’s 0.75). So Stack Overflow’s experts left faster than earlier generations did. The evidence is not conclusive, for two reasons:
- earlier generations were already dropping out about 2 points faster each time (extrapolated: 65.5% and 55.2%)
- only four earlier groups exist to compare against
After the pre-registered correction for testing four hypotheses at once (Holm 1979), the result does not reach significance (p = 0.024, adjusted 0.095). H5 is consistent with the data but not confirmed.
Is the community still responsive?
Less than it was. The share of questions answered within 24 hours fell from 67% (2019–21) to 55% (2025). Among questions answered within a week, the median wait rose from 0.7 to 2.8 hours. More answers now go to old questions: 37% of 2025 answers were to questions over a year old, against 13% in 2019. This part is exploratory, not pre-registered.
Where did people go?
There is no sign the questions moved to GitHub issues. Take the main GitHub repository for each of 20 technologies (pandas, React, Flutter and so on). Their issue activity drifted down to a median index of 55 (2019–21 = 100) by early 2025. Stack Overflow questions in the matching tags fell to 9, while GitHub as a whole grew to 140.
The likelier destination is chatbots and coding assistants. In Stack Overflow’s own Developer Survey, the share of respondents using AI tools rose from 44% (2023) to 62% (2024) and 78% (2025). Survey respondents are themselves Stack Overflow visitors, so these shares describe that group, not all developers. This analysis can’t observe chatbot use directly, so this part is context, not proof.
What this doesn’t show
- It doesn’t prove ChatGPT alone caused the fall. Several things coincide:
- GitHub Copilot and other coding assistants
- Stack Overflow’s own changes: Staging Ground (2024), opinion-based questions (October 2025) and AI features
- the 2023 moderator strike
- It measures Stack Overflow relative to other sites that also fell. The total effect of AI on Stack Overflow is probably larger.
- It doesn’t count deleted questions in most measures. Including them changes the headline from −72% to −72%. The share of questions later deleted rose after 2022, which affects the question-mix measures most.
- It doesn’t say the remaining questions are “better” or “worse”, only how they changed on four measures.
- It ends at the dump cut-off (30 September 2026). It is a snapshot, not a live tracker.
Scorecard against the pre-registration
| Hypothesis | Result | |
|---|---|---|
| H1 (primary) | Questions fell below what the trend and comparison sites predict | Supported under the monthly-interval reading: −72%, ranked 1 of 16. See the note on the interval below. |
| H2 | Older/larger tags fall more; post-2021 tags fall less | Not supported. Older tags fell less; post-2021 tags did fall less |
| H3 | Remaining questions are “harder” (3 of 4 measures) | Not supported (2 of 4) |
| H4 | Newcomer arrivals fall more than questions, and newcomers return less | Not supported. Arrivals fell in step; returns did fall (44.0% → 28.5%) |
| H5 | The core answerer group shrank faster than before | Not confirmed after correcting for four tests (adjusted p = 0.095) |
A note on H1’s interval. The pre-registration asked for a “95% conformal confidence interval” for the average gap (Chernozhukov et al. 2021). Two readings were possible:
- Monthly reading: average the per-month 95% intervals. This gives −76% to −68%, with 42 of 46 monthly intervals excluding zero.
- Constant-effect reading: test a single effect assumed equal in every month. This gives −15% to +62%, which does not even contain the point estimate, because the gap grew from about 0 to −92% and was nowhere near constant.
I chose the monthly reading after seeing both, and logged it as a deviation. Under the constant-effect reading, H1 would fail its interval criterion. Every other check (rank, time placebos, robustness checks, three other methods) points the same way.
Methods appendix
Data. Stack Exchange data dump, 2026-Q3 (cut-off 2026-09-30), 27 sites, parsed from XML to Parquet. File hashes match the community checksum list, and no watermarked rows were found. Questions are counted by creation date (UTC), with tags as they were when the question was asked. The dump omits deleted posts; deleted-question counts come from the Stack Exchange Data Explorer (SEDE), the site’s public query tool, and are used in a robustness check.
Synthetic control. Monthly log questions, each site demeaned by its pre-period mean. Pre-period 2017-01 to 2022-11, excluding March–June 2020 (COVID); post-period 2022-12 to 2026-09. Weights are non-negative and sum to one, fitted on pre-period months only. The fit’s pre-period error (root mean squared) is 0.059 log points.
Interrupted time series. Weekly log questions with a linear trend, yearly seasonality (Fourier terms), holiday weeks and a COVID indicator. Standard errors are Newey–West (8 lags), which allow for autocorrelation. The level change (+39%) and slope change (−67% per year) at the break can’t be read separately: a positive level change is offset by a steep slope. The article therefore reports gaps. Refitting on pre-period weeks only gives the same gap. A one-year holdout before ChatGPT had errors of −7% to +5%.
Forecasts. ETS and Prophet were trained to November 2022. In a backtest (trained to 2019-11, predicting three years), their errors by horizon year were:
- ETS: +4%, −12%, −23%
- Prophet: −13%, −43%, −62%
Prophet failed that backtest. It is reported because it was pre-registered, not because it is reliable.
Tag model. A hierarchical negative binomial model of monthly question counts for each tag. Each tag has its own pre-trend, seasonality and a post-period line; predictors are age, size, documentation maturity (share closed as duplicates, 2019–21) and family. Tags that Stack Overflow retired or merged are excluded by a fixed rule.
Multiple comparisons. H2–H5 are corrected together with Holm’s method at α = 0.05 (Holm 1979). Each family’s decision p-value is defined in the pre-registration’s deviations log.
Deviations. Every change to the pre-registered plan, with its date and whether it came before or after seeing results, is in the deviations log. The full validation review is in validation.md.
Reproducing. make all rebuilds everything from the raw dump, in about 2.5 hours on a laptop. make all SAMPLE=1 runs the whole pipeline on a small synthetic dump in about 4 minutes. Every number in this article is read from the pipeline’s output files, not typed by hand.
Data licence and attribution. Stack Overflow and Stack Exchange content is licensed CC BY-SA 4.0 by its contributors. This analysis publishes derived counts only, no post text or user names. It is independent and not affiliated with or endorsed by Stack Overflow or Stack Exchange.