Scaling Laws

The year is 2022, San Francisco. Inside OpenAI’s offices, engineers are about to spend tens of millions of dollars’ worth of electricity and silicon to build an object none of them has ever seen: the model that will be called GPT-4. The job will keep thousands of processors busy for months. Nobody can afford to get it wrong.

So before switching the machine on, they do a strange thing. They write down the result in advance.

Not a hunch: a number. By training a series of miniature models, down to ten thousand times smaller in compute, they draw a curve, extend it, and read off it the final error score of their future colossus[1]1. OpenAI, « GPT-4 Technical Report », arXiv:2303.08774 (2023)↗ arxiv.org. The prediction was set down shortly after the run started, without peeking at any partial result, the technical report notes. Months later, the job completes. The number comes in.

It lands on the curve. Exactly where the paper said it would.

That is the anomaly this page is about. AI progress passes for unpredictable, chaotic, full of surprises. Yet at its center sits the most stubborn regularity in modern technology: make the model bigger, feed it more text and more compute, and its error walks down a line so steady people call it a law.

To see it, you need distance. A lot of distance.

So this page is built as a zoom-out. We start from the smallest learning machine ever built, and at every step, the world gets ten times bigger. Count the clicks: there are about twenty-five.

In 1950, Claude Shannon, the father of information theory, built a mechanical mouse named Theseus. It explored a maze of twenty-five squares, bumped into walls, tried again, remembered. It was one of the first machines that learn. Its entire training cost about forty operations of arithmetic[2]2. Epoch AI, base « Notable AI Models » (consultée en juillet 2026)↗ epoch.ai.

Forty. No hidden zeros.

One elementary operation, an addition or a multiplication: specialists call it a FLOP. It is the accounting unit of this whole story, the calorie of artificial intelligence. Theseus consumed forty of them. Today’s models consume up to ten million billion billion times more. Between the two: seventy-five years, and our descent.

In 1957, Frank Rosenblatt’s Perceptron learned to tell simple shapes apart: a few hundred thousand operations[2]2. Epoch AI, base « Notable AI Models » (consultée en juillet 2026)↗ epoch.ai. In 1998, Yann LeCun’s LeNet-5 read bank checks by the millions (its 1989 ancestor already deciphered the zip codes on American mail): trillions of operations. In 2012, AlexNet, trained on two consumer gaming GPUs, crushed an image-recognition contest and set off the deep-learning rush: half a billion billion[2]2. Epoch AI, base « Notable AI Models » (consultée en juillet 2026)↗ epoch.ai.

From forty to half a billion billion. And here is the curious fact: along that dizzying road, some people had already seen the line.

As early as 1993, at Bell Labs, a team that included Corinna Cortes and Vladimir Vapnik noticed that a neural network’s error falls, as you feed it data, along a curve of suspicious regularity, a power law, and proposed using it to predict a model’s final quality without paying for the full training[3]3. C. Cortes, L. D. Jackel, S. A. Solla, V. Vapnik, J. S. Denker, « Learning Curves: Asymptotic Values and Rate of Convergence », NIPS 1993↗ proceedings.neurips.cc. The idea that would justify hundred-million-dollar bets thirty years later already existed, dressed in 1990s clothes.

In 2001, at Microsoft, Michele Banko and Eric Brill fed their algorithms a thousand times more text than usual, one billion words, on a disambiguation task. The curves climbed, straight, without flinching: “log-linear even out to one billion words”[4]4. M. Banko, E. Brill, « Scaling to Very Very Large Corpora for Natural Language Disambiguation », ACL 2001↗ aclanthology.org. And they dropped a sentence that sounds like sacrilege: the worst of their algorithms, fed twenty million words, beats the best one fed a single million[5]5. M. Banko, E. Brill, « Mitigating the Paucity-of-Data Problem », HLT 2001↗ aclanthology.org. The lesson settled slowly into the field’s culture: data crushes cleverness[6]6. A. Halevy, P. Norvig, F. Pereira, « The Unreasonable Effectiveness of Data », IEEE Intelligent Systems (2009)↗ static.googleusercontent.com.

In 2017, at Baidu, Joel Hestness’s team checked it systematically across translation, speech, images, language: power laws everywhere. With one finding that gives a shiver: better architectures shift the curve down, but do not change its slope[7]7. J. Hestness et al., « Deep Learning Scaling is Predictable, Empirically », arXiv:1712.00409 (Baidu, 2017)↗ arxiv.org. Engineering genius buys you a length. Scale decides the race.

By then, training compute had already unhooked from the electronics carrying it: since 2010 it has doubled roughly every six months, where Moore’s law took twenty[10]10. J. Sevilla et al., « Compute Trends Across Three Eras of Machine Learning », arXiv:2202.05924 (2022)↗ arxiv.org[11]11. Epoch AI, « Training compute of frontier AI models grows by 4-5x per year » (2024, mis à jour février 2026)↗ epoch.ai. It is no longer chip progress pulling AI. It is budgets.

January 2020. An OpenAI team led by Jared Kaplan and Sam McCandlish publishes a paper that could pass for cosmology: cascades of curves, exponents, millimetric fits[9]9. J. Kaplan, S. McCandlish et al., « Scaling Laws for Neural Language Models », arXiv:2001.08361 (OpenAI, 2020)↗ arxiv.org. Kaplan is a theoretical physicist. He has just turned onto neural networks the kind of gaze usually reserved for galaxies.

The result rests on three ingredients and one obsession. Take a language model, the program trained to guess the next word. Its performance is measured by an error score, the “loss”: the lower it goes, the better the model guesses. Kaplan’s team varies three quantities: the model’s size (its number of parameters, its “adjustable neurons”), the amount of text swallowed, and the total compute spent. Verdict: each of the three, as long as the other two do not choke it, drives the error down along a power law[9]9. J. Kaplan, S. McCandlish et al., « Scaling Laws for Neural Language Models », arXiv:2001.08361 (OpenAI, 2020)↗ arxiv.org.

A power law is a relationship you can only read properly by changing glasses.

On an ordinary scale, the curve collapses fast: every further gain looks derisory. But switch both axes to a log scale, where each tick multiplies by ten, and the curve unrolls into a straight line. That is the signature of power laws: multiplying the input by ten always buys the same step of progress. Nature was under no obligation to be this legible.

And the line does not hold over some narrow domain of validity. In the paper, it runs across more than seven orders of magnitude[9]9. J. Kaplan, S. McCandlish et al., « Scaling Laws for Neural Language Models », arXiv:2001.08361 (OpenAI, 2020)↗ arxiv.org. As if a single rule governed the coin in your pocket and the gross domestic product.

It must also be said what the law promises, because it is remarkably stingy. The compute exponent is about 0.05: to cut the error in half, you need roughly a million times more compute. Read that way, it is a law of diminishing returns, a wall. The miracle is not the speed of progress.

The miracle is that it never lies.

For there is that admirable confession tucked into the paper’s appendix: “At present we do not have a solid theoretical understanding for any of our proposed scaling laws”[9]9. J. Kaplan, S. McCandlish et al., « Scaling Laws for Neural Language Models », arXiv:2001.08361 (OpenAI, 2020)↗ arxiv.org. Kepler described the planets’ ellipses decades before Newton explained their why; scaling laws are our ellipses, and the Newton has not come yet. Meanwhile, like seventeenth-century navigators, the industry has taken to sailing by laws it does not understand.

A law never tested outside its laboratory is just a pretty curve. Four months after Kaplan’s paper, OpenAI puts it on the table.

May 2020: GPT-3. One hundred and seventy-five billion parameters, one hundred and seventeen times its predecessor GPT-2, ten times the largest dense model in existence[12]12. T. Brown et al., « Language Models are Few-Shot Learners », arXiv:2005.14165 (OpenAI, 2020)↗ arxiv.org. The paper explicitly cites the scaling laws as its working hypothesis, and trains eight models of staggered sizes to put them to the test. Estimated cost of the giant: around 4.6 million dollars of compute, as a floor[13]13. Lambda Labs, « Demystifying GPT-3 » (2020)↗ lambda.ai.

The result, in the caption of a now-famous figure: the power-law behavior “continues for an additional two orders of magnitude with only small deviations from the predicted curve”[12]12. T. Brown et al., « Language Models are Few-Shot Learners », arXiv:2005.14165 (OpenAI, 2020)↗ arxiv.org. The laboratory’s line had just crossed the real world without flinching.

Along the way, something unplanned happens: the giant starts pulling off tasks nobody taught it, just by reading a few examples on the fly. That oddity has an appointment with chapter 5.

This is where the prologue’s bet finds its meaning. In 2023, for GPT-4, OpenAI no longer merely checks the law after the fact: it uses it as a navigation instrument. The final loss is predicted from replicas ten thousand times smaller in compute; some applied scores, like success on programming exercises, are extrapolated from models a thousand times smaller[1]1. OpenAI, « GPT-4 Technical Report », arXiv:2303.08774 (2023)↗ arxiv.org. Two different numbers for two different predictions, the report specifies, and it concedes the limit too: “Certain capabilities remain hard to predict”[1]1. OpenAI, « GPT-4 Technical Report », arXiv:2303.08774 (2023)↗ arxiv.org.

The bill, meanwhile, had changed scale: serious estimates put GPT-4’s training between 40 and 80 million dollars, depending on the accounting[14]14. B. Cottier et al., « The rising costs of training frontier AI models », arXiv:2405.21015 (Epoch AI, 2024)↗ arxiv.org. When that is what you stake on a single roll of the dice, you want to know the outcome in advance.

A law can be right and misread. Kaplan’s paper contained a recipe: at a given compute budget, mostly grow the model; the amount of text matters little. The entire 2020-2022 generation followed it. GPT-3: 175 billion parameters for just 300 billion word-tokens. Gopher, at DeepMind: 280 billion parameters, barely more than one token per parameter. Immense brains, fed like sparrows.

In March 2022, DeepMind redoes the measurements, more carefully, and publishes the correction: at equal budget, size AND data should grow in equal parts[15]15. J. Hoffmann et al., « Training Compute-Optimal Large Language Models », arXiv:2203.15556 (DeepMind, 2022)↗ arxiv.org. To prove it, the team trains Chinchilla: four times smaller than Gopher, fed nearly five times more text, for exactly the same compute bill. Chinchilla beats Gopher everywhere, with a seven-point lead on the reference exam MMLU[15]15. J. Hoffmann et al., « Training Compute-Optimal Large Language Models », arXiv:2203.15556 (DeepMind, 2022)↗ arxiv.org. The giants of the era, the paper concludes, were “significantly undertrained”.

From that bowl, the community distilled a rule of thumb: about twenty tokens of text per parameter. A hygiene note: the formula appears nowhere in the paper; it is derived from it. And it is no constant of nature: Meta’s own measurements, two years later, put it closer to forty at very large scale[19]19. Meta AI, « Introducing Meta Llama 3 » (avril 2024)↗ ai.meta.com.

The story could have ended there. It has a coda, and it may be the passage that says the most about the health of this science.

In 2024, an independent team, Epoch AI, tries to reproduce Chinchilla’s numbers. To recover the data, they go as far as dissecting the vector file of one of the paper’s figures[16]16. T. Besiroglu et al., « Chinchilla Scaling: A replication attempt », arXiv:2404.10102 (Epoch AI, 2024)↗ arxiv.org. Verdict: one of the three statistical fits is defective, with confidence intervals that would have required six hundred thousand experiments where the team had run fewer than five hundred[16]16. T. Besiroglu et al., « Chinchilla Scaling: A replication attempt », arXiv:2404.10102 (Epoch AI, 2024)↗ arxiv.org. One of the lead authors publicly confirms the bug. And the headline conclusion? It comes out reinforced: the recomputation lands on the same twenty tokens per parameter. A law audited by strangers armed with an SVG reader, corrected, and stronger for it: science, when it works, looks exactly like this.

Last twist, the most counterintuitive. Today, nearly everyone “violates” Chinchilla, on purpose. Llama 3’s small model was fed seventy-five times past its theoretical optimum[19]19. Meta AI, « Introducing Meta Llama 3 » (avril 2024)↗ ai.meta.com. An aberration? No: Chinchilla optimizes the training bill, paid once. But a model then lives through billions of requests, and each request costs in proportion to its size. Overfeeding a small model means paying more for school to pay less for life[18]18. H. Touvron et al., « LLaMA: Open and Efficient Foundation Language Models », arXiv:2302.13971 (Meta, 2023)↗ arxiv.org[20]20. N. Sardana et al., « Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws », arXiv:2401.00448 (ICML 2024)↗ arxiv.org. The law is not broken: we merely changed the question we ask of it.

So far, everything glides down. Time to confess what the line does not show.

The loss, that global error score, slides along its power law, smooth as a ski slope. Capabilities do not slide: they jump. A model fails three-digit arithmetic, fails, keeps failing, and suddenly, one click of scale later, it succeeds[21]21. J. Wei et al., « Emergent Abilities of Large Language Models », arXiv:2206.07682 (TMLR 2022)↗ arxiv.org. In 2022, a team led from Google catalogs these abilities; its first author, Jason Wei, would count one hundred and thirty-seven on his blog: abilities absent from small models, surfacing in large ones, unpredictable from the bottom of the curve[21]21. J. Wei et al., « Emergent Abilities of Large Language Models », arXiv:2206.07682 (TMLR 2022)↗ arxiv.org. The team names them “emergent abilities”.

Anthropic had coined the sharpest image as early as February 2022: general scaling is predictable like a climate; specific capabilities are unpredictable like the weather on a given day[22]22. D. Ganguli et al., « Predictability and Surprise in Large Generative Models », arXiv:2202.07785 (Anthropic, FAccT 2022)↗ arxiv.org. Same system, two horizons. With a consequence that worries safety researchers: if useful abilities show up unannounced, dangerous ones can too[22]22. D. Ganguli et al., « Predictability and Surprise in Large Generative Models », arXiv:2202.07785 (Anthropic, FAccT 2022)↗ arxiv.org.

Then, in 2023, three Stanford researchers ask the brat’s question: what if the cliff were in your ruler, not in the model[23]23. R. Schaeffer, B. Miranda, S. Koyejo, « Are Emergent Abilities of Large Language Models a Mirage? », arXiv:2304.15004 (NeurIPS 2023)↗ arxiv.org?

Their argument is disarmingly simple. Take a ten-digit addition, graded all-or-nothing: ten correct digits, or zero points. A model improving steadily, digit after digit, will look “hopeless” at the exam for years, then look brilliant overnight. The staircase lives in the grading, not in the machine. Change the metric, grant partial credit, and the studied cliffs unfold into gentle slopes. Indeed, over 92% of the cataloged emergences on BIG-Bench appear only under two metrics, one discontinuous (all-or-nothing multiple choice), the other nonlinear (exact string match)[23]23. R. Schaeffer, B. Miranda, S. Koyejo, « Are Emergent Abilities of Large Language Models a Mirage? », arXiv:2304.15004 (NeurIPS 2023)↗ arxiv.org. The paper, titled “Are Emergent Abilities of Large Language Models a Mirage?”, takes an award at NeurIPS 2023.

So, mirage or reality? The honest answer: each camp held half of the truth, and the 2023 dispute dissolved into something finer.

First, the mirage authors write it themselves, black on white: nothing in their paper claims that large models cannot display emergent abilities[23]23. R. Schaeffer, B. Miranda, S. Koyejo, « Are Emergent Abilities of Large Language Models a Mirage? », arXiv:2304.15004 (NeurIPS 2023)↗ arxiv.org. Second, for the user, practical emergence is real: when you ask a question, what you need is the right answer, not an answer “continuously less wrong”[24]24. J. Wei, « Common arguments regarding emergent abilities » (blog, mai 2023)↗ www.jasonwei.net. Going from 3% to 60% success is a real flip, however smooth the mechanics underneath. Third, later work showed that even under continuous metrics, some abilities only take off below a precise loss threshold[25]25. Z. Du et al., « Understanding Emergent Abilities of Language Models from the Loss Perspective », arXiv:2403.15796 (NeurIPS 2024)↗ arxiv.org; that the smooth curve can be read as the average of a million tiny stairs, each small skill acquired at once[26]26. E. Michaud, Z. Liu, U. Girit, M. Tegmark, « The Quantization Model of Neural Scaling », arXiv:2303.13506 (NeurIPS 2023)↗ arxiv.org; and, a 2025-2026 finding, that at strictly identical scale, the luck of initialization sometimes decides that one model “breaks through” and another does not[27]27. R. Zhao et al., « Random Scaling of Emergent Capabilities », arXiv:2502.17356 (2025, v5 février 2026)↗ arxiv.org.

Which leaves the chapter’s most uncomfortable lesson: the line predicts the global error score, not what the model will know how to do. We know how to pay our way down the curve. We do not always know what we are buying.

December 2024, Vancouver. Ilya Sutskever, OpenAI cofounder and one of the craftsmen of the scaling era, walks onto the NeurIPS stage to receive an award for his decade-old work. He uses the moment to deliver the eulogy: “Pre-training as we know it will unquestionably end”[30]30. I. Sutskever, talk « Test of Time » (seq2seq), NeurIPS, 13 décembre 2024 (vidéo)↗ www.youtube.com. His reason is physical: “data is the fossil fuel of AI”. And this sentence that snaps: “We have but one internet.”

He is not inventing the problem. Epoch AI had priced it: the effective stock of usable public human text sits around 300 trillion tokens, and training runs will have absorbed all of it somewhere between 2026 and 2032[28]28. P. Villalobos et al., « Will we run out of data? Limits of LLM scaling based on human-generated data », arXiv:2211.04325 (v2, 2024)↗ arxiv.org[29]29. Epoch AI, « Will we run out of data? » (juin 2024)↗ epoch.ai. A window, not a dated cliff, and public text only: synthetic data, private data, images and video sit outside the count[29]29. Epoch AI, « Will we run out of data? » (juin 2024)↗ epoch.ai. Still: for the first time, one of the law’s three ingredients has a visible bottom.

Doubt had already crept in, for a more trivial reason. February 2025: OpenAI ships GPT-4.5, presented as “our largest and best model for chat yet”[38]38. OpenAI, « Introducing GPT-4.5 » (27 février 2025)↗ openai.com. Sam Altman, disarmingly candid, warns the same day: “bad news: it is a giant, expensive model”, one that “won’t crush benchmarks”[39]39. S. Altman, X (27 février 2025)↗ x.com. Price: thirty times the going model. Four and a half months later it is pulled from the API catalog (it would linger a while in ChatGPT). A big, expensive, lukewarm giant, quickly shelved: the trade press writes scaling’s obituary.

It commits a category error, and it deserves naming. A scaling law is logarithmic: it promises a constant gain for every tenfold of compute, not constant wonder per product launch. A disappointing model is an industrial event; a broken law would be a curve that bends. The curve did not bend. What changed is where money buys the most progress.

Because while some were drafting the obituary, the graph’s axis had split in two.

September 2024: OpenAI presents o1, the first model trained at length to reason, and publishes two curves side by side: performance climbs with reinforcement-learning compute, and climbs too “with more time spent thinking” at answer time[32]32. OpenAI, « Learning to reason with LLMs » (12 septembre 2024)↗ openai.com. Thinking is paid in compute, and that compute follows its own scaling laws, formalized the same summer: on some problems, a model that thinks for a while beats a model fourteen times bigger answering cold[33]33. C. Snell et al., « Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters », arXiv:2408.03314 (2024)↗ arxiv.org.

The show of force lands in December 2024. On ARC-AGI, a reasoning test built to resist language models, o3 reaches 75.7% in a reasonable configuration; pushed to 172 times more compute per task, thousands of dollars apiece, it climbs to 87.5%[35]35. ARC Prize, « OpenAI o3 Breakthrough High Score on ARC-AGI-Pub » (20 décembre 2024)↗ arcprize.org. Same model, same weights: only the thinking time changed. The middle of the graph had just changed axis.

A month later, China adds its piece to the file: DeepSeek-R1 shows that reasoning can be brought out through pure reinforcement learning, with no human reasoning examples[36]36. DeepSeek-AI, « DeepSeek-R1 », arXiv:2501.12948 ; Nature 645, 633-638 (2025)↗ arxiv.org, for a final training phase costing 294,000 dollars on top of the roughly 6 million of the base model, partial figures but published, rare thing, in a peer-reviewed journal[36]36. DeepSeek-AI, « DeepSeek-R1 », arXiv:2501.12948 ; Nature 645, 633-638 (2025)↗ arxiv.org. The week it ships, Nvidia sheds nearly 600 billion dollars of market value in one session. Markets had just understood that the race has several tracks.

Since then, the clues read like a detective novel. GPT-5, released in August 2025, was trained with less compute than GPT-4.5 by Epoch’s estimates: progress had gone shopping elsewhere, in reasoning and refinement[40]40. OpenAI, « Introducing GPT-5 » (7 août 2025)↗ openai.com[41]41. Epoch AI, « Why GPT-5 used less training compute than GPT-4.5 (but GPT-6 probably won’t) » (2025)↗ epoch.ai. Three months later, Google ships Gemini 3, a performance jump carried by pre-training, and its co-lead savors it: “Contra the popular belief that scaling is over [...] No walls in sight!”[42]42. O. Vinyals, X (18 novembre 2025)↗ x.com. And Epoch’s analysts compute that the reasoning-compute surge, ten times more every few months, was a catch-up: it will fold back into the general cruising speed, about four times a year[37]37. Epoch AI, « How far can reasoning models scale? » (mai 2025)↗ epoch.ai.

The mid-2026 state of play, then: giant pre-training is pausing more than stopping, reinforcement and reflection absorb the growth, and practitioners sum it up without lyricism: the scaling laws still work, the low-hanging fruit is mostly picked, and one now scales on every axis at once[54]54. L. Fridman, « State of AI in 2026 » (#490, avec N. Lambert et S. Raschka, transcript)↗ lexfridman.com.

That leaves the question everyone asks, and it deserves to be asked properly: how many clicks of zoom separate us from the end of the axis?

Epoch AI’s analysts ran the exercise constraint by constraint[43]43. Epoch AI, « Can AI scaling continue through 2030? » (août 2024)↗ epoch.ai. Their conclusion: today’s growth, four times more compute per year, is technically sustainable until about 2030. At the end: training runs around 2×10^29 operations, ten thousand times GPT-4. The distance from GPT-2 to GPT-4, one more time. Four clicks above GPT-4; fewer than three above today’s records.

What bites first, in order: electricity, then chip manufacturing, then data, then a constraint of pure machine physics, latency[43]43. Epoch AI, « Can AI scaling continue through 2030? » (août 2024)↗ epoch.ai. Electricity supplies the useful vertigo: the largest training runs already draw over one hundred megawatts; by 2030, the largest run would demand between 4 and 16 gigawatts[44]44. Epoch AI, « How much power will frontier AI training demand in 2030? » (août 2025)↗ epoch.ai, the output of four to sixteen nuclear reactors[45]45. U.S. Department of Energy, « How much power does a nuclear reactor produce? »↗ www.energy.gov.

Training Grok 4, the documented record to date, cost about half a billion dollars and consumed 310 gigawatt-hours[46]46. Epoch AI, « What did it take to train Grok 4? » (septembre 2025)↗ epoch.ai; the billion-dollar training run is expected around 2027[14]14. B. Cottier et al., « The rising costs of training frontier AI models », arXiv:2405.21015 (Epoch AI, 2024)↗ arxiv.org. Note in passing that promises run faster than invoices: the “5 to 10 billion in 2025-2026” announced by Dario Amodei in 2024[47]47. D. Amodei, entretien avec Ezra Klein, The New York Times (12 avril 2024)↗ www.nytimes.com had, by mid-2026, materialized on no confirmed run.

As for the captains of the race, quote them with their dates; the landscape shifts every quarter. In late 2024, Sam Altman settled it in four words: “there is no wall”[49]49. S. Altman, « there is no wall », X (14 novembre 2024)↗ x.com, before raising his own observation in 2025, a model’s intelligence grows “roughly [as] the log of the resources”[50]50. S. Altman, « Three Observations » (blog, 9 février 2025)↗ blog.samaltman.com, and concluding in 2026: “Betting against LLMs scaling at this point feels quite misguided to me”[51]51. S. Altman, propos sur le scaling des LLM (juin 2026, rapportés par The Decoder)↗ the-decoder.com. Dario Amodei holds that pre-training “is continuing to give us gains” and sees reinforcement following the same laws[48]48. D. Amodei, entretien avec Dwarkesh Patel (13 février 2026)↗ www.dwarkesh.com. Demis Hassabis pushes scaling “to the maximum” while betting one or two more breakthroughs are still needed, and places his AGI around 2030[52]52. D. Hassabis, entretien Axios (26 mai 2026)↗ www.axios.com. Sundar Pichai, the most cautious, had warned as early as 2024: “the low-hanging fruit is gone”, the hill is steeper, but “I don’t fully subscribe to the wall notion”[53]53. S. Pichai, sommet DealBook du New York Times (4 décembre 2024, via CNBC)↗ www.cnbc.com.

None of them can invoke a theorem. The most elegant piece in the file is that the founding paper had foreseen its own expiry: as early as 2020, Kaplan noted that his trends must eventually level off, “since natural language has non-zero entropy”, and even priced, with enormous error bars, the horizon where his curves would turn incoherent[9]9. J. Kaplan, S. McCandlish et al., « Scaling Laws for Neural Language Models », arXiv:2001.08361 (OpenAI, 2020)↗ arxiv.org. The modern, serious version of that intuition is Epoch’s window: somewhere between here and 2032, the regime that carried this story will have to reinvent itself, through synthetic data, multimodality, reasoning, or an idea that does not exist yet[29]29. Epoch AI, « Will we run out of data? » (juin 2024)↗ epoch.ai[31]31. I. Sutskever, entretien avec Dwarkesh Patel (25 novembre 2025)↗ www.dwarkesh.com.

There is, finally, a scenario where the question would change nature: the one where models become good at improving models. The line would then start feeding on itself. That is another story, and we have already told it: our story on recursive self-improvement.

No one knows whether the line will hold. We only know that as you read, somewhere between Texas and Memphis, turbines are warming up, racks are lining up, and an engineer has just written a number on a slip of paper.