When Bigger Doesn't Mean Smarter: What Scaling Laws Tell Us About the Future of AI

The article analyzes why scaling laws in AI fail to deliver exponential improvements in model performance despite exponentially growing investment. Using the examples of GPT-4.5 and GPT-5, it shows that a power law with a small exponent means drastically diminishing marginal returns — doubling compute yields only a three-percent improvement. The future of AI therefore lies in algorithmic innovation, inference-time compute, and specialization (MoE), not in merely scaling models up.

Analysis — February 2026

The scaling laws promised that all you had to do was add more data, parameters, and computing power — and the models would get smarter. By 2026, we know it is not that simple. But the story of diminishing marginal returns is more complicated than the headlines suggest.

In February 2025, OpenAI released a model that was supposed to be GPT-5. It was significantly larger than GPT-4, trained with substantially more compute. The result was such a disappointment that the company named it GPT-4.5. When unveiling it, Sam Altman described it as a "giant, expensive model" that "won't crush benchmarks." When the creator of the world's most ambitious AI project introduces its new product this way, something has changed.

Half a year later, in August 2025, the real GPT-5 arrived. On Polymarket, a prediction market, OpenAI had roughly an eighty percent chance of holding the leading AI model going into the presentation. After the presentation, that figure collapsed to about fourteen percent. Google climbed to around eighty-five. This is not a reaction to a bad model — GPT-5 is objectively more capable than anything before it. It is a reaction to disappointed expectations. The leap from GPT-3 to GPT-4 transformed the industry. The leap from GPT-4 to GPT-5 did not.

This is the story of why — and what it means for the future of artificial intelligence.

In 2020, a team at OpenAI led by Jared Kaplan published a paper that became one of the most cited in the history of AI research. They found that the performance of language models (measured as test loss — a measure of inaccuracy) improves predictably as three quantities grow: the number of model parameters, the volume of training data, and the amount of compute used. The relationship takes the form of a power law — a smooth, predictable curve that appears as a straight line on a logarithmic chart.

Two years later, a DeepMind team led by Jordan Hoffmann published a study known as the "Chinchilla paper," which refined the ratios: to make optimal use of a compute budget, you must scale parameters and data simultaneously. The Chinchilla model, with 70 billion parameters, outperformed the 280-billion-parameter Gopher because it was trained on four times more data. The lesson: it is not just about size, it is about balance.

These two papers — Kaplan and Chinchilla — became the compass of the entire industry. If performance grows predictably with compute, then all you need to do is invest more money in bigger clusters and progress is guaranteed. Project Stargate (500 billion dollars, 400,000 GPUs) is the logical consequence of this belief.

Toby Ord, a philosopher at Oxford and author of the book The Precipice, drew attention in a series of 2025 essays — beginning with the text "The Scaling Paradox" — to something most commentators had overlooked. Those smooth, promising-looking charts in the original papers have both axes logarithmic. On a normal chart, the same data would look dramatically different.

A concrete example from Kaplan's paper: halving the test loss (from 6 to 3) requires a millionfold increase in compute. That is not a promising curve — those are extreme diminishing returns. In Ord's words: the model's accuracy is "extraordinarily insensitive" to scaling resources. The exponent in the power law is very small (around 0.05 for compute). This means that doubling compute yields an improvement in accuracy of just three percent.

In computer science, exponential resource consumption for a linear gain is usually taken as a signal that the problem is intractable for the given approach. Ord argues that this is exactly what the scaling laws tell us — they merely present it on charts that visually mask the fact.

Theory predicts diminishing returns. What do the data from the past two years say?

GPT-4.5, unveiled in February 2025, was a "clean" test of scaling — a bigger model, more data, more compute, the same architecture. The improvement over GPT-4 was measurable, but not transformative. Sam Altman admitted this publicly.

Meta delayed Llama 4 Behemoth because, according to internal sources, it was not improving dramatically enough. Google quietly abandoned some ambitious scaling projects. Bloomberg, The Information, and Reuters independently reported that the leading labs were hitting a ceiling.

GPT-5, released in August 2025, both confirmed and refuted this trend at the same time. On benchmarks it is impressive — 74.9 percent on SWE-bench Verified (solving real GitHub issues), 94.6 percent on AIME 2025 without tools (in the Pro configuration with the Python interpreter enabled, even 100 percent), a dramatic jump on ARC-AGI-2. But the leap was nowhere near as large as from GPT-3 to GPT-4. And GPT-5.2, from December 2025, although it achieved perfect scores on some benchmarks, triggered a wave of disappointment on community forums and prediction markets.

The key paradox: the models are objectively better. But the improvement is slowing down. And users, accustomed to exponential leaps, perceive incremental progress as stagnation.

Scaling is not running into one wall, but three — and each requires a different solution.

The data wall. Epoch AI estimates the total volume of publicly available high-quality text on the internet at roughly 300 trillion tokens. At the current pace of over-training (more than 200 tokens per parameter), high-quality training data will run out sometime between 2026 and 2032. Ilya Sutskever summed it up at NeurIPS 2024: "Pre-training as we know it will end." Synthetic data — texts generated by other models — is a potential solution, but it carries the risk of "model collapse" — degeneration, where a model trained on its own outputs gradually loses diversity.

The energy wall. Project Stargate will consume on the order of gigawatts. Further scaling requires new power plants — and those are not built in months. The cost of training frontier models is rising from millions to billions of dollars. At a certain point, the investment becomes economically irrational: if doubling compute yields a three percent improvement but costs a billion dollars, the return on investment plummets.

The benchmark wall. Benchmarks are saturating. MMLU, until recently the standard yardstick, was practically solved in 2025 — frontier models achieve scores above 90 percent. New benchmarks (MMLU-Pro, GPQA, ARC-AGI-2) are harder, but even those are being solved faster than their authors expected. ARC Prize announced it was preparing ARC-AGI-3 for early 2026, because ARC-AGI-2 will soon be saturated. When benchmarks are solved faster than they can be created, the question is whether they measure genuine intelligence or the ability to optimize for a specific test format.

But the story does not end with the verdict that "scaling has failed." More precisely: the scaling of pre-training is running into limits. But new dimensions are opening up.

Inference-time compute — a paradigm introduced by OpenAI's o1 and o3 models — shifts computing power from training to inference. The model "thinks" longer about its answer, breaks the problem into steps, and verifies its own reasoning. This is a qualitatively different approach from mere scaling: instead of a bigger brain, you give the existing brain more time to think.

The results are remarkable. OpenAI's o3 achieved a score on AIME that would place it among the best human mathematicians. On Codeforces it outperformed 99.8 percent of human programmers. But here too diminishing returns appear: the chart of performance versus inference compute exhibits a logarithmic shape. More thinking helps, but each additional minute of thinking helps less.

Post-training optimization — RLHF, DPO, constitutional AI, and other techniques that improve a finished model without new pre-training — has become the main path to improvement. GPT-5 is to a large extent a product of post-training: the same (or similar) base as GPT-4.5, but significantly improved through reinforcement learning and human feedback. The team at Epoch AI found that algorithmic efficiency doubles roughly every eight months — meaning that for the same performance you need half the compute.

Specialization and mixture-of-experts (MoE) — instead of one giant model, a system of smaller expert modules, of which only the relevant subset is activated for each query. DeepSeek demonstrated that an MoE architecture can achieve performance comparable to much larger dense models at a fraction of the cost.

The scaling laws were not refuted. They were refined. They tell us exactly what they always told us — we just read them wrong. A power law with an exponent of 0.05 never promised exponential progress. It promised predictable, but ever-slower progress for ever-larger investments.

For the industry, this means the end of the era when it was enough to throw more GPUs at a problem. Future progress will depend on three factors: algorithmic innovation (new architectures, better training methods), more efficient use of inference (thinking instead of memory), and better specialization (the right model for the right task).

For users, it means calibrating expectations. Leaps like GPT-3→GPT-4 were an anomaly, not the norm. Future improvements will be incremental, but cumulatively transformative — much like progress in hardware after the end of Dennard scaling. Processors did not speed up exponentially, but the combination of more cores, better architectures, and software optimization brought overall progress that the user perceives as smooth.

For investors, it means a question: if doubling compute yields a three percent improvement, is another 500 billion dollars a rational investment? The answer depends on whether algorithmic efficiency and new paradigms (inference scaling, MoE, agentic AI) can shift this curve. So far, yes — but there is no guarantee.

In January 2020, the most expensive training run for an AI model cost on the order of millions of dollars. In January 2026, it runs into the billions. The improvement over that period? Measurable, in some areas dramatic, but not proportional to the investment.

Toby Ord sums it up elegantly: imagine that reducing the error rate from ten to five percent costs a million dollars. Reducing it from five to 2.5 percent costs a billion. From 2.5 to 1.25 percent, a trillion. Each further halving costs a thousandfold more. That is the essence of a power law with a small exponent.

OpenAI believes that new paradigms will raise this exponent. Skeptics like Yann LeCun argue that the transformer architecture has fundamental limits and that we need a new approach. The empirical data from 2025 and early 2026 do not yet give an unambiguous answer. GPT-5 and GPT-5.2 are better, but not by as much as they cost. Inference-time scaling brings a new dimension, but it too exhibits diminishing returns.

What we know for sure: the era when it was enough to scale and the result came on its own is over. The era in which you have to be smarter — not just bigger — is only just beginning.

This article follows on from the text "Moore's Law in the Age of Artificial Intelligence" (Médium.cz, February 2026), which analyzes the pace of AI growth from the perspective of inputs (compute, algorithms, investment). This text focuses on outputs — why exponential inputs do not bring exponential improvements in capabilities.

Key sources: Kaplan et al. "Scaling Laws for Neural Language Models" (arXiv:2001.08361, 2020); Hoffmann et al. "Training Compute-Optimal Large Language Models" (Chinchilla, arXiv:2203.15556, 2022); Toby Ord "The Scaling Paradox" (Forethought Foundation / tobyord.com, 2025); Epoch AI "Algorithmic Progress in Language Models" (2024); Sutskever, keynote NeurIPS 2024; HEC Paris "AI Beyond the Scaling Laws" (2026); reporting by Bloomberg, The Information, Reuters, TechCrunch (2024–2025); OpenAI GPT-5 / GPT-5.2 benchmarks and reactions (August–December 2025).

Methodological note

The conception, structure, and editorial line of the article are the work of the author, who prepared the content outline, set out the key theses, and directed the entire creative process. Generative AI (Claude, Anthropic) was used as a technical tool for research, fact-checking, and fleshing out the author's draft.

The author edited the outputs throughout, verified the key findings, and approved the final wording. No part of the text was published without human review. All factual data were verified against the publicly available sources cited in the text.

The procedure complies with the requirements of Article 50 of EU Regulation 2024/1689 (the AI Act) on transparency of AI-generated content. #poweredByAI