Who Reviews the Code After the AI — and Why a Weaker Model Won't Do

Article draws on peer-reviewed studies and preprints to show that language models systematically overlook errors in their own output (64.5% of cases, per one paper) and cannot reliably self-correct without external feedback. What matters, therefore, is not the size of the reviewing model but its independence from the code's author. It also examines the strongest counterargument — verification asymmetry, which holds mainly for tasks with objective ground truth — and maps the interests of the players involved, from academia to code-review tool vendors. Practical takeaway: stop asking which model to deploy and start asking how many independent layers (deterministic tools, a different strong model, a human) to put between generated code and production — and measure error-catch rates on your own codebase.

Give a language model its own solution containing a bug and ask whether it is all right. According to a July 2025 preprint (Tsui, arXiv 2507.02778), the model overlooks such a self-authored error in roughly 64.5 % of cases and waves the faulty output through as correct. But then comes a small detail that illuminates the whole topic. As soon as the very same passage is presented to the model as someone else's input, the blindness disappears; and the minimal nudge "Wait" alone reduced it by 89.3 % in the test. Same solution, same model, same bug. Only the angle of view changed.

This scene is not an academic curiosity. Models today write production code, not toy exercises, and the question of who or what will check that code after them is moving from conference slides into real CI pipelines. Meanwhile a convenient answer has taken root in teams: let's put a smaller, cheaper model on review, because watching is easier than creating. That reasoning contains a grain of truth and a large chunk of accounting.

The reliability of the check is not primarily determined by the size of the reviewing model, but by its independence from whoever wrote the code. The thesis "a weaker model is enough for review" holds for easy, mechanically verifiable bugs and is largely an artifact of cost and throughput, not of detection quality. For subtle and critical bugs — architectural, security-related, logical deep in the domain — a weak reviewer systematically misses what matters most. And the stronger the generator, the harder its bugs are to find.

Let's start with what is best documented. That a model favors its own output was shown by a peer-reviewed paper from NeurIPS 2024 (Panickssery, Bowman, Feng; arXiv 2404.13076): across thousands of texts it found a linear relationship between how well a model recognizes its own text and how strongly it then favors it. Both GPT-4 and Llama "recognize" their own output and rate it more favorably. More important than the effect itself, however, is its cause. According to an analysis from late 2024 (Wataoka et al., arXiv 2410.21819), this is not ego but perplexity — the model prefers texts that feel familiar to it, because it would have written them similarly itself. From this follows an unpleasant practical consequence: this bias reliably disappears only when the check is performed by a different model, not when you swap the prompt for the same model.

The second pillar is even harder. Three independent peer-reviewed studies agree that a model cannot reliably correct its own reasoning without external feedback. Google DeepMind, in work for ICLR 2024 (arXiv 2310.01798), showed that after an attempt at self-correction the models' performance sometimes even got worse, and that multi-agent "debate" does not beat simple agreement across several runs of the same model. A team from Cambridge and Google Research (ACL Findings 2024, arXiv 2311.08516) then localized the bottleneck: the problem is not fixing the error, but finding it. A small classifier trained on out-of-domain data outperformed prompting a large model at error finding — which is itself a strong argument that detection is not primarily a question of size. And a critical review from Penn State (TACL 2024, arXiv 2406.01297) concluded that none of the work to date has demonstrated successful self-correction with feedback from a prompted model alone; it works only where a reliable external signal exists.

When the same thing is measured directly on error detection, the picture does not improve. The ReaLMistake benchmark (Kamoi et al., COLM 2024, arXiv 2404.03602) found that both GPT-4 and Claude 3 look for errors in model responses with very low recall, markedly worse than humans, and that neither self-consistency nor majority voting lifts detection. Naively scaling a weak model with "more attempts" therefore does not help.

That brings us to the core of the dispute about size. If finding errors is hard, shouldn't a stronger model handle it better? Partly yes — but with a catch that works against weak reviewers. The paper "Variation in Verification" (arXiv 2509.17995) showed that errors from weaker generators are easier to detect than errors from strong ones. For the verifier Qwen2.5-72B, the ability to catch a faulty solution fell from 0.68 on tasks solved by Llama-3.1-8B to a mere 0.17 on tasks from the stronger Qwen3-32B — measured on mathematical problems with an unambiguously verifiable result. Translated into practice, and assuming the same mechanism holds for code: the better the model that wrote the solution, the more you need a strong — and above all a different — model to check it. In parallel, the paper "Learning to Self-Verify" (arXiv 2602.07594) documented that improvements in generation do not lead to a corresponding improvement in self-verification; that asymmetry persists.

For code the evidence is thinner and it diverges. A May 2026 preprint (Reddy et al., arXiv 2605.21537), whose authors work at AWS but declare the research independent of their employer and free of proprietary data, went through 1,980 code-modernization calls across eleven production models. Semantic traps, which arose in 7 % of cases on innocuous code, jumped to 39.7 % on risky code, and worse: the model silently approved 31.7 % of its own semantic errors. For one model it was 100 %. Drift, moreover, did not track model size cleanly, fluctuating from 5.6 to 46.7 %, which points more to a structural problem than to a lack of capability.

A peer-reviewed 2026 study (Dolcetti et al., Springer, DOI 10.1007/s44163-026-01009-5) found no favoritism toward a model's own output in code evaluation. According to it, models are simply bad at bug detection across the board, regardless of authorship — F1 below 80 % for correctness, below 30 % for vulnerabilities. A possible reconciliation of the two findings: it is not "ego", but the combination of models generally overlooking subtle bugs plus their own code additionally feeling familiar to them. Work from the University of Virginia (arXiv 2504.03846) adds a disquieting detail — stronger models are indeed better judges, but their harmful favoritism worsens precisely when they themselves are wrong. A stronger model is worse at recognizing that it is mistaken.

Now the strongest version of the opposing view, because the topic deserves it. The asymmetry of verification is real: for a large class of tasks, checking is cheaper than creating, and empirically that fits. On a hundred tasks from LiveCodeBench, verification via property-based tests reached 87 % against 63 % for generation alone (PGS, arXiv 2506.18315, on the DeepSeek-R1-32B model) — so this was not pure self-review, but structured feedback from tests. Weak verifiers, moreover, do work in aggregate: Stanford's Weaver system (arXiv 2506.18203) closed the gap between generation and verification by 14.5 % by combining many weak signals, and the multiple agentic verification method (arXiv 2502.20379) delivered gains even where weak verifiers judged strong generators. Add the economics: review runs on every pull request many times more often than generation, so the pressure toward a smaller and faster model is rational. That reasoning is valid — but only under one condition. All the results cited come from tasks with objective ground truth, typically from mathematics, where summing many weak signals works and where an unambiguous "correct" exists. Open-ended detection of a subtle bug in an unfamiliar codebase has no such truth. A weak reviewer is therefore defensible as a first filter for easy and verifiable bugs, not as the last word over security- or architecture-sensitive code. And precisely in brownfield code, where integration and architectural errors dominate, the weak model fails most.

None of this, however, implies "deploy the biggest model". It implies something more useful. A perfect model does not exist, so it makes no sense to look for the one right reviewer; it makes sense to stack layers with as little overlap in their blind spots as possible. The first layer is deterministic tools: a compiler or type checker, a linter, sanitizers, tests, static analysis. In the language of models they are "dumb", but their failure mechanism differs from a model's, and that is exactly what is valuable. A comparison of SAST tools and models (Zhou et al., arXiv 2407.16235) showed the complementarity of the two worlds: SAST has a low catch rate but few false alarms, while models uncover 90 to 100 % of vulnerabilities at the cost of a high false-positive rate. When the two are combined, the weaknesses are damped — the hybrid SAST-Genius (arXiv 2509.15433) reduced Semgrep's false alarms from 225 to 20. The second layer is a different, ideally strong model. And the third layer remains the human, because intent, business context, exploitability and maintainability are invisible to a model without external context. Here, though, lurks a trap called automation bias: a meta-analysis of clinical decision support systems (Goddard, Roudsari, Wyatt; JAMIA 2012, PMC3240751) found that when a tool gives bad advice, it raises the probability of an erroneous human decision by 26 % (risk ratio 1.26; confidence interval 1.11 to 1.44). The reviewer must therefore actively exercise the running code, not merely read the diff and nod along.

And because many voices speak on this topic, it pays to know who has what interest. The most sober conclusions come from academic teams without a product of their own — from Penn State, Cambridge, the University of Virginia, or the Italian-British group around Dolcetti. Makers of frontier models have a dual motivation. Sometimes they admit a limit, as when DeepMind openly wrote that self-correction does not work; such a conclusion runs against their own interest, and is thereby more credible. At other times they want to show that their checking is good enough. Characteristic is OpenAI's text on code verification at scale (alignment.openai.com), which is not peer-reviewed and comes from a company selling both models and a review agent: it claims that its checking remains comparably effective even on its own code, yet itself admits that a clean direct measurement of this claim is missing and that recall is persistently higher on human-written code. Makers of code review tools — Greptile, CodeRabbit, Qodo, cubic and others — have an interest in showing that automated review works, or that a cheaper model suffices, which lowers their costs. Their own benchmarks are mutually inconsistent, and numbers like "three to five times more bugs found with a different model" belong in the category of marketing, not science; one vendor measured self-favoritism at under two percentage points in its own cross-test. It is worth being especially wary of preprints on arXiv: they are not peer-reviewed and a large share come precisely from commercial labs with a stake in the result.

It is worth adding where higher model capability shifts the risk. An October 2025 report from the company Sonar, built on more than 4,400 programming tasks in Java, measured a 6.3 % higher test pass rate for Claude Sonnet 4 against the older version — but its bugs were 93 % more often of the highest severity. "Code smells" made up over 90 % of all findings for every tested model. Sonar sells code quality tools, so it has an interest in showing that AI-written code needs checking; the direction of that bias is worth keeping in mind. Even so, the finding agrees with a mechanism we see elsewhere too: higher reasoning does not erase the error rate, it merely shifts it toward subtler and more dangerous traps.

What to take from this in practice? That you should not look for the answer in vendor benchmarks, but in your own data. Seed a controlled set of bugs into your own codebase and measure how many of them your reviewer finds and how many faulty solutions it lets through. If the ability to catch a bug on your type of code falls below a threshold that no reviewer should go under, then that reviewer is too weak for critical code — regardless of what its vendor promises. The real question, after all, is not which model to put on review, but how many independent layers you are willing to build between generated code and production. One reviewer, even a frontier one, is a monoculture. And monocultures fall to the first disease they have no antibody for.

This text was written on the basis of structured desk research and underwent source verification at the primary level: the existence of all cited works, their identifiers (arXiv, DOI, PMC), authorship and place of publication were verified against arXiv, Springer and PubMed Central. Some specific numbers from the 2026 preprints and from full texts (the exact extreme values of drift in Reddy, the risk ratio value from the JAMIA meta-analysis, the F1 thresholds in Dolcetti) match the abstracts and snippets, but before being cited in an expert report or used further they deserve a check against the full text. Vendor materials carry the lowest certainty. Confidence levels: HIGH (peer-reviewed primary source), MEDIUM (preprint or a single independent study), LOW (marketing-laden or unverified claim).

Three honest reservations about the topic itself. First: a clean controlled experiment of "the same model versus a different model on identical code" is essentially missing from the peer-reviewed literature. The mechanism behind the advantage of a different model (decorrelated blind spots) is documented at the level of principle; its magnitude specifically in code review has so far not been independently measured by anyone. Second: most of the strongest evidence comes from mathematics and reasoning, not from production code; the transfer to code is a reasonable but unproven extrapolation — wherever a number measured on a mathematical task appears in the text, this is stated explicitly. Third: effect sizes depend strongly on task difficulty, language and error type — no single number can be generalized.

Transparency of creation:

The concept, structure and editorial line of the article are the work of the author, who prepared the content outline, established the key theses and directed the entire creative process. Generative AI (Claude, Anthropic) was used as a tool for research, locating primary sources, and the drafting elaboration of the author's content outline.

The author continuously edited the outputs, verified the key findings and approved the final wording. No part of the text was published without human review. All factual data were verified against the publicly available sources cited in the text.

This procedure complies with the transparency requirements for AI-generated content under Art. 50 of EU Regulation 2024/1689 (AI Act). #poweredByAI