Will GPT-5 Pass the Czech Bar Exam?

The article examines the capabilities of large language models (GPT-5, Gemini 3 Pro, Claude 4.5 Sonnet) in the domain of Czech law and their potential to pass the Czech bar exam. While the WAIR system—with access to Czech legal databases—was the only one to pass, standalone models have yet to reach the required 85% threshold. The author highlights the absence of systematic testing of current models on Czech law and identifies the key barriers: a shortage of Czech training data, the specifics of the continental legal system, and the lack of an open benchmark.

In the autumn of 2025, the Prague attorney Pavol Kehl filed a complaint with the Constitutional Court on behalf of a client who had been unsuccessful before the administrative courts with actions challenging the refusal of a temporary residence permit. Formally, the complaint appeared flawless. The trouble was that of the twelve decisions of the Constitutional Court and the European Court of Human Rights cited in the text, a substantial portion did not exist at all and the rest were grossly misinterpreted. In drafting it the attorney had used an artificial-intelligence tool and had not adequately verified the result. On 1 December 2025, the presiding judge Tomáš Langášek imposed a disciplinary fine of CZK 25,000 on him for grossly obstructing the conduct of the proceedings. The Czech Bar Association looked into the case.

At the other end of the same spectrum stands the WAIR system, which in test conditions was the only artificial-intelligence system to demonstrably pass the Czech bar examination. Between these two extremes unfolds a story that is changing rapidly in 2026: the most advanced models, such as GPT-5, Gemini 3 Pro, or Claude 4.5 Sonnet, are delivering performance on international legal benchmarks that no one would have dreamed of two years ago. Yet no one has so far systematically turned them loose on Czech law.

This article analyses how far the capabilities of the latest large language models have advanced in legal reasoning, why Czech law remains a particular challenge, and what this means for attorneys, courts, and regulators.

The only systematic test of artificial-intelligence performance on the Czech bar examination remains the one published on 28 March 2024 by Jiří Nečas and Dalibor Kovář of the firm HAVEL & PARTNERS. They tested eight models on sets drawn from a bank of 1,840 official questions of the Czech Bar Association — five rounds of a hundred questions each, i.e. 500 questions per model. The correct answer is always one of three offered options, and the threshold for passing is 85%. The results were unequivocal:

WAIR (GPT-4 + retrieval-augmented generation + Czech legal databases) — above 85% in all 5 rounds. The only one to pass the examination.

Claude-3-Opus — the best of the standalone models, below 85%. Did not pass.

GPT-4-Turbo — just behind Claude-3-Opus. Did not pass.

Gemini-1.0-Pro — at some distance. Did not pass.

GPT-3.5-Turbo — barely over 50%. Did not pass.

The test revealed that criminal law was the hardest area for most models (the exception being Gemini-1.0-Pro, for which it was, on the contrary, the strongest discipline). A fundamental limitation: only the written multiple-choice part was tested, not written argumentation nor the oral examination.

Since March 2024, however, the landscape of large language models has changed dramatically. Models that did not exist back then are today achieving performance on international legal benchmarks that, with high probability, would have crossed the 85% line in the HAVEL & PARTNERS test. But — no one has verified this.

On the LegalBench platform — 162 tasks of American law covering rule identification, application of the law, and legal interpretation — Gemini 3 Pro leads as of February 2026 with an accuracy of 87.04%. It is followed by Gemini 3 Flash (86.86%), GPT-5 (86.02%), and GPT-5.1 (85.68%). The older GPT-4o reached roughly 78%, o1 Preview around 80%. The increase of 7–9 percentage points in under two years illustrates the pace of progress.

Important, however, is the authors' own caveat: model performance differs dramatically across tasks. A model that excels at identifying a legal rule may fail at applying it to a specific set of facts. Legal reasoning is not a monolithic competence.

The LEXam benchmark (ETH Zurich, the University of Zurich, and other institutions; May 2025, updated on an ongoing basis) represents the most demanding examination to date — 4,886 questions from 340 law exams in English and German, including 2,841 open-ended questions requiring structured legal argumentation.

The results are more sobering than those of the multiple-choice tests. In the original May 2025 version of the paper, the highest overall mean score was achieved by Gemini-2.5-Pro (82.2), followed by Claude-3.7-Sonnet. Models with extended reasoning (Gemini-2.5-Pro, Claude-3.7-Sonnet, o3-mini) substantially outperform conventional large language models, but even they visibly struggle with multi-step legal reasoning: identifying the problem, recalling the relevant rule, and applying it to the facts.

A key finding of LEXam: increasing the number of choices in multiple-choice questions consistently lowers model accuracy. With four choices, the most advanced models reach 80–90%; with thirty-two choices, they drop below 60%. This is relevant to the Czech examination, which, although it has only three choices, contains subtle differences in wording that require precise knowledge of Czech law.

On the Brazilian bar examination (oab-bench, April 2025, 105 questions from the written part), Claude 3.5 Sonnet passed all 21 examination sittings with an average score of 7.93 out of 10. The results of the other models were weaker. The study showed that Claude excelled in constitutional and criminal law.

International benchmarks show impressive progress, but transferring them directly to Czech law is methodologically problematic. There are three structural barriers that set the Czech bar examination apart from English-language tests.

A study by Arnett and Bergen (November 2024) demonstrated that the performance gap for morphologically rich languages correlates primarily with the volume of training data, not with linguistic complexity. Cross-sectional measurements of model performance on non-English, including the Czech benchmark BenCzechMark (50 tasks, 90% natively Czech content, published in the Transactions of the ACL, 2025), show that models perform markedly worse on Czech tasks than on comparable English ones. The specialist literature typically puts estimates of the gap in the range of 10–30%, depending on the type of task and the model.

BenCzechMark moreover showed that the overall score on multiple-choice questions need not correspond to the quality of generated Czech text — a model may answer Czech questions correctly yet generate poor-quality Czech text.

The cause is not an inherent difficulty of Czech, but the imbalance in training data. A solution therefore exists — but it requires investment in Czech corpora.

Continental legal systems, to which Czech law belongs, pose different challenges for artificial intelligence than the Anglo-American common-law system. General clauses — concepts such as "good morals" or "reasonableness" — are deliberately indeterminate and cannot be resolved by mere information retrieval. Gaps in the law require reasoning by analogy. The hierarchy of norms (lex specialis, lex posterior) presupposes an understanding of systemic relationships. Case law, which is non-binding but persuasive, operates in the grey zone between law and practice. And shifting burdens of proof require contextual assessment that exceeds the capabilities of mere pattern recognition.

The American MBE test, with its common-law logic, is more natural terrain for large language models: a rich body of case law offers patterns that are easy to learn. Czech codified law requires systematic interpretation, not the search for precedents.

The Czech bar examination requires an 85% pass rate in the written multiple-choice part. The American UBE in effect has a lower threshold. The Japanese examination requires 93 out of 175 points (53%). This difference means that even a model achieving a solid 78–80% on the Czech test would not pass.

The key finding of the HAVEL & PARTNERS test is not the performance of individual models, but the chasm between a standalone large language model and a system with access to Czech legal resources. WAIR — combining GPT-4 agents with the technique of retrieval-augmented generation (RAG) and Czech legal databases — passed the examination in all five rounds. The best standalone model (Claude-3-Opus) did not.

This finding is confirmed internationally as well. On the Japanese bar examination, a fine-tuned model with a self-correction mechanism succeeded, not the base GPT-4. On the LEXam benchmark, the authors note that models with extended reasoning and access to relevant resources substantially outperform models without context.

For Czech law this has a direct implication: what matters is not how "smart" the model itself is, but whether it has access to up-to-date, high-quality Czech legal resources. WAIR verified this empirically. With the most advanced models of 2026 (GPT-5, Gemini 3 Pro, Claude 4.5 Sonnet), a retrieval-augmented system with a Czech legal database would, with high probability, achieve even significantly better results — but this is an unverified hypothesis.

The Kehl case of December 2025 illustrates the risk of the uncritical use of artificial intelligence in law. But it equally shows that the problem is not the technology itself, but the absence of processes for its responsible deployment.

In September 2023, the Czech Bar Association issued a position statement: artificial intelligence cannot provide legal services, it may be used only as a tool, and the attorney bears full responsibility. In January 2026 it translated the CCBE Guidelines on the use of generative artificial intelligence by lawyers, and the Czech Bar Association's Artificial Intelligence Section, led by Barbora Vlachová, is preparing an updated strategy.

The innovation current is represented by the Bar Association for AI (founded in July 2024) and products such as WAIR (HAVEL & PARTNERS), Lawrence AI (a project by students of the Faculty of Law of Charles University), and AI Advokát (the firm Petráš Rezek).

At the academic level, a key role is played by the OpenEuroLLM project coordinated by Charles University (Prof. Jan Hajič, ÚFAL MFF UK) with a budget of EUR 34 million — it targets open, multilingual large language models for European languages, compatible with the EU Artificial Intelligence Act. These models could be fine-tuned for Czech law without the concerns about breaching attorney confidentiality that accompany the use of closed American models.

The biggest problem in the entire debate is not whether artificial intelligence "knows" Czech law — but that no one is systematically measuring it. The HAVEL & PARTNERS test is valuable, but proprietary and limited to the multiple-choice format with models from March 2024. The Czech benchmarks BenCzechMark and CzechBench contain no specialized legal tasks. There is no open, reproducible benchmark of Czech legal AI.

This means that the debate about artificial intelligence in Czech law still rests — as the authors of the HAVEL & PARTNERS test aptly noted — more on anecdotal experience than on hard data.

What would such a benchmark require? First, extending the multiple-choice tests to the current most advanced models (GPT-5, Gemini 3 Pro, Claude 4.5 Sonnet, o3). Second, the addition of a written argumentation part — LEXam showed that it is precisely open-ended questions that are the critical differentiating factor. Third, testing systems with retrieval-augmented generation and Czech legal databases alongside standalone models. And fourth, an open methodology enabling reproduction and updating.

On the basis of the available data, three qualified estimates can be formulated.

First, the most advanced models of 2026, with retrieval-augmented generation and a high-quality Czech legal database, would with high probability cross the 85% threshold on the multiple-choice part of the Czech bar examination. Extrapolation from LegalBench (87% for Gemini 3 Pro on English law) minus the estimated language penalty (10–15% for Czech) gives a rough estimate of around 74–78% for a standalone model. With a retrieval-augmented system, the result could be above 90%. But this remains an unverified projection — not a prediction.

Second, the written and oral parts of the examination remain a markedly tougher nut to crack. LEXam shows that even the best models achieve substantially lower scores on open-ended legal questions than on multiple-choice tests — and that is in English and German. In Czech, performance would be lower.

Third, the practical impact of artificial intelligence on the Czech legal profession will depend less on the capabilities of the models and more on infrastructure — the quality of Czech legal databases, the availability of retrieval-augmented tools, the Czech Bar Association's regulatory framework, and the training of attorneys in the responsible use of artificial intelligence.

This article draws on the following primary sources: the HAVEL & PARTNERS test (published 28 March 2024), the LegalBench benchmark (vals.ai, current as of February 2026), the LEXam benchmark (arxiv:2505.12864, May 2025, updated on an ongoing basis), the oab-bench benchmark (arxiv:2504.21202, April 2025), the BenCzechMark benchmark (Transactions of the ACL, 2025), and the study by Arnett and Bergen (2024). Czech institutional sources: Advokátní deník, iROZHLAS, Česká justice, Lupa.cz.

Main limitations of the analysis: the absence of a current test of the most advanced models on the Czech bar examination (the latest data are from March 2024), the proprietary nature of the HAVEL & PARTNERS test, which does not allow independent replication, and the extrapolation from English and German benchmarks to Czech law, which may not be reliable.

Transparency of creation:

The conception, structure, and editorial line of the article are the work of the author, who prepared the content outline, set the key theses, and directed the entire creative process. Generative AI (Claude, Anthropic) was used as a technical tool for research, fact-checking, and fleshing out the author's draft.

The author edited the outputs throughout, verified the key findings, and approved the final wording. No part of the text was published without human review. All factual data were verified against the publicly available sources cited in the text.

The process complies with the requirements of Article 50 of EU Regulation 2024/1689 (AI Act) on the transparency of AI-generated content. #poweredByAI