AI Hallucination Statistics 2026: Rates, Benchmarks & Why They Disagree

Published hallucination rates in 2026 span from 0.7% to 94%. That is not measurement noise — it is four different questions being asked and reported under one word.

If you have seen "AI hallucinates 3% of the time" and also "AI hallucinates most of the time," both citations were probably accurate. The gap is the benchmark. Grounded summarization — hand the model a document, ask it to summarize only what is in it — produces low single digits. Open recall — ask the model a factual question with no source — produces double digits and worse. Generating SQL against a real enterprise schema produces the worst numbers of all. Below are the figures worth knowing, each with the test that produced it.

Grounded summarization: the flattering number

Vectara's Hallucination Leaderboard is the most-cited source in this space and the one behind most low headline figures. Its method is narrow and worth understanding before you quote it: over 7,700 articles across news, technology, science, medicine, legal, sports, business and education are fed to each model, which is asked to summarize each one using only the facts presented in the document. A detector then flags output that introduces unsupported content.

As of the 11 May 2026 update:

• Antgroup finix_s1_32b — 1.8% (98.2% factual consistency), top of the board.
• OpenAI gpt-5.4-nano — 3.1%; Google Gemini 2.5 Flash-Lite — 3.3%; Microsoft Phi-4 — 3.7%; Meta Llama 3.3 70B Instruct Turbo — 4.1%.
• GPT-5.4 Mini — 5.5%.
• Mistral ministral-3-3b-2512 — 24.2%, last among listed models.

This measures intrinsic hallucination — fabrication relative to a source you provided — not general factuality. A 1.8% score means the model is good at not inventing things when the answer is sitting in front of it. It says nothing about what happens when the answer is not.

The same model, 0.7% or 7.6%

The clearest demonstration that the benchmark makes the number: Gemini-2.0-Flash-001 scores 0.7% on grounded summarization. Run the same model against Vectara's harder FaithJudge evaluation and its effective rate rises to 7.6% — more than a tenfold difference, same model, same week.

Widen to open-recall benchmarks, where the model answers from its own parameters with no document to anchor to, and Stanford HAI-style evaluations report rates from roughly 22% to 94% depending on the domain and how strictly "correct" is defined.

So a vendor claiming a 2% hallucination rate and a researcher claiming 60% can both be telling the truth. Always ask which test.

The reasoning-model paradox

An intuition worth discarding: that models which "think longer" hallucinate less. Multiple 2026 evaluations report the opposite on factual recall — reasoning models including OpenAI's o3 and Grok-4-fast show higher hallucination rates than non-reasoning models on those tasks.

More deliberation produces more elaborate output, and elaborate output has more surface area on which to be wrong. Buying a more expensive reasoning tier does not buy factual reliability.

The number that matters for business data: the text-to-SQL cliff

Summarization benchmarks tell you little about what happens when you point an assistant at your company's database. For that, the relevant literature is text-to-SQL, and the results are the most sobering in this article.

Spider 1.0 is the long-standing academic benchmark: clean, small schemas. Spider 2.0 was built to simulate real enterprise conditions, with database schemas averaging around 800 columns. The same models were run on both.

86.6% to 10.1% for the same model. Nothing changed except the realism of the database. On BIRD, another execution-based benchmark with a strict exact-match standard, reported execution accuracy sits around 73.0%, with the leading published system reaching 81.95% on the test set — better, but still meaning roughly one query in five returns the wrong result.

This is the statistic to keep. Demos run on clean, small tables and look excellent. Your warehouse is not clean or small. Accuracy measured on an academic schema does not predict accuracy on yours, and the failure is silent — a wrong query returns a plausible number, not an error.

Why wrong answers do not look wrong

A hallucinated summary is often detectable by reading it. A hallucinated number is not. If an assistant joins two tables incorrectly and reports net sales of $1.83M instead of $1.94M, nothing about the output signals a problem. It is formatted correctly, it is the right order of magnitude, and it arrives with a confident explanation.

There are two independent sources of variance stacked on top of each other:

• Query generation. The model writes the query from scratch each time. Same question next week, different join path, different filter on status, different number.
• Data retrieval. Where the tool retrieves excerpts rather than reading everything — context windows are finite — the model may be reasoning over a different slice of your data than it saw last time.

Both can be present at once, and neither announces itself. We wrote about the practical version of this in why AI gives a different number every time.

What actually reduces the error rate

The pattern across all four benchmark families is consistent: models hallucinate least when they are given the answer and asked to relay it, and most when they are asked to derive it. Grounded summarization scores 1.8%; open recall scores up to 94%. That difference is the entire design principle for reliable AI over business data.

Practically, that means moving work out of the model:

• Define metrics once, not per question. If "net sales" is a stored definition that compiles to SQL, the model is not re-deriving it every time — which removes the query-generation source of variance entirely.
• Let the database do arithmetic. Aggregation executed in SQL is deterministic. Aggregation performed by a language model reading rows is not.
• Keep the dataset out of the context window. If the answer comes from a query result rather than from as much of the file as fits, retrieval variance stops applying.
• Prefer tools that show the query. An answer you cannot audit is an answer you cannot check.

This is the approach Quiriz takes — governed metric definitions that compile to SQL, arithmetic executed by the database, and the model used for interpreting the question and phrasing the answer rather than for computing it. It does not make a language model factual; it removes the steps where a language model was being asked to be. For where each general assistant stops, see our comparisons with ChatGPT, Claude, Copilot and Gemini, and our 2026 AI pricing comparison for what each costs.

If you are quoting these numbers

• Name the benchmark. "3% hallucination rate" is meaningless alone; "3% on Vectara's grounded summarization leaderboard" is a fact.
• Do not generalise from summarization to reasoning. They differ by an order of magnitude or more.
• Check the date. Leaderboards move. Vectara's was last updated 11 May 2026 at the time of writing.
• Treat model-family comparisons carefully. Model names and versions change faster than benchmark runs, so a leaderboard row may not describe the model you are actually using.

Sources

• Vectara, Hallucination Leaderboard — HHEM grounded-summarization methodology, 7,700+ source articles, per-model rates. Last updated 11 May 2026.
• Vectara, "Introducing the Next Generation of Vectara's Hallucination Leaderboard" — FaithJudge evaluation and the 0.7% vs 7.6% divergence.
• Spider 2.0 — enterprise-scale text-to-SQL benchmark; schema complexity and execution-accuracy results versus Spider 1.0.
• BIRD text-to-SQL benchmark — execution accuracy under strict exact-match evaluation on real databases.
• Stanford HAI, AI Index Report — open-recall factuality ranges.

Figures compiled from published benchmarks and leaderboards as of 17 August 2026. Hallucination rates are not comparable across benchmarks and should always be quoted together with the evaluation that produced them; leaderboard positions change frequently. Model names reflect those published by the benchmark maintainers at the time of the run and may not correspond to the current release of a given product.