Compendium / Brand Risk

Hallucination

TypeConsensus concept
Term maturityestablished
Operator maturityplausible
Lifecyclein flux
Relevancestrategic
Verified2026-10-08
Hallucination is when an AI generates factually incorrect content presented as true. For brands, this means AI assistants can state wrong facts about products, pricing, personnel, or capabilities — with no correction mechanism inside the response. A brand with high visibility but high hallucination rate is being introduced incorrectly at the most influential moment in the buyer journey.
17–33 %
of responses from leading RAG-based legal research tools contained hallucinations (Stanford RegLab, 2024) — retrieval narrows the failure, it does not remove it.source

Consensus definition

In large language models, hallucination refers to the generation of content that is plausible-sounding but factually incorrect or fabricated, presented without uncertainty signals1. The model generates statistically likely output — it does not look facts up. When its training data or retrieval context is thin, incomplete, or outdated, it fills the gap with plausible-sounding approximations that can be demonstrably false.

rhinegold operator caution

Rhinegold's reframe: hallucination is not primarily an AI quality problem — it is a brand risk management problem. When an AI assistant states an incorrect product feature, a wrong price, or a fabricated claim about an organisation, a buyer may act on that information before ever reaching the brand's website. The exposure is real and entirely invisible to standard analytics. The measure Rhinegold is building for this is a Hallucination Risk Index (HRI): statements about a brand in AI answers, checked against a confirmed fact base — the brand's own website and its grounding page. A first version that scored wording patterns in the answer text was withdrawn after review: it flagged phrasing, not false facts. The fact-based version is in development; it is not yet a calibrated metric. The dimension it targets is the one Citation Rate and Mention Rate do not capture — being cited often while being described wrongly is an actively damaging combination.

Operational use

Use when making the case for ongoing AI-answer monitoring beyond visibility tracking. A brand with a high citation rate whose facts are stated wrongly is being cited with incorrect information — which can be worse than not being cited at all. The errors found also show which content assets and structured data need urgent attention.

Measurement boundary

Hallucination rate varies by provider, model version, query type, and how well a brand is represented in training data and grounding sources. No platform publishes per-brand or per-vertical hallucination rates. Measurement requires running a structured brand-fact prompt set against a verified reference, then scoring each response — it does not scale without tooling, and results are point-in-time snapshots that shift with model updates.

What can still be observed

A directional error check per platform can be run with a curated set of brand-fact prompts — covering known products, prices, team, and key claims — against each AI platform and scoring responses against a verified fact base. This gives a relative risk ranking across platforms that is actionable even without perfect precision.

Distinct from

From Grounding: grounding is the mechanism that anchors AI responses to verifiable sources — better grounding reduces hallucination risk but does not eliminate it. A grounded response can still misrepresent the source it cites. From Citation Rate: Citation Rate measures whether the brand is named as a source; a fact check measures whether what the AI says about the brand is correct. The two metrics are independent — and need to be tracked together. From AI Reputation Risk: the broader strategic frame, of which hallucination is one driver alongside negative framing, missed mentions and consistently weaker positioning relative to competitors.

Obstacles & resolutions

Competing definitions make hallucination figures incomparable
Venkit et al. (EMNLP 2024) audited 103 publications and found no agreement on what 'hallucination' means — the term covers closed- and open-domain errors, factual errors and unfaithful summaries alike. When a platform announces 'we reduced hallucination by 30 %' and a brand auditor reports an error rate for the brand, neither number is comparable to the other. The number is determined by the definition, not by the model's accuracy on the facts that matter to the brand.
rhinegold resolutionDefine the test narrowly: statements about the brand on claims with commercial consequence, checked against the brand's own confirmed facts — not an academic benchmark. A narrow definition does not make a number comparable across companies, but it keeps one brand's measurements comparable over time, provided the fact base and the prompts stay fixed.[ref]
RAG does not eliminate hallucination — it shifts the failure mode
Retrieval-Augmented Generation is widely cited as the solution to hallucination. Magesh et al. (Stanford RegLab, 2024) tested leading RAG-based legal AI tools and found hallucination rates of 17–33 % — comparable to or worse than non-RAG baselines in controlled conditions. The failure mode shifts: instead of fabricating from scratch, the model retrieves the right source and then misattributes, cherry-picks, or paraphrases it incorrectly. Practitioners who assume 'we use RAG, therefore hallucination is solved' are measuring the architecture, not the output.
rhinegold resolutionTest the output, not the architecture: brand-fact accuracy has to be checked on live answers, whatever the platform's retrieval set-up. The question is never 'does this system use RAG?' but 'what does it actually say about the brand?'[ref]
Domain-specialised models are not automatically more accurate
A common expectation: a model trained on domain data hallucinates less in that domain. A 2025 study of foundation models in medicine found the opposite: general-purpose models produced fewer hallucinations than medical-specialised ones, which ranged from 28.6 % to 61.9 % despite explicit training on medical texts. The model that sounds most like a domain expert is not necessarily the most reliable one.
rhinegold resolutionDo not infer accuracy from specialisation. Re-check brand facts whenever a platform the brand relies on changes its model — not only when domain training was explicitly announced.[ref]
Reported hallucination improvements may reflect benchmark memorisation
Most LLM hallucination benchmarks are publicly known, and static test sets can leak into training data. HalluLens (Bang et al., 2025) generates its test data dynamically for exactly this reason: to keep leaked test items from saturating the benchmark. A brand that uses published hallucination leaderboards to decide which AI platforms to trust is relying on a signal that training can inflate — without any intent to deceive.
rhinegold resolutionUse brand-specific prompts derived from the brand's own fact base and keep them unpublished. They are far less exposed to leakage than public benchmarks — which is also why third-party leaderboards cannot replace a brand's own check.[ref]

Empirical anchor

Measured hallucination rates vary widely with the task and the domain: RAG-based legal research tools hallucinated in 17–33 % of responses2, and medical-specialised models in 28.6–61.9 %3. For brands with limited training-data coverage — smaller organisations, niche products, recent changes in pricing or personnel — hallucination risk is structurally higher: the model fills low-confidence gaps with plausible-sounding fabrications. This asymmetry means hallucination risk is not uniform across a competitive set and cannot be inferred from a competitor's experience.

Common mistakes

  • Treating hallucination as an AI vendor problem rather than a brand monitoring responsibility.
  • Assuming high Citation Rate implies accurate citation — the content of a citation can still be factually wrong.
  • Measuring only visibility (Mention Rate, Citation Rate) without any factual accuracy check.
  • Assuming hallucination risk is uniform across competitors — it scales inversely with training-data coverage.

Where consensus is missing

There is no standardised per-brand or per-vertical hallucination benchmark. Platform providers do not publish hallucination rates by topic or brand. Measurement is bespoke and depends critically on what facts are chosen as the test set and how scoring is defined — making cross-company comparisons unreliable.

Sources & deeper reading

Related research
Industry observation
Methodological source

Update history

  • 2026-10-08 — Corrected the description of the Hallucination Risk Index: a first, pattern-based version was withdrawn; a version checked against confirmed brand facts is in development and not yet calibrated. Removed the 'under 5 % to over 25 %' range, which the cited survey does not contain. Corrected the titles and findings of three cited papers (Venkit et al., the 2025 medical-hallucination study, HalluLens) and the link to Huang et al.
Last verified 2026-10-08 · Next review 2027-01-06
Question this measurementA metric can be right and the conclusion still wrong.Ask Dietrich →
Related terms
GroundingCitation RateMention RateGenerative Engine OptimizationAI Brand Reputation Risk: The Complete GuideSentiment in AI AnswersMulti-Axis Sentiment in GEONegated MentionAI Reputation RiskPhantom URL
Cite this entry
rhinegold. “Hallucination.” The Rhinegold Compendium. https://insights.rhinegold.de/compendium/hallucination-brand-risk/. Updated 2026-10-08.