Checking AI Responses: Recognizing Hallucinations

AI answers are most reliably tested by employees with four questions: Is every number, every quote and every name verifiable? Does the answer really answer the question? Is it up-to-date? And would I be responsible for it with my name? Language models can provide convincing-sounding but invented information. These hallucinations are not a rare slip-up, but a property of technology.

Anyone who uses AI in their everyday work therefore needs one skill above all: to read the result critically. This article shows how to recognize hallucinations and what a quick check looks like.

Why AI invents things

A language model generates text by choosing the probably appropriate next word word for word. It does not "know" whether a statement is true. If it lacks information, it fills the gap with something that sounds plausible. This is how invented paragraphs, studies, quotes or prizes are created. More on this in the glossary under Hallucination.

Warning signs of hallucinations

Very precise figures without a source, such as "73.4% of enterprises ..."
Paragraphs, judgments or norms that you can't look up
Quotes from people without citation
Links that don't exist or point to something else
Statements about your company that AI can't even know
Answers to questions about current events when the tool is not searching the web

The Four-Question Exam

Question	What you do
Verifiable?	Check numbers, names, paragraphs and quotes against the original source
Suitable?	Does the text answer the question asked or a similar one?
Currently?	Is the information still valid? Laws, prices and deadlines change
Responsible?	Would I give this to customers or superiors?

For most everyday texts, this check takes only a few minutes. In the case of legal, medical or financial statements, it is mandatory before something leaves the house.

Where testing is particularly important

Application	Risk	Exam
Formulation help for your own texts	low	Read, check sound
Summary of an existing document	medium	Match samples with the original
Facts, figures, legal issues	high	check each information against a source
Decisions about people	very high	AI only as an aid, decision is made by a human

How to reduce hallucinations in the first place

Giving material instead of letting the AI answer from memory.
Expressly prohibit adding: "Only use the information from the text."
Ask for sources and open them yourself.
Use source-cited tools that access your own documents.

How AI systems are built that back up their answers with sources is explained in the article AI Answers with Source Citation. On the user side, a good prompt helps – see Learning to write prompts.

Human control is also a question of responsibility

Anyone who passes on an AI result is responsible for it. This applies to customers as well as internally. The EU AI Act therefore relies on AI competence: According to Art. 4, companies that use AI must take measures that support the competence of their employees – this includes knowing where AI is wrong. The article AI competence according to Art. 4 explains the obligation.

Practice testing in a team

The best way to recognize hallucinations is to experience them yourself. In our AI training for companies, participants check AI answers to questions from their own field of work and learn when a result is good and when it is not – with a certificate of participation for your documentation.

Funding: Pure training is usually not subsidized by INQA coaching, because the program promotes consulting. If the AI introduction is accompanied as INQA coaching in its entirety, the federal government will cover 80% of the consulting costs of up to €1,200 per day for up to 12 days. The funding check shows which programs are suitable for you.

Frequently Asked Questions

What is an AI hallucination?

A convincingly worded but false or invented statement of a language model, such as a non-existent paragraph or an invented number. It arises because the model produces probable texts without checking facts.

How do I know if an AI answer is correct?

By checking numbers, names, quotes and legal information from the original source and checking whether the answer fits the question and is up-to-date. This check is indispensable for important statements.

Can hallucinations be avoided altogether?

No, but significantly reduce it: through material provided, the instruction not to add anything, and tools that substantiate answers with sources from your own documents.

Who is liable for incorrect AI answers?

Anyone who passes on an AI result is responsible for it. Therefore, a human should always check and decide on statements with consequences for customers or employees.