19 Oct 2026 AI hallucinations: how to test whether your AI application makes things up

In short: An AI hallucination is an answer that sounds right but is made up. In your own AI application, that is a wrong promise in your company’s name. You test for it with five kinds of questions: known facts, questions outside the scope, source checks, invented details and leading questions. Then you measure the share of made-up answers after every change.

Most articles about AI hallucinations are written for people who use ChatGPT. Check the sources, write a better prompt, don’t trust it blindly. Good advice. But if your company runs a chatbot, an assistant or a search tool with AI inside, your customers can’t check anything. They take the answer at face value. So the question for your team is different: how do you find out how often your application makes things up, before your customers do?

What is an AI hallucination?

An AI hallucination is an answer from an AI model that sounds confident and fluent, but is false or made up. It can be a wrong fact, a policy that doesn’t exist or a made-up source. It can also be a fake link or a price that was never on your website. It happens because a language model guesses the most likely next words. It doesn’t look up the truth. When the model doesn’t know, it often still produces something that sounds plausible. Research by OpenAI explains part of the reason: the way models are trained and scored rewards guessing over saying “I don’t know”. Newer models make fewer mistakes, but no model is free of them. The OWASP Top 10 for LLM Applications lists misinformation, including hallucinations, as one of the main risks for applications built on language models.

Why is a hallucination a testing problem?

A hallucination becomes a testing problem the moment your application gives the answer to a customer. To that customer, the answer comes from your company, not from a model. A well-known case shows what that means. Air Canada’s website chatbot told a customer he could ask for a bereavement discount after his flight. The airline’s real policy said otherwise. In 2024 a Canadian tribunal ruled that Air Canada was responsible for what its chatbot said and had to pay. The airline had argued that the chatbot was responsible for its own words. The tribunal didn’t accept that. The lesson for any team: every answer your AI gives is a statement by your company. You test it like any other feature that makes promises to customers.

How do you test for hallucinations?

You test for hallucinations with questions where you know the right answer. And with questions that tempt the AI to make something up. Five kinds of tests cover most of the risk:

1. Known facts. Questions whose answers are in your own documents: prices, terms, opening hours, product details. Compare the answer with the source.
2. Out of scope. Questions your application shouldn’t answer, or can’t. The right answer is “I don’t know” or a referral to a person. An answer full of confidence is a fail.
3. Source checks. If your AI looks up documents before it answers, check that every claim in the answer is in those documents. A true fact that isn’t in the source still means the AI went off on its own.
4. Invented details. Look for sources, links, names, numbers and article references that don’t exist. These are easy to check automatically, so check them every time.
5. Leading questions. Questions with a false premise, like “Since my warranty is five years, can I…?” when it is two. A good answer corrects the premise. A bad one plays along.

Ask important questions more than once. Software with AI inside doesn’t always give the same answer, so one good answer proves little. A simple pass or fail isn’t enough; you score the answers and look at the spread.

How do you keep measuring it?

You keep measuring hallucinations by running the same test set on a schedule and after every change. Then you track the share of made-up answers per category. One number for the whole application hides too much. Ten percent made-up answers about opening hours is a nuisance. Two percent about contract terms can be a legal problem. So group the questions by topic and risk, and look at each group. Run the set again after every new model version, prompt change or new batch of documents. A model update can lower hallucinations in one area and raise them in another. It is the same loop as in our five-step model for testing AI after go-live. Questions, schedule, scores, failures linked to rules, and reports.

Can you prevent hallucinations?

You can reduce hallucinations, but you can’t prevent them completely, so you still need to test. Several measures help. Let the AI answer from your own documents instead of its memory, often called retrieval-augmented generation (RAG). Tell it clearly that “I don’t know” is a good answer. Limit what it may talk about. Add checks that block answers with prices or terms that don’t match your data. Each of these lowers the risk. None of them proves that the risk is gone. Only tests show how often your application still makes things up, and in which areas.

That is where most teams get stuck after a few months. That is why we built VeritasAI, our in-house tool for testing AI applications. We use it to test your AI application as a service. It runs your test questions continuously and checks whether the answers are correct. The operational dashboard shows which questions fail, with the prompt, the answer and a priority score, so your team fixes the most harmful answers first. The management dashboard shows whether your application gets better or worse over time. Read more on our AI Compliance Testing page.

Frequently asked questions

Does AI still hallucinate in 2026?

Yes. Newer models make fewer factual mistakes than older ones, but every language model still hallucinates sometimes. How often depends on the model, the topic and how your application is built. That’s why you measure it in your own application instead of relying on general benchmarks.

Why does ChatGPT hallucinate?

Language models predict likely words; they don’t look up facts. When the model has no good answer, it often produces one that sounds plausible. Training methods that reward guessing over admitting uncertainty make this worse.

Can you stop AI from hallucinating?

Not completely. Answering from your own documents, clear instructions and checks on the output reduce hallucinations a lot. Regular tests show what’s left and where.

What triggers AI hallucinations most often?

Questions about things the model doesn’t know, and questions with a false premise. Requests for sources or exact numbers. And topics where the AI’s documents are incomplete or out of date.