When your AI vendor tells you "that was a hallucination", it sounds like a technical diagnosis. It almost never is. In the AI agents we've reviewed at AutoBoost, what gets filed under that word is actually three or four different failures, with causes and fixes that have nothing to do with each other. The word is useful mostly because it saves the vendor from having to explain which of the four is actually yours.
This isn't a semantic nitpick. Whichever answer you accept completely changes what you do next. If the problem really is the model, trying a different one might help. If it's any of the other three, switching models fixes nothing: the same failure comes back with the new model, because the model was never the cause. And that's exactly the diagnosis a comfortable vendor would rather not give, because the real fix is almost always integration work, not a new subscription tier.
The four things that get called "hallucination"
1. It's missing a tool, not a fact
The most common case we see: an AI agent tells a customer there are 40 units in stock when only 3 are left, or applies a rate that expired weeks ago. It's not that it "made something up" in the way people usually mean. Nobody gave it a way to check that data at the exact moment it answered, so it answered with whatever was in its context, which can be days out of date. The fix isn't more information stuffed into the prompt: it's a function that queries the ERP live. We cover this in more detail in our post on tools versus context.
2. It didn't have the source in front of it, so it filled in from memory
Different from the case above: here the agent did have access to documentation, but wasn't required to cite it or stay within it. You ask about a clause in a manual and it gives you something plausible that appears nowhere. This is the classic problem a well-designed RAG setup solves: the agent searches the real documents first and only answers with what it finds there, explicitly flagging when there's nothing to cite. It's exactly the approach we used when building an AI assistant over internal manuals that doesn't make things up: the source rules, and if the source says nothing, the correct answer is "I don't know", not a good-looking guess.
3. The source data was already wrong, and the AI just repeated it with confidence
This is the hardest one to catch, because it doesn't look like an AI failure: it looks like one, but it's actually a data failure the AI inherited. In a data platform we built for a pharmacy group, the AI had to reason over more than a million rows to reconstruct a sales figure that the client's own ERP was calculating at almost half its real value. The problem was never the AI: it was how the source system aggregated those rows. The AI didn't "hallucinate" the correct figure, it reconstructed it properly because it was verified line by line against the real data, landing at a final deviation of 0.0004%. Without that verification step, any number it had produced, the wrong ERP figure or any other, would have sounded just as convincing. We go into detail in the AI data platform case study and in why data comes before AI.
4. The real hallucination: it makes something up even with everything in front of it
This is the only one of the four that deserves the name in the strict sense: the agent had the right tool, the right document in front of it, and correct source data, and it still invented something that appeared nowhere. It tends to happen with ambiguous questions or ones outside the scope the agent was designed for. Here it does help to tighten the prompt, add an explicit guardrail ("if it's not in the documents, say you don't know"), or, as a last resort, try a different model. But it's by far the least frequent of the four.
The four failures, side by side
| What it looks like | Real cause | Who fixes it | Does switching models help? |
|---|---|---|---|
| Invents stock or price | Missing a tool to check live data | Whoever integrates the AI | No |
| Invents a clause not in the manual | No real document in front of it, no source restriction | Whoever integrates the AI | No |
| Gives a confident figure that's wrong | The source data was already wrong before the AI touched it | The data system, before the AI | No |
| Invents something despite correct tool, document and data | Real model or prompt failure | The AI vendor | Can help |
Three questions before you accept "it was a hallucination"
In this order, because each "no" already gives you the answer and saves you from asking the next one:
- Did it have a tool to check the real data at the moment it answered, or did it respond with whatever was already in its context?
- Did it have the actual document or record it was supposed to cite in front of it, or was it generating something plausible with no source restriction?
- Was the source data it used correct, or was the underlying system (the ERP, the spreadsheet, the CRM) already wrong before the AI touched it?
If the answer to any of the three is "no", you're not looking at a hallucination in the strict sense: you're looking at a design failure, and it gets fixed without touching the model. If all three answers are "yes" and it still made something up, then you're in the fourth case, the only one that truly earns the word.
Checklist before you accept the diagnosis
- Did the agent have a tool to check live data (stock, price, customer status), or only fixed context pasted into the prompt?
- Did it have the real document or record it was supposed to use in front of it, or was it working from general memory?
- Has anyone verified the source data was correct before blaming the AI?
- Does the process verify the result before acting (reconciling a figure, checking a total), or does it trust the model's answer blindly?
- If all of the above checks out and it still invents things, has it been tried with a different model or a different prompt?
Why this isn't just semantics
Accepting "it was a hallucination" at face value sends you to the wrong fix three times out of four: you switch vendors, switch models, or pay for a higher tier, and the same failure comes back as soon as the stock data goes stale again or someone asks something outside the documents the AI has access to. The real fix is almost always in how the AI is integrated into your actual systems (what tools it has, what sources it can cite, whether it verifies before acting), not in which model sits behind it. It's the same principle we apply on every project: sort out the data and clearly define what the AI can query and cite first, and only then judge whether the model itself is the problem. You can see how we build this in our AI integration services.
If your team keeps hearing "hallucination" every time something goes wrong and you want to know which of the four cases is actually yours, get in touch. Usually within half an hour we can tell whether the problem is the model or, as is almost always the case, how it's been set up around it.
