An AI hallucination is an answer that is fluent, confident and wrong, produced because the model generated plausible text rather than checking anything.
What it looks like when it happens to you
A customer asks whether a dress can be returned after thirty days. Your policy says
fourteen. The bot says thirty, warmly and clearly. Three weeks later the customer arrives
expecting a refund, with a screenshot. You are now choosing between honouring a rule you
never set and arguing with a customer who has your own chat in writing. The cost of the
hallucination is not the wrong sentence. It is that decision.
Why it happens
A language model produces likely text. When it
has the facts, likely text is usually true. When it does not, likely text is still
produced, because there is no step in which the model notices it is missing something.
Nothing in the mechanism distinguishes recall from invention.
So hallucination is not a bug that gets fixed in a later version. It is what happens
when a text predictor is asked a question it has no material for. The engineering answer
is to stop asking it questions it has no material for.
Four things that actually reduce it
- Answer from your content, not from memory.
Retrieval puts your real policy in front
of the model before it writes. This is the single largest improvement available. - Look up customer-specific facts in the system that holds them. Order
status, booking slots and stock levels should come from a query, never from the model’s
impression. See WISMO. - Allow “I do not know”. A system that must always produce an answer
will always produce an answer. A system permitted to hand over will hand over. See
live chat handover. - Put limits outside the model. If the refund ceiling is enforced by
the software, no wording persuades the model past it. If it is only written in the
instructions, wording eventually will.
The merchant’s version of this worry
Business owners rarely use the word hallucination. They say it differently. One
request we have seen put plainly: “Respond only using the content I provide, without
making things up.” That is the whole requirement in one sentence, and it is a reasonable
thing to ask of any tool you are evaluating.
Worth keeping in proportion. When we read 2,793 public reviews of 67 customer service
and chatbot apps on the Shopify App Store, control over what the AI says ranked fifth as a
complaint theme, at 8%. Support quality came first, at 19%. Hallucination is a real risk
to manage, not the thing most people are angry about.
Related terms
Read next
Start for free
AeroChat runs customer service and sales workflows you describe in plain English. There is a free plan, so there is nothing to decide today.
Start for free on the Shopify App Store
Not on Shopify? Create an AeroChat account here.