Retrieval-augmented generation, usually shortened to RAG, is the practice of looking information up in your own documents or data and handing it to the language model before it writes an answer.
Why it exists
A large language model on its own answers from
what it absorbed during training. It has never seen your refund policy, your opening
hours or your stock list. Asked about them, it will still produce an answer, because
producing answers is what it does. That is where
hallucination comes from.
RAG changes the order of operations. The question arrives. The system searches your
content for the passages most likely to contain the answer. It puts those passages in
front of the model with the question. The model answers from the passages rather than from
memory.
What it looks like for your business
A customer asks whether they can return a sale item. Without retrieval, the model
guesses from how return policies generally work. With retrieval, the system pulls the two
paragraphs of your actual returns page, and the answer matches what you wrote. If your
returns page does not cover sale items, a well-built system says it does not know and
hands over, rather than inventing a rule.
This is also why “train the chatbot on our website” is a reasonable instruction and
“train the chatbot on our business” is not. Retrieval works on text it can find. Whatever
lives only in a staff member’s head will not be retrieved.
Where retrieval stops being enough
Retrieval answers questions. It does not do things. “What is your returns policy” is a
retrieval question. “Where is my order” is not, because the answer is not in a document.
It is in your order records, and it is different for every customer. That needs a lookup
against a live system, which is a different mechanism. Most disappointment with business
chatbots comes from this gap. The tool retrieves well and acts not at all, so every
question with a customer-specific answer still reaches a person. See
WISMO for the most common example.
Practical notes
- Retrieval quality is mostly content quality. Vague policy pages produce vague
answers. - One fact in two places with two different values will produce inconsistent answers.
Fix the content, not the prompt. - Retrieval does not keep the model from being asked to do something it should not. That
is a separate question about limits, covered under
AI agent.
Related terms
Start for free
AeroChat runs customer service and sales workflows you describe in plain English. There is a free plan, so there is nothing to decide today.
Start for free on the Shopify App Store
Not on Shopify? Create an AeroChat account here.