What retrieval-augmented generation is, why it keeps support AI accurate, and how to set up a knowledge base your chat agent can answer from.
Your customers ask about refund windows, delivery zones and plan limits, and a general-purpose AI model does not know your answers. Left alone, it will either decline to help or produce something that sounds right and is wrong. Retrieval-augmented generation, usually shortened to RAG, is the standard way to fix that. This article explains how it works in plain terms, where it goes wrong, and how to build a knowledge base that makes it reliable on website chat, WhatsApp and Messenger.
What RAG means in plain language
A language model is trained on a large body of text, then frozen. It knows a lot about the world in general and nothing about your business: not your returns policy, not last week's price change, not the exact wording of your warranty.
RAG adds a lookup step before the model writes its reply. The name describes the three parts:
- Retrieval: the system searches your own content for passages relevant to the customer's question.
- Augmented: those passages are placed in front of the model along with the question.
- Generation: the model writes an answer using the passages as its source material.
A useful comparison is an open-book exam. A model without RAG answers from memory. A model with RAG is handed the right pages of your handbook and told to answer from them.
Why support teams use it instead of other approaches
There are three common ways to get a business-specific AI. Each has trade-offs.
Prompt stuffing
You paste your whole policy document into the instructions. This works for a tiny business with a one-page FAQ. It breaks down as content grows, because there is a limit to how much text a model can take at once, and a crowded prompt makes it harder for the model to find the one sentence that matters.
Fine-tuning
You retrain a model on your content. This changes how a model writes, but it is a poor way to teach it facts that change. Every price update would mean another training round, and you cannot easily point to where an answer came from.
RAG
You keep your content in a knowledge base and fetch only the relevant parts per question. Updating an answer means editing a document. Removing an outdated policy means deleting it. That is why RAG is the usual choice for support, where the facts change often.
How a RAG reply is built, step by step
Behind a single chat answer, the sequence looks like this.
- The customer asks. For example, on WhatsApp: "Can I return a sale item after 30 days?"
- The question is interpreted. The system works out what is being asked, including from earlier messages in the conversation.
- Relevant passages are retrieved. The knowledge base is searched and the most relevant passages are selected, such as the returns policy and the sale terms.
- The model drafts a reply. It receives the question, the passages and the agent's persona and rules, then writes an answer grounded in that material.
- The reply is sent. If nothing relevant was found, a well-configured agent should say so and hand the conversation to a person rather than guess.
- 01Customer question
- 02Search knowledge base
- 03Model reads passages
- 04Grounded reply or handoff
Step 5 matters most for trust. A support agent that can say "I don't have that information, let me bring in a colleague" is more valuable than one that always has an answer.
Where RAG fails, and how to prevent it
RAG reduces made-up answers but does not eliminate every problem. Most failures trace back to the knowledge base, not the model.
Missing content
If the answer is not in your sources, retrieval has nothing to find. Customers will ask things you never documented. Review real conversations to find these gaps.
Contradictory content
If an old FAQ says returns take 14 days and a newer page says 30, the model may quote either one. Retire outdated material instead of leaving it beside the current version.
Poorly structured content
A 40-page PDF with a policy buried in a footnote is hard to retrieve from. Short pages with clear headings, each covering one topic, work better than long documents covering everything.
Vague wording
"Delivery usually takes a few days" gives the model nothing precise. "Orders ship within 2 business days; delivery takes 3 to 5 business days in the US" gives it something to quote. Write answers the way you would want a customer to read them.
Overreach
Even with good sources, a model can drift into topics you do not want it to discuss. Guardrails in the agent's instructions, such as "do not give legal or medical advice" or "never promise refunds", keep it in scope.
Building a knowledge base that retrieves well
You do not need a large knowledge base. You need a clean one.
Start with the questions customers actually ask
Pull the ten to twenty most common questions from your inbox, chat history or order support tickets. Write or collect an authoritative answer for each. These cover most of the real traffic.
Use the right format for each type of content
- Question-and-answer pairs suit short, exact answers such as opening hours, shipping costs and return windows.
- Policy documents and manuals suit longer material, as long as they are current and well organised.
- Website pages suit content you already maintain publicly.
- Help-centre or wiki articles suit teams who already document in a tool such as Confluence or Zendesk Guide.
Keep one source of truth per topic
When the same fact lives in four places, one of them will go stale. Decide which source is authoritative and remove or update the rest.
Write for the channel
Messaging channels reward brevity. A customer on WhatsApp or Messenger wants a two-sentence answer with a next step, not a policy essay. Include a plain-language summary at the top of longer documents so the agent has a concise passage to draw from.
Protect sensitive material
Anything in a knowledge base may be shown to strangers in some form. Keep internal pricing rules, staff notes and personal data out of it unless you have a reason to include them.
Measure and maintain
Treat the knowledge base as a living asset:
- Review conversations where the customer was unhappy or asked for a human.
- Add the missing answer, or correct the wrong one.
- Test the same question again to confirm the fix.
- Repeat on a regular schedule, weekly at first and monthly once things settle.
Doing this in AI SNS CHAT
AI SNS CHAT's agents answer from the knowledge sources you attach to them, and the whole workflow described above maps onto a few places in the dashboard.
Add your sources
Open Data sources in the sidebar. You can add:
- Files, for manuals and policy documents
- Text snippets, for short facts you want to type directly
- Website, to read your existing public pages
- Q&A, for exact question-and-answer pairs
- Notion, Social, Tickets and Suggestions
If your documentation lives elsewhere, the Channels area lists connectors for Confluence, GitBook, ReadMe, Guru, Zendesk Guide, Google Drive, OneDrive, Dropbox, Box, SharePoint and others, along with esa.io, DocBase and Qiita Team for Japanese teams. Wiki-style connectors read up to 1.5 million characters and 400 pages per sync, and if a sync is truncated you are told, so you know when a source was only partly read. Storage by plan runs from 30 MB on Free to 3 GB on Corporate, with extra storage available as an add-on.
Set the persona and guardrails
In the agent's settings you define the persona and the rules it follows. This is where you write instructions such as what tone to use, what it must never promise, and when to pass a conversation to a person. A prompt generator, a prompt library and 14 starter personas help you begin.
Test before you publish
Use Playground to ask the questions your customers ask and see how the agent answers. Playground also has an A/B model compare, which helps when you want to check how two models handle the same knowledge.
Improve from real conversations
Every conversation outside email appears under Activity → Chat logs, where you can filter, flag, escalate and review thumbs up and down ratings. In both Chat logs and the email Inbox you can edit a reply into the ideal answer and use it to train the agent, which is the fastest way to close a knowledge gap you have just spotted.
Hand off when the knowledge runs out
Under Helpdesk → Scheduling you set shifts per person and timezone, and handoffs are assigned to whoever is on shift. Alerts can go to Slack, Teams, LINE WORKS, WhatsApp, email or a webhook. The customer message on handoff includes a "what happens next" line based on your rota, and it never invents a wait time. Helpdesk needs the Standard plan or higher.
Web search is a separate, limited feature
On LINE WORKS, the agent can answer from the web when the knowledge base has no answer, with its own settings under Behaviour → Web search. For other agents, the Tavily Action adds real-time web search. Neither replaces a maintained knowledge base for your own policies.
Each reply uses one credit. Plans and credit allowances are on the pricing page, and the channels you can deploy to are listed on the connections page. To try it, sign up and add your first source.
Conclusion
RAG is a simple idea: look up the right passage, then answer from it. Its quality depends less on clever technology than on the content you give it. Keep your sources current, write answers that are specific and short, test with real questions, and let conversations tell you what is missing. Do that, and an AI agent becomes a dependable first line of support.
Frequently asked questions
Is RAG the same as training an AI on my data?
No. Training changes the model itself, while RAG leaves the model unchanged and gives it your documents at the moment it answers. That is why updating a RAG-based agent is as simple as editing or replacing a source.
Can RAG still give wrong answers?
Yes, usually because the source is missing, outdated or contradictory. Clear content, sensible guardrails and a handoff path for questions the agent cannot answer reduce this, and reviewing real conversations helps you find the remaining gaps.
How much content do I need to start?
Far less than most teams expect. A set of accurate answers to your most common questions covers a large share of everyday traffic, and you can expand from there based on what customers actually ask.
Does the same knowledge base work across website chat, WhatsApp and Messenger?
In AI SNS CHAT, an agent's knowledge sources belong to the agent, and you can deploy that agent to several channels. Write answers that are short enough to read comfortably in a messaging app, since that is where customers will see them.
Should I use web search instead of building a knowledge base?
No. Web search is for general questions, while your own policies, prices and procedures must come from content you control. A maintained knowledge base is what keeps answers consistent and accurate.


