What does RAG stand for in AI? Retrieval-augmented generation, explained
In AI, RAG means retrieval-augmented generation: the AI looks things up in your documents before it answers. Here is how it works, when it helps and when a simpler setup is enough.
What does RAG stand for in AI?
In AI, RAG stands for retrieval-augmented generation. Before an AI model answers, the system retrieves relevant information from a knowledge source, such as your documents or help articles, and adds it to the prompt. The model then generates an answer grounded in that information instead of relying only on what it learned in training.
- RAG stands for
- Retrieval-augmented generation
- First described
- Lewis et al., 2020 paper, accepted at NeurIPS 2020
- What it does
- Looks up relevant facts and adds them to the prompt before answering
- Main benefit
- New or private knowledge without retraining the model (AWS)
- Small knowledge base
- Under 200,000 tokens, Anthropic suggests putting it all in the prompt
What does RAG stand for?
In artificial intelligence, RAG stands for retrieval-augmented generation. The name describes the three moves: retrieve relevant information, augment (add it to) the prompt, and generate the answer.
The term comes from a 2020 research paper, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis and colleagues, accepted at the NeurIPS 2020 conference. It described models that combine what a model learned in training with a searchable index of documents, in their case Wikipedia.
Today the idea is everywhere. AWS describes RAG as optimizing a large language model’s output so it references an authoritative knowledge base outside its training data before generating a response. A support bot that answers from your help center, or a coach app that answers from the coach’s own method, is usually doing some form of RAG.
How does RAG work?
RAG works in two stages: first you prepare your knowledge so it can be searched, then, for each question, the system finds the best pieces and hands them to the model with the question.
- Prepare the knowledge: documents, help articles, notes or product data are split into small chunks and turned into numbers (called embeddings) that capture their meaning, stored in a vector database.
- Retrieve: when a question comes in, it is turned into numbers the same way, and the system finds the chunks closest in meaning.
- Augment: the best chunks are added to the prompt as context, along with instructions like “answer only from these sources.”
- Generate: the model writes the answer, ideally citing which source it used.
- Keep it fresh: when documents change, their chunks are updated so answers stay current.
A simple example: a nutrition coach uploads 40 pages of her meal-planning method. A client asks, “Can I swap rice for quinoa on day 3?” The system finds the two paragraphs about grain swaps, adds them to the prompt, and the model answers in the coach’s own rules rather than generic internet advice.
- Write one clear article per question people actually ask, rather than one long document.
- Remove old versions of policies and prices so the search can’t find them.
- Tell the model to answer only from the sources and to say when it doesn’t know.
- Ask it to name the source it used, so you and your users can check.
- Test with 20 real questions from customers, including awkward ones, before launch.
Why use RAG? The benefits and limits
RAG is the cheapest common way to make an AI model answer from your own, current information. AWS lists the main benefits:
- Cost: adding new data through retrieval is far cheaper than retraining a model.
- Current information: you can feed in recent facts, prices, policies or news.
- Trust: answers can cite their sources, so people can check them.
- Control: you choose the sources, can restrict them by who is asking, and can fix a wrong answer by fixing the document.
The limits are real too. RAG reduces made-up answers but does not remove them; IBM notes that even a well-built system cannot fully remove the risk of hallucinations. And RAG is only as good as its retrieval: if the search pulls the wrong paragraph, the model answers confidently from the wrong source.
RAG vs fine-tuning: which do you need?
Use RAG to give a model knowledge; use fine-tuning to change how it behaves. IBM explains that RAG connects a model to an organization’s own data at question time, while fine-tuning means retraining a pretrained model on a smaller, focused set of training data.
| RAG | Fine-tuning | |
|---|---|---|
| What changes | What the model can see when it answers | The model itself |
| Best for | Facts, documents, policies, product data | Tone, format, specialist tasks and vocabulary |
| Updating knowledge | Edit the document; the next answer uses it | Train again |
| Can cite sources | Yes | No, the knowledge is blended into the model |
| What you need | Your documents and a search setup | Labeled examples and compute to train |
For most small AI products, RAG plus good instructions is enough. Fine-tuning makes sense later, if you have many examples of exactly how answers should look.
What is agentic RAG?
Agentic RAG is RAG run by an AI agent that decides where to look, whether the results are good enough and whether to search again. IBM defines it as the use of AI agents to facilitate retrieval-augmented generation.
| Traditional RAG | Agentic RAG | |
|---|---|---|
| Sources | Usually one knowledge base | Several knowledge bases and tools, such as web search |
| Search | One lookup per question | Plans, searches, checks and searches again if needed |
| Checks its results | No | Yes, it can refine earlier work |
| Cost and speed | Cheaper and faster | More AI calls, more tokens, slower |
IBM is candid about the downsides: more agents mean higher costs, and agents are not always reliable. It suggests agentic RAG fits best when a question needs several data sources. For how agents plan and loop, see agentic workflows.
When don’t you need RAG?
If your knowledge is small, you may not need retrieval at all. In its contextual retrieval write-up (September 2024), Anthropic suggests that for a knowledge base under 200,000 tokens, roughly 500 pages, you can include the whole thing in the prompt, and prompt caching makes that faster and cheaper.
When you do need RAG, small changes to retrieval matter. In Anthropic’s tests, adding context to each chunk cut failed retrievals by 35%; adding keyword search on top brought the cut to 49%, and adding a reranking step brought it to 67%.
The takeaway for a small product: start simple. Put your rules and a short knowledge base straight into the instructions, see where answers go wrong, and add retrieval only when the material outgrows the prompt or changes too often to paste in by hand.
| Your knowledge | Simplest approach |
|---|---|
| A one-page FAQ or rule list | Put it in the instructions |
| A short handbook or a few dozen help articles | Include it all in the prompt, with caching |
| Thousands of pages, or data that changes daily | RAG |
| Several systems plus the web | Agentic RAG |
What are real examples of RAG?
RAG shows up wherever an AI has to answer from specific material rather than general knowledge:
- Customer support agents that answer from your help articles and refund policy.
- Expert tools: a fitness coach’s programs, a tutor’s course notes or a dietitian’s rules, turned into an assistant for their audience.
- Document Q&A: “What does my lease say about pets?” over an uploaded file.
- Internal search: staff asking questions across company docs.
Owning that knowledge is also what makes an AI product hard to copy; see what an AI wrapper is and when it becomes a real business. To build one without wiring a vector database yourself, an AI app builder can handle the setup.
Give your AI app your knowledge, without the plumbing
Starty is our product. It builds AI apps where you add your knowledge as text and files, and Starty runs and meters the AI for you.
- Build AI agents such as a coach, tutor or meal planner from the capability catalog, with no API keys and no model to set up.
- The customer agent sees each user’s plan, payments and AI usage, drafts the reply and waits for your approval.
- No vector database, API keys or embeddings to set up; every AI use is metered against your credits.
- Free to start; paid plans are credit-based, with no limit on the number of products.
Related guides
Where every fact comes from
Official pricing pages, docs, announcements and named studies, checked on November 10, 2026. Where a company doesn’t list something, we say so rather than guess.
Questions people ask
In AI, RAG stands for retrieval-augmented generation. In project management and reporting, RAG usually means red, amber, green, a traffic-light status for whether work is on track.
RAG is an open-book exam for AI. Instead of answering from memory, the AI first looks up the relevant pages in your documents, then answers using them, ideally saying where the answer came from.
Agentic RAG adds an AI agent to the retrieval step. The agent decides which sources to search, checks whether the results answer the question and searches again if not. It is more flexible than plain RAG, but uses more AI calls and costs more.
It reduces them but does not stop them. If retrieval finds the wrong passage, or the model ignores it, the answer can still be wrong. Asking the model to answer only from the sources and to cite them helps people catch mistakes.
They solve different problems. RAG gives the model knowledge it can look up and cite, and is easy to update. Fine-tuning changes the model’s behavior, such as tone or format. Most small AI products start with RAG.
Most RAG setups use one to search by meaning, but not all. Keyword search can work too, and if your knowledge is small enough to fit in the prompt, you may not need retrieval at all.
Enter your email to join Starty. We’ll email you your invite, and you can start building for free.
No. You describe what you want, or fork an idea, and Starty builds it. You review and approve; you never have to open the code.
No. Starty takes no fee on your revenue. Payouts go to your bank account, and Starty earns from plans and credits.