Glossary

What does RAG stand for in AI? Retrieval-augmented generation, explained

In AI, RAG means retrieval-augmented generation: the AI looks things up in your documents before it answers. Here is how it works, when it helps and when a simpler setup is enough.

Definition · the short answerRetrieval-augmented generation

What does RAG stand for in AI?

In AI, RAG stands for retrieval-augmented generation. Before an AI model answers, the system retrieves relevant information from a knowledge source, such as your documents or help articles, and adds it to the prompt. The model then generates an answer grounded in that information instead of relying only on what it learned in training.

At a glance
RAG stands for
Retrieval-augmented generation
First described
Lewis et al., 2020 paper, accepted at NeurIPS 2020
What it does
Looks up relevant facts and adds them to the prompt before answering
Main benefit
New or private knowledge without retraining the model (AWS)
Small knowledge base
Under 200,000 tokens, Anthropic suggests putting it all in the prompt
01

What does RAG stand for?

In artificial intelligence, RAG stands for retrieval-augmented generation. The name describes the three moves: retrieve relevant information, augment (add it to) the prompt, and generate the answer.

The term comes from a 2020 research paper, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis and colleagues, accepted at the NeurIPS 2020 conference. It described models that combine what a model learned in training with a searchable index of documents, in their case Wikipedia.

Today the idea is everywhere. AWS describes RAG as optimizing a large language model’s output so it references an authoritative knowledge base outside its training data before generating a response. A support bot that answers from your help center, or a coach app that answers from the coach’s own method, is usually doing some form of RAG.

02

How does RAG work?

RAG works in two stages: first you prepare your knowledge so it can be searched, then, for each question, the system finds the best pieces and hands them to the model with the question.

  1. Prepare the knowledge: documents, help articles, notes or product data are split into small chunks and turned into numbers (called embeddings) that capture their meaning, stored in a vector database.
  2. Retrieve: when a question comes in, it is turned into numbers the same way, and the system finds the chunks closest in meaning.
  3. Augment: the best chunks are added to the prompt as context, along with instructions like “answer only from these sources.”
  4. Generate: the model writes the answer, ideally citing which source it used.
  5. Keep it fresh: when documents change, their chunks are updated so answers stay current.

A simple example: a nutrition coach uploads 40 pages of her meal-planning method. A client asks, “Can I swap rice for quinoa on day 3?” The system finds the two paragraphs about grain swaps, adds them to the prompt, and the model answers in the coach’s own rules rather than generic internet advice.

  • Write one clear article per question people actually ask, rather than one long document.
  • Remove old versions of policies and prices so the search can’t find them.
  • Tell the model to answer only from the sources and to say when it doesn’t know.
  • Ask it to name the source it used, so you and your users can check.
  • Test with 20 real questions from customers, including awkward ones, before launch.
03

Why use RAG? The benefits and limits

RAG is the cheapest common way to make an AI model answer from your own, current information. AWS lists the main benefits:

  • Cost: adding new data through retrieval is far cheaper than retraining a model.
  • Current information: you can feed in recent facts, prices, policies or news.
  • Trust: answers can cite their sources, so people can check them.
  • Control: you choose the sources, can restrict them by who is asking, and can fix a wrong answer by fixing the document.

The limits are real too. RAG reduces made-up answers but does not remove them; IBM notes that even a well-built system cannot fully remove the risk of hallucinations. And RAG is only as good as its retrieval: if the search pulls the wrong paragraph, the model answers confidently from the wrong source.

04

RAG vs fine-tuning: which do you need?

Use RAG to give a model knowledge; use fine-tuning to change how it behaves. IBM explains that RAG connects a model to an organization’s own data at question time, while fine-tuning means retraining a pretrained model on a smaller, focused set of training data.

RAGFine-tuning
What changesWhat the model can see when it answersThe model itself
Best forFacts, documents, policies, product dataTone, format, specialist tasks and vocabulary
Updating knowledgeEdit the document; the next answer uses itTrain again
Can cite sourcesYesNo, the knowledge is blended into the model
What you needYour documents and a search setupLabeled examples and compute to train
RAG vs fine-tuning

For most small AI products, RAG plus good instructions is enough. Fine-tuning makes sense later, if you have many examples of exactly how answers should look.

05

What is agentic RAG?

Agentic RAG is RAG run by an AI agent that decides where to look, whether the results are good enough and whether to search again. IBM defines it as the use of AI agents to facilitate retrieval-augmented generation.

Traditional RAGAgentic RAG
SourcesUsually one knowledge baseSeveral knowledge bases and tools, such as web search
SearchOne lookup per questionPlans, searches, checks and searches again if needed
Checks its resultsNoYes, it can refine earlier work
Cost and speedCheaper and fasterMore AI calls, more tokens, slower
Traditional RAG vs agentic RAG

IBM is candid about the downsides: more agents mean higher costs, and agents are not always reliable. It suggests agentic RAG fits best when a question needs several data sources. For how agents plan and loop, see agentic workflows.

06

When don’t you need RAG?

If your knowledge is small, you may not need retrieval at all. In its contextual retrieval write-up (September 2024), Anthropic suggests that for a knowledge base under 200,000 tokens, roughly 500 pages, you can include the whole thing in the prompt, and prompt caching makes that faster and cheaper.

When you do need RAG, small changes to retrieval matter. In Anthropic’s tests, adding context to each chunk cut failed retrievals by 35%; adding keyword search on top brought the cut to 49%, and adding a reranking step brought it to 67%.

The takeaway for a small product: start simple. Put your rules and a short knowledge base straight into the instructions, see where answers go wrong, and add retrieval only when the material outgrows the prompt or changes too often to paste in by hand.

Your knowledgeSimplest approach
A one-page FAQ or rule listPut it in the instructions
A short handbook or a few dozen help articlesInclude it all in the prompt, with caching
Thousands of pages, or data that changes dailyRAG
Several systems plus the webAgentic RAG
Pick the simplest setup that works
07

What are real examples of RAG?

RAG shows up wherever an AI has to answer from specific material rather than general knowledge:

  • Customer support agents that answer from your help articles and refund policy.
  • Expert tools: a fitness coach’s programs, a tutor’s course notes or a dietitian’s rules, turned into an assistant for their audience.
  • Document Q&A: “What does my lease say about pets?” over an uploaded file.
  • Internal search: staff asking questions across company docs.

Owning that knowledge is also what makes an AI product hard to copy; see what an AI wrapper is and when it becomes a real business. To build one without wiring a vector database yourself, an AI app builder can handle the setup.

Our product

Give your AI app your knowledge, without the plumbing

Starty is our product. It builds AI apps where you add your knowledge as text and files, and Starty runs and meters the AI for you.

  • Build AI agents such as a coach, tutor or meal planner from the capability catalog, with no API keys and no model to set up.
  • The customer agent sees each user’s plan, payments and AI usage, drafts the reply and waits for your approval.
  • No vector database, API keys or embeddings to set up; every AI use is metered against your credits.
  • Free to start; paid plans are credit-based, with no limit on the number of products.
Keep reading

Related guides

Sources

Where every fact comes from

Official pricing pages, docs, announcements and named studies, checked on November 10, 2026. Where a company doesn’t list something, we say so rather than guess.

FAQ

Questions people ask