Brave Search API

More Search API pages

API glossary

Retrieval-augmented generation (RAG)

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Retrieval-augmented generation (RAG) is a technique that improves a language model’s answers by retrieving relevant information from an external source, and supplying this information to the model as context before it generates a response. Rather than relying exclusively on what it learned in training, the model answers using documents fetched at query time, so its output can be current, specific, and grounded in real sources.

In short: RAG looks up relevant information first, then lets the model generate its answer from that retrieved context, grounding the response in real sources rather than memory alone.

How RAG works

First introduced in a 2020 research paper, RAG splits the work into preparation and query time:

  1. Prepare the knowledge: Source documents are split into chunks, turned into embeddings, and stored in a vector database or search index.
  2. Retrieve: When a query arrives, the system searches the vector database using semantic, keyword, or a combination of both (i.e. hybrid search) and returns the most relevant passages.
  3. Rerank: An optional reranking step reorders the candidates so the best passages come first.
  4. Augment: The top passages are placed into the model’s prompt (and within its context window) alongside the user’s question.
  5. Generate: The LLM writes an answer grounded in the supplied passages, often citing its sources.

The defining idea is the separation of knowledge from reasoning: the model supplies language and reasoning while retrieval supplies the facts, so knowledge can be updated without retraining the model.

RAG vs. fine-tuning and plain LLMs

  • Fine-tuning bakes new knowledge into the model’s weights, which is costly and quickly goes stale; RAG supplies knowledge at query time, so it is cheap to update and can stay current.
  • A plain LLM answers from training data alone, bounded by its knowledge cutoff and prone to hallucinations for many reasons (not only stale training data). RAG can reduce unsupported claims by grounding answers in retrieved sources and enabling citations, but it does not eliminate hallucinations entirely.
  • Standard RAG retrieves once and then generates an answer; agentic search lets the model opt to search repeatedly and refine, a combining technique sometimes called “agentic RAG.”

Where RAG is used

  • Question answering over private data: Assistants that answer from a company’s own documents, wikis, and tickets.
  • AI answer engines: Grounding responses in fresh web results so they stay current and citable.
  • Support and documentation search: Surfacing the right passage in response to a natural-language question.
  • Search APIs as the retrieval layer: For web-scale or up-to-date RAG, a search API is often the “R” in the “RAG” acronym, returning fresh, ranked web results that become the model’s context. The Brave Search API offers endpoints (including an LLM Context service) designed to return pre-extracted, relevance-ranked content for grounding. Coverage, freshness, and latency directly shape answer quality.

RAG matters whenever answers must be accurate, current, or specific to knowledge the model was never trained on—a common requirement in production AI assistants and answer engines.

Chunking, embeddings, vector database, semantic search, hybrid search, reranking, context window, hallucination, knowledge cutoff, large language model (LLM), agentic search, search API.