Retrieval-augmented generation (RAG)
Look up relevant documents first, then give them to the model to answer from.
Read first: How a model predicts the next token.
D1 · StoryFor young children and total beginners
If you ask a friend a question about your school, they might check the school website before answering.
RAG makes the AI look things up first instead of guessing from memory.
Like: Checking the website before answering.
D2 · PictureFor curious beginners
Documents are split into chunks and turned into vectors (embeddings) stored in an index.
At question time the question is embedded too; the closest chunks are retrieved, often combined with keyword search and re-ranked, and placed in the prompt.
The model answers from that context, ideally with citations. Quality depends mostly on retrieval: if the right chunk isn't found, the model can't use it.
Answering with retrieval
- Question
- Embed: to a vector
- Search: vector + keyword
- Re-rank: best chunks
- Prompt: chunks + question
- Answer: with citations
Like: An open-book exam.
Where it breaks: In an open-book exam you choose what to read. In RAG the model only sees what retrieval hands it, and it can still blend in its own, possibly wrong, memory.
D3 · PlayFor motivated learners
Before the model sees anything, retrieval decides what it reads. The documents are cut into chunks, every chunk is scored against the question, and only the top few are pasted into the prompt.
Here the score is simple word overlap, weighted so rare words count for more (TF-IDF). Real systems use embeddings, which match meaning rather than exact words, but the shape is the same: rank, cut, paste.
Change the chunk size and how many chunks are kept, then read the box at the bottom. That box is everything the model gets. If the answer is not in it, the model cannot give it.
Try the interactive version in the map.
Things to try
- Pick "Last winter ferry" with chunks of 20 words. Is the time in the context? Now raise the chunk size until it is.
- Pick "North cliffs in June". No passage says "June". Is the answer in the context anyway, and what does the model still have to work out for itself?
- Type your own question using different words from the passages, such as "seabird breeding season". What does word overlap miss that meaning-based search might find?
D5 · FrontierFor experts
Open problems: agentic retrieval where the model decides what to search, long-context models versus retrieval, measuring faithfulness to sources, and poisoning of indexed content.