A retriever is the component of a RAG system that takes a query and returns the passages most likely to contain the answer. Everything downstream depends on it: if the right passage is never retrieved, no reranker or LLM can recover it. The retriever’s job is recall, finding the relevant material; precision can be improved later.
Key takeaways
- Retrievers can be dense, sparse or hybrid.
- Recall@k is the key metric: is the answer in the top k results?
- Retrieve generously, then narrow with reranking.
- Query rewriting often improves retrieval more than changing the model.
- Metadata filtering, including permissions, happens at retrieval time.
A retriever explained in simple terms
A retriever is the research assistant who goes to the archive with your question and comes back with a stack of documents. They do not write the answer; they decide what you get to read. If they bring the wrong stack, the report will be wrong no matter how good the writer is.
Types of retrievers
| Type | How it matches | Strength |
|---|---|---|
| Dense | Embedding similarity (vector search) | Meaning and paraphrase |
| Sparse | Term matching such as BM25 | Exact words, codes, names |
| Hybrid | Both, fused | Best general performance |
| Structured | SQL or metadata queries | Precise filters and numbers |
| Graph | Traversing linked entities | Multi-hop relationships |
| Multi-query | Several rewritten queries in parallel | Ambiguous or broad questions |
Query transformation
The user’s words are often a poor search query. Common improvements:
- Rewriting for context: “does it cover that?” becomes “does the basic plan cover international shipping?” using conversation history.
- Multi-query expansion: generate three phrasings and merge results.
- Decomposition: split a compound question into parts, retrieve for each.
- Hypothetical document embedding (HyDE): have the model draft a plausible answer and embed that, since answers resemble source passages more than questions do.
- Keyword extraction: pull out identifiers for the sparse retriever.
A worked example
Question: “Why was my refund rejected and how long does the appeal take?”
A single-query retriever returns chunks about refunds but nothing about appeals. Decomposing into “reasons refunds are rejected” and “refund appeal processing time”, retrieving for each and merging, returns passages covering both parts, so the model can answer completely.
Measuring retrieval
| Metric | Meaning |
|---|---|
| Recall@k | Share of questions whose correct passage is in the top k |
| Precision@k | Share of retrieved passages that are relevant |
| MRR | How high the first relevant result ranks |
| Hit rate | Whether any relevant passage was retrieved at all |
Build a test set of real questions with their known source passages. Measure recall@20 for the retriever and precision@5 after reranking. Debug retrieval separately from generation.
Tuning a retriever
- Increase k for the retriever; let the reranker cut it down.
- Improve chunking before changing models.
- Add hybrid search if queries contain codes or names.
- Apply metadata filters for recency and access.
- Log queries where no result scores well; they reveal content gaps.
Limitations
- Cannot find information that is not indexed.
- Multi-hop questions need multiple rounds.
- Embedding models vary in quality across languages.
- More results raise cost and can dilute the prompt if not reranked.