Enhancing Language Models with RAG: Best Practices and Benchmarks (2020–2026)

Retrieval-Augmented Generation, commonly known as RAG, has grown from a small group of retrieval-aware language-model architectures into one of the most active areas of applied natural-language processing.

The central idea is simple: instead of relying entirely on knowledge stored inside a language model's parameters, a RAG system searches an external knowledge source and supplies the retrieved evidence to the model before or during generation.

This external memory can help a language model:

• Access private, specialized, or recently updated information.
• Reduce dependence on facts memorized during pretraining.
• Attach sources and provenance to generated claims.
• Update knowledge without retraining the full model.
• Support domain-specific question answering.
• Separate factual evidence from generative reasoning.

However, retrieval does not automatically make a model reliable. A RAG system can still fail when it retrieves irrelevant passages, misses the required evidence, selects outdated documents, follows malicious instructions embedded in retrieved text, or generates claims that are not supported by the retrieved sources.

This article traces the major development of RAG from 2020 through July 28, 2026. It focuses on influential general-purpose architectures, retrieval techniques, training strategies, evaluation frameworks, benchmarks, security findings, and production lessons. It does not attempt to enumerate every domain-specific application paper, since the complete literature now contains thousands of works.

The evolution of RAG at a glance

2020 — Foundations of modern RAG

Modern RAG emerged from the convergence of dense information retrieval and pretrained generative models.

Dense Passage Retrieval

Dense Passage Retrieval, or DPR, demonstrated that open-domain question answering could use learned dense representations instead of depending exclusively on lexical retrieval systems such as BM25.[3]

DPR uses two encoders:

• A question encoder.
• A passage encoder.

The question and passages are mapped into a shared vector space. Retrieval is then performed using vector similarity and nearest-neighbour search.

DPR became the retriever used in the original RAG architecture and strongly influenced the vector-search systems later adopted in production RAG applications.

REALM

REALM — Retrieval-Augmented Language Model Pre-Training integrated retrieval directly into language-model pretraining.[2]

Instead of treating retrieval as an independent component added after training, REALM treats the retrieved document as a latent variable and trains retrieval using a masked-language-model objective.

A terminology distinction is important here: REALM jointly trains a retriever and a knowledge-augmented language model or encoder. It is not a sequence-to-sequence generator architecture in the same sense as the original RAG model.

REALM established one of the two foundational architectural families:

1. Add retrieval to an already pretrained generator.
2. Train the model from the beginning to use external retrieval.

REALM was published at ICML 2020.

Retrieval-Augmented Generation

Lewis and colleagues introduced the name Retrieval-Augmented Generation in their NeurIPS 2020 paper.[1]

The architecture combined:

• A DPR-based dense retriever.
• A Wikipedia passage index.
• A BART sequence-to-sequence generator.
• Marginalization over several retrieved documents.

The paper proposed two variants:

RAG-Sequence

One retrieved document conditions the complete generated sequence.

RAG-Token

Different retrieved documents can influence different generated tokens.

The paper established a useful distinction between:

• Parametric memory — knowledge stored in the model's weights.
• Non-parametric memory — knowledge stored in an external corpus or index.

This division remains central to modern RAG design.

Fusion-in-Decoder

Fusion-in-Decoder, or FiD, was first released in 2020 and published at EACL 2021.[4]

FiD encodes each retrieved passage independently. The decoder then attends across the encoded representations of all passages while generating the answer.

This avoids forcing every retrieved document into one flat encoder sequence and allows the system to process a larger evidence set.

FiD became one of the most influential baselines for retrieval-grounded open-domain question answering.

2021—Stronger retrieval, evidence fusion, and benchmark standardization

RAG research in 2021 began separating retrieval quality, evidence integration, provenance, and answer generation into independently measurable problems.

KILT

The Knowledge Intensive Language Tasks benchmark, or KILT, placed several knowledge-intensive NLP tasks against a common Wikipedia snapshot.[5]

Its tasks included:

• Open-domain question answering.
• Fact verification.
• Entity linking.
• Slot filling.
• Knowledge-grounded dialogue.

KILT evaluated not only whether a system produced the correct answer, but also whether it retrieved the correct provenance.

This made KILT an important early end-to-end benchmark for retrieval-grounded language models.

BEIR

BEIR introduced a heterogeneous benchmark for zero-shot information retrieval.[6]

Its original collection covered 18 datasets across tasks such as:

• Question answering.
• Fact checking.
• Citation prediction.
• Duplicate-question retrieval.
• Bio-medical retrieval.
• Argument retrieval.

BEIR demonstrated a practical lesson that remains important today:

A retriever performing well on one benchmark or domain may generalize poorly to another.

Sparse, dense, and late-interaction retrieval methods have different strengths. No single retrieval model is universally optimal.

Retrieval augmentation and hallucination

Shuster and colleagues studied retrieval-augmented conversational systems and showed that retrieval-in-the-loop architectures could reduce knowledge hallucination while preserving conversational ability.[7]

This helped establish retrieval as more than a mechanism for accessing additional information. It also became a potential intervention for factuality.

Later research, however, showed that retrieval reduces hallucination only when the evidence is relevant, trustworthy, sufficient, and correctly used by the generator.

ColBERTv2

ColBERTv2 improved neural retrieval through lightweight late interaction.[8]

Instead of compressing an entire query or document into a single vector, ColBERT preserves token-level representations and performs efficient interactions between query and document tokens.

This provides more expressive matching than a standard bi-encoder while remaining more scalable than applying a full cross-encoder to every document in a large corpus.

2022 — Retrieval-augmented pretraining at scale

By 2022, researchers were asking whether retrieval could partially substitute for parameter growth.

RETRO

DeepMind's Retrieval-Enhanced Transformer, or RETRO, retrieved neighbouring chunks from a database containing approximately two trillion tokens.[9]

Its 7.5-billion-parameter model achieved performance comparable to much larger non-retrieval models on parts of the Pile benchmark while using approximately 25 times fewer parameters.

The important conclusion was not that retrieval always replaces scale. Rather, RETRO showed that model capability can depend on both:

• Parametric capacity.
• Access to a large non-parametric memory.

A smaller model with a sufficiently large external memory can compete with larger models whose knowledge must be stored entirely in their weights.

Atlas

Atlas combined retrieval-aware pretraining with a Fusion-in-Decoder architecture and focused on few-shot knowledge-intensive learning.[10]

Using only 64 training examples, Atlas achieved more than 42% accuracy on Natural Questions and outperformed a 540-billion-parameter model despite containing approximately 50 times fewer parameters.

Atlas also demonstrated one of RAG's most important operational properties:

The knowledge index can be updated without fully retraining the generator.

Contriever

Contriever introduced unsupervised dense information retrieval using contrastive learning over unlabelled text.[11]

This was especially important for domains where large collections of query-document relevance labels are unavailable.

Contriever showed that a retriever could learn transferable semantic retrieval behaviour without conventional supervised relevance annotations.

MuRAG

MuRAG extended retrieval-augmented generation beyond text by allowing a system to retrieve from a multimodal memory containing images and text passages.[12]

It was an early foundation for later document and multimodal RAG systems that must reason across:

• Text.
• Figures.
• Screenshots.
• Diagrams.
• Page images.
• Visual evidence.

MTEB

The Massive Text Embedding Benchmark, or MTEB, expanded embedding evaluation across retrieval, reranking, classification, clustering, semantic similarity, and other tasks.[13]

Like BEIR, MTEB demonstrated that embedding quality is task- and domain-dependent.

Selecting an embedding model based only on popularity or one leaderboard is therefore risky.

2023 — Iterative retrieval, self-reflection, and dedicated RAG evaluation

The widespread adoption of instruction-tuned large language models changed RAG research in 2023.

Retrieval became more dynamic, and researchers began developing evaluation systems specifically for RAG pipelines.

HyDE

Hypothetical Document Embeddings, or HyDE, asks a language model to generate a hypothetical document that might answer the user's question.[14]

The hypothetical document is then embedded and used as the retrieval query.

Even when the generated document contains imperfect details, its semantic structure can retrieve real documents that are more relevant than those found from a short or ambiguous original query.

REPLUG

REPLUG showed that retrieval augmentation could be applied to a frozen or black-box language model without modifying the generator's parameters.[15]

The retriever is trained using signals produced by the language model, while the generator remains frozen.

This made retrieval augmentation more applicable to models available only through hosted or black-box interfaces.

REPLUG first appeared as a 2023 preprint and was subsequently published at NAACL 2024.

FLARE

Forward-Looking Active Retrieval-Augmented Generation, or FLARE, retrieves information during generation rather than retrieving only once before generation begins.[16]

The model predicts a future sentence. When confidence in the predicted tokens is low, the system uses the predicted content to retrieve additional evidence before continuing.

FLARE was published at EMNLP 2023.

Its primary contribution is not a universal percentage improvement. Its importance lies in demonstrating that retrieval can be an adaptive, mid-generation action.

IRCoT

Interleaving Retrieval with Chain-of-Thought Reasoning, or IRCoT, alternates retrieval and reasoning.[17]

Intermediate reasoning steps identify missing information. That information becomes the basis for the next retrieval query.

This is particularly valuable for multi-hop questions whose complete answer cannot be found in one passage.

Iter-RetGen

Iterative Retrieval-Generation, or Iter-RetGen, uses an initial generated response to construct a more informative retrieval query.[18]

The system then retrieves additional evidence and generates a revised answer.

This demonstrated how retrieval and generation can progressively improve one another rather than operating as fixed independent stages.

Self-RAG

Self-RAG trained a language model to retrieve, generate, and critique its answers through special reflection tokens.[19]

The correct author list is:

• Akari Asai.
• Zeqiu Wu.
• Yizhong Wang.
• Avirup Sil.
• Hannaneh Hajishirzi.

The model learns to assess:

• Whether retrieval is necessary.
• Whether a retrieved passage is relevant.
• Whether a generated statement is supported.
• Whether the response is useful.

Self-RAG first appeared as a 2023 preprint and was published at ICLR 2024.

It represents one of the clearest transitions from static RAG toward retrieval-aware reasoning agents.

RAGAS

RAGAS introduced a set of reference-free or weakly supervised metrics for RAG evaluation.[20]

Common dimensions include:

• Faithfulness.
• Answer relevance.
• Context precision.
• Context recall.

The work first appeared as a 2023 preprint and was published as an EACL 2024 demonstration paper.

RAGAS made rapid development-stage evaluation more accessible, although LLM-based evaluation still needs calibration against human judgments.

ARES

ARES uses synthetic examples, lightweight judges, and Prediction-Powered Inference to evaluate RAG systems with statistical confidence estimates.[21]

It evaluates dimensions such as:

• Context relevance.
• Answer relevance.
• Answer faithfulness.

A key bibliographic correction is important:

ARES is a NAACL 2024 paper, not an ICLR paper and not a 2025 publication.

Citation evaluation with ALCE

ALCE evaluated long-form answers containing citations.[22]

It distinguishes among:

• Answer correctness.
• Citation correctness.
• Citation completeness.
• Overall citation quality.

This is important because the presence of citation markers does not prove that a cited document actually supports the associated claim.

A common vocabulary for RAG

The survey by Gao and colleagues organized RAG development into three broad stages.[23]

Naive RAG

Retrieve documents and place them directly into the prompt.

Advanced RAG

Improve indexing, query transformation, retrieval, reranking, context selection, and generation.

Modular RAG

Construct flexible retrieval and reasoning workflows from specialized components.

This vocabulary became widely used when discussing the evolution of RAG systems.

2024 — Adaptive, corrective, hierarchical, graph-based, and modular RAG

In 2024, research moved beyond the basic retrieve-then-generate pipeline.

Systems began routing queries, assessing retrieval quality, constructing hierarchical memories, reasoning over graphs, compressing evidence, and evaluating security.

GraphRAG

Microsoft Research's GraphRAG extracts entities and relationships from a corpus, constructs a graph, identifies communities, and produces summaries of those communities.[24]

For global questions, the system retrieves relevant community reports and combines partial answers into a final response.

Traditional vector retrieval is often strongest when a question is semantically close to a specific passage. GraphRAG targets a different class of queries:

• Global themes.
• Distributed patterns.
• Entity relationships.
• Corpus-level sensemaking.
• Questions whose evidence is spread across many documents.

RAPTOR

RAPTOR recursively embeds, clusters, and summarizes text chunks into a tree.[25]

Leaf nodes retain detailed information, while higher levels represent increasingly abstract summaries.

Retrieval can operate across several tree levels, helping the system answer both:

• Detailed factual questions.
• High-level thematic questions.

When combined with GPT-4, RAPTOR improved the previous best result on the QuALITY benchmark by 20 percentage points in absolute accuracy.

Adaptive-RAG

Adaptive-RAG routes questions according to estimated complexity.[26]

A simple factual question may require no retrieval or one retrieval step. A complex multi-hop question may require iterative retrieval.

This made explicit an increasingly important design principle:

The most expensive retrieval strategy should not be applied to every query.

Corrective RAG

Corrective Retrieval-Augmented Generation, or CRAG, was introduced by Yan, Gu, Zhu, and Ling.[27]

The system adds a retrieval evaluator that estimates whether the retrieved evidence is reliable.

Depending on the evaluation, CRAG can:

• Accept the retrieved evidence.
• Decompose and refine partially relevant evidence.
• Trigger broader search, including web search.
• Replace unsuccessful retrieval results.

CRAG addresses a central weakness of standard RAG: generation quality is constrained by retrieval quality.

DRAGIN

DRAGIN — Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models was published at ACL 2024.[28]

The correct authors are:

• Weihang Su.
• Yichen Tang.
• Qingyao Ai.
• Zhijing Wu.
• Yiqun Liu.

DRAGIN addresses two questions:

1. When should retrieval happen?
2. What should be retrieved?

Its retrieval trigger considers token uncertainty, semantic importance, and influence on later generation. Once retrieval is triggered, the method uses self-attention over the preceding context to construct a query that reflects the model's broader information need.

It is therefore more precise to describe DRAGIN as an information-need-aware retrieval framework than simply as a system that reacts to "entropy and attention spikes."

RAFT

RAFT — Retrieval-Augmented Fine-Tuning trains a model on questions accompanied by relevant documents and distractor documents.[29]

The model learns to:

• Identify the useful evidence.
• Ignore irrelevant context.
• Reason from the relevant document.
• Produce an evidence-grounded answer.

RAFT is especially relevant when a model must operate inside a specialized domain corpus containing many plausible distractors.

RA-DIT

RA-DIT — Retrieval-Augmented Dual Instruction Tuning separately tunes the retriever and generator through retrieval-aware instruction tuning.[30]

It provides a way to add stronger retrieval behaviour to an existing language model without requiring complete retrieval-augmented pretraining.

HippoRAG

HippoRAG combines language models, knowledge graphs, and Personalized PageRank to approximate associative human memory.[31]

The graph supports traversal among related concepts, making the system useful for:

• Multi-hop retrieval.
• Entity relationships.
• Associative recall.
• Questions whose evidence is indirectly connected.

RankRAG

RankRAG instruction-tunes one language model to perform both context ranking and answer generation.[32]

The work reinforces the importance of reranking:

The passage closest to a query in embedding space is not always the passage most useful for answering it.

RECOMP

RECOMP compresses retrieved documents into concise, query-focused representations.[33]

It can also produce an empty result when retrieval does not provide useful evidence.

This helps control:

• Context length.
• Irrelevant content.
• Generation cost.
• Noise introduced by excessive passages.

Chain-of-Note

Chain-of-Note generates reading notes from retrieved documents before composing the final answer.[34]

It also encourages the model to acknowledge when neither the retrieved evidence nor its internal knowledge is sufficient.

This made abstention and evidence inspection more explicit parts of the generation process.

Modular RAG

The Modular RAG framework decomposes RAG systems into configurable modules and operators.[35]

Instead of treating RAG as one linear sequence, it describes patterns including:

• Linear pipelines.
• Conditional routing.
• Branching retrieval.
• Iterative loops.
• Scheduling.
• Evidence fusion.
• Memory integration.

This conceptualization closely resembles the architecture of modern production RAG and agentic search systems.

Searching for best practices in RAG

Wang and colleagues published Searching for Best Practices in Retrieval-Augmented Generation at EMNLP 2024.[58]

This work systematically examined combinations across:

• Indexing.
• Query transformation.
• Retrieval.
• Reranking.
• Context selection.
• Generation.
• Multimodal retrieval.

It is a separate publication from the later COLING 2025 paper titled Enhancing Retrieval-Augmented Generation: A Study of Best Practices.

Both are relevant and should not be conflated.

Evaluation and benchmark expansion

Several important benchmarks and diagnostic frameworks appeared during this period.

CRUD-RAG

CRUD-RAG evaluates create, read, update, and delete-style knowledge tasks, with an initial focus on Chinese-language data.[36]

RGB

RGB — Benchmarking Large Language Models in Retrieval-Augmented Generation evaluates four core abilities:[41]

• Noise robustness.
• Negative rejection.
• Information integration.
• Counterfactual robustness.

RAGBench

RAGBench introduced more than 100,000 examples across several industry domains and provided explainable evaluation signals for RAG systems.[37]

MultiHop-RAG

MultiHop-RAG evaluates questions requiring evidence from multiple documents.[38]

RAGChecker

RAGChecker provides fine-grained diagnostics separating retriever and generator failures.[39]

RAGTruth

RAGTruth provides manually annotated hallucinations found in retrieval-grounded responses.[40]

BERGEN

BERGEN provides an end-to-end benchmarking library intended to standardize RAG experiments across retrievers, rerankers, generators, datasets, and metrics.[64]

Retrieval poisoning

PoisonedRAG demonstrated that attackers can insert malicious documents into a knowledge base so they are retrieved for targeted questions and influence the generated answer.[42]

The work established that RAG introduces a security boundary between:

• The language model.
• The retrieval system.
• The external knowledge source.

Retrieved documents must therefore be treated as untrusted data, not as instructions.

2025 — Security, memory, efficiency, and search agents

By 2025, RAG research increasingly focused on autonomous search, security, long-term memory, long-context competition, and production efficiency.

SafeRAG

SafeRAG introduced a dedicated benchmark for evaluating security vulnerabilities in RAG components.[43]

It studied four attack categories:

• Silver noise.
• Inter-context conflict.
• Soft advertising.
• White denial-of-service.

Tests across representative retrievers, filters, and generators found substantial vulnerability to manipulated external knowledge.

SafeRAG helped move RAG security beyond conventional hallucination testing toward deliberate adversarial evaluation.

Agentic RAG

The Agentic Retrieval-Augmented Generation survey organized systems that combine retrieval with planning, reflection, tool use, and multi-step decision-making.[44]

The confirmed authors are:

• Aditi Singh.
• Abul Ehtesham.
• Saket Kumar.
• Tala Talaei Khoei.

In an agentic RAG architecture, the model may:

1. Analyze the question.
2. Select a retrieval tool.
3. Search multiple sources.
4. Inspect the evidence.
5. Reformulate the query.
6. Retrieve again.
7. Verify the emerging answer.
8. Stop or abstain.

This can improve complex research and multi-hop question answering, but it also introduces additional latency, cost, and failure modes.

Vague attributions such as "Du, S., Brown, A., et al." should not be used without a specific paper title and stable author list.

LightRAG

LightRAG combines vector representations with a graph-based index and performs retrieval at both low and high semantic levels.[45]

It also supports incremental updates, addressing one practical limitation of graph systems that may otherwise require expensive graph reconstruction after corpus changes.

LightRAG first appeared as a 2024 preprint and was published in Findings of EMNLP 2025.

HippoRAG 2

HippoRAG 2 extended the original graph-memory architecture with stronger passage integration and improved interaction between non-parametric memory and the language model.[46]

Its goal was to preserve the factual recall of conventional vector RAG while improving associative and sense-making tasks that benefit from graph traversal.

Speculative RAG

Speculative RAG uses smaller specialist models to draft several answers from different subsets of retrieved evidence.[47]

A larger generalist model then verifies and selects among the drafts.

The authors reported improvements of up to 12.97% in accuracy and reductions of up to 50.83% in latency on PubHealth compared with their conventional RAG baseline.

These are maximum reported results, not universal gains across every benchmark, model, or infrastructure configuration.

Sufficient Context

Sufficient Context distinguishes between:

• Retrieving a relevant document.
• Retrieving enough evidence to answer the question.

A document can be topically relevant without containing the facts required for a supported answer.

This distinction is essential when designing abstention systems.

SimRAG

SimRAG creates synthetic queries and answers from an unlabelled target-domain corpus and uses them to adapt retrieval and generation.[49]

This provides a path for domain adaptation when an organization has many documents but little manually labelled question-answer data.

Search-R1

Search-R1 applies reinforcement learning to teach a language model to interleave reasoning with search operations.[50]

Instead of following a fixed retrieval workflow, the model learns:

• When to search.
• How to formulate a search query.
• How to incorporate search results.
• When sufficient evidence has been collected.

This is an important transition from pipeline-based RAG toward learned search policies.

LongCite

LongCite focuses on fine-grained citation generation for long-context question answering.[51]

It addresses a frequent production failure in which a system provides broadly relevant sources but does not identify which passage supports each individual claim.

RAG or long context?

The LaRA benchmark compared retrieval-augmented systems with long-context language models.[52]

Its main conclusion is that neither approach is universally superior.

Performance depends on:

• Corpus size.
• Evidence density.
• Query complexity.
• Retrieval quality.
• Context length.
• Model capability.
• Computational cost.

For a bounded document collection, directly reading the relevant documents in a long context may outperform a retrieval pipeline.

For large, private, frequently changing, or permission-controlled corpora, RAG remains more scalable and manageable.

Empirical best-practice studies

A COLING 2025 study titled Enhancing Retrieval-Augmented Generation: A Study of Best Practices examined practical design choices including:[53]

• Model size.
• Prompt design.
• Chunk size.
• Retrieval stride.
• Knowledge-base size.
• Query expansion.
• Multilingual retrieval.
• Sentence-level evidence selection.

The broader lesson is that RAG quality depends on interactions among system components rather than on one isolated technique.

2026 — Adaptive routing, hierarchical retrieval interfaces, and renewed GraphRAG scrutiny

The 2026 literature is still developing.

The works below were publicly available by July 28, 2026. Some remain recent preprints whose conclusions may change after further peer review, replication, or benchmark expansion.

A-RAG

A-RAG — Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces exposes retrieval tools directly to a language-model agent.[54]

The confirmed authors are:

• Mingxuan Du.
• Benfeng Xu.
• Chiwei Zhu.
• Shaohan Wang.
• Pengyu Wang.
• Xiaorui Wang.
• Zhendong Mao.

A-RAG provides three primary interfaces:

• Keyword search.
• Semantic search.
• Chunk reading.

Instead of receiving a predefined list of passages, the model decides how to navigate the corpus and when more detailed retrieval is needed.

This treats retrieval as an interactive environment rather than as one fixed function call.

RAGRouter-Bench

RAGRouter-Bench evaluates routing among different RAG paradigms.[55]

It contains:

• 7,727 queries.
• 21,460 documents.
• Several knowledge domains.
• Three canonical query types.
• Five standardized RAG paradigms.
• Measurements of both answer quality and resource use.

The benchmark frames routing as a query-corpus compatibility problem.

A router must consider not only the wording of the question, but also:

• Corpus size.
• Corpus structure.
• Evidence distribution.
• Query type.
• Retrieval cost.
• Expected benefit from advanced mechanisms.

The benchmark finds that no single RAG paradigm is universally optimal.

Do we still need GraphRAG?

The preprint Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems compares dense RAG and GraphRAG under agentic search settings.[56]

Its results suggest that agentic search can substantially improve dense RAG and narrow the gap to GraphRAG on some tasks.

This does not establish that graphs are obsolete.

Precomputed graph structure can remain useful when:

• Relationships must be queried repeatedly.
• Global corpus summaries are required.
• Entity provenance must be explicit.
• Inference latency must be predictable.
• The corpus contains stable relational knowledge.

The likely direction is not simply "GraphRAG versus agentic RAG."

A more realistic architecture will route or combine:

• Lexical retrieval.
• Dense vector retrieval.
• Late-interaction retrieval.
• Graph retrieval.
• Hierarchical retrieval.
• Long-context reading.
• Agentic search.

Is GraphRAG needed?

A second 2026 study, Is GraphRAG Needed? From Basic RAG to Graph-/Agentic Solutions with Context Optimization, compares multiple standardized RAG scenarios.[65]

The work reports that expanded retrieval does not always produce proportional generation gains.

This reinforces a retrieval-generation gap:

Better retrieval-oriented metrics do not automatically guarantee better final answers.

The study also explores context optimization for graph and agentic systems and reports reduced token usage in its tested settings.

MegaRAG

MegaRAG combines multimodal retrieval with a knowledge graph.[57]

It incorporates visual information into:

• Knowledge-graph construction.
• Retrieval.
• Answer generation.

This direction is important for scientific, industrial, and enterprise documents in which knowledge is divided among:

• Text.
• Figures.
• Tables.
• Diagrams.
• Images.
• Page layout.

The emerging 2026 pattern

It is too early to claim that adaptive RAG has become a universal industry standard.

A more defensible conclusion is that adaptive routing is an emerging research direction.

The direction is nevertheless clear:

• Simple factual questions should use inexpensive retrieval.
• Some questions may require no retrieval.
• Multi-hop questions may require iterative or agentic search.
• Relationship-heavy questions may benefit from graph retrieval.
• Bounded high-density documents may be better handled through long context.
• Multimodal documents require structure-aware retrieval.
• High-risk questions require verification, provenance, and calibrated abstention.

The central question is shifting from:

How should we build one RAG pipeline?

to:

How should a system choose, combine, and verify different knowledge-access strategies?

Core benchmarks and datasets

RAG evaluation often uses both dedicated RAG benchmarks and older question-answering datasets.

The older datasets remain important because they test retrieval, multi-hop reasoning, and evidence composition.

Notable RAG systems and frameworks

Unsupported universal gain claims should be avoided.

The effect of a RAG method varies across:

• Dataset.
• Generator.
• Retriever.
• Prompt.
• Corpus.
• Metric.
• Index configuration.
• Available compute.

A safer comparison is shown below.

Best practices for building RAG systems

The research from 2020 through 2026 supports several practical principles.

1. Begin with the knowledge base

A powerful generator cannot compensate for a poorly constructed corpus.

Each document should preserve:

• Stable source identifiers.
• Titles.
• Section hierarchy.
• Page or paragraph references.
• Publication date.
• Effective date.
• Expiration date.
• Version number.
• Source authority.
• Access-control metadata.
• Content hash.
• Replacement or supersession relationships.

Duplicate and near-duplicate documents should be identified before indexing.

Otherwise, several top-ranked passages may appear diverse while repeating the same source.

2. Preserve document structure

Fixed-size character splitting is convenient but often destroys meaning.

Prefer structure-aware segmentation based on:

• Sections.
• Paragraphs.
• Sentences.
• Lists.
• Tables.
• Figure captions.
• Parent-child relationships.
• Page layout.

For long documents, store both:

• Small child chunks for precise retrieval.
• Larger parent sections for interpretation.

RAPTOR and other hierarchical systems demonstrate why retrieval at multiple abstraction levels can be useful.[25]

3. Evaluate sparse, dense, and hybrid retrieval

A production experiment should normally include:

1. BM25 or another sparse lexical baseline.
2. A dense embedding retriever.
3. Hybrid sparse-dense retrieval.
4. An optional late-interaction method such as ColBERT.
5. A cross-encoder or LLM reranker.

BEIR and MTEB demonstrate that retrieval quality varies substantially across domains and tasks.[6][13]

Hybrid retrieval is often valuable because:

• Sparse systems preserve exact names, identifiers, and rare terminology.
• Dense systems handle paraphrases and semantic similarity.
• Late-interaction methods preserve fine-grained token matching.
• Rerankers estimate usefulness more precisely over a smaller candidate set.

4. Retrieve broadly and rerank narrowly

A common high-quality pipeline is:

User query
    ↓
Query analysis, expansion, or decomposition
    ↓
Retrieve 30–100 candidates
    ↓
Apply tenant, permission, metadata, and date filters
    ↓
Rerank the strongest candidates
    ↓
Remove duplicates and improve source diversity
    ↓
Select a small evidence set
    ↓
Generate and verify the answer

The exact values must be measured for the application.

Sending every retrieved candidate to the generator increases:

• Latency.
• Cost.
• Duplication.
• Conflict.
• Exposure to malicious or irrelevant text.

5. Use query transformation selectively

Useful query transformations include:

• HyDE-generated hypothetical documents.
• Multi-query expansion.
• Query decomposition.
• Entity extraction.
• Step-back questions.
• Iterative query rewriting.
• Generated subquestions for multi-hop tasks.

Transformations can improve recall, but they can also introduce concepts absent from the original question.

The transformed queries should therefore remain traceable to the user's original request.

6. Do not retrieve for every request

Retrieval creates cost, latency, and new opportunities for error.

A router should be able to choose among:

• No retrieval.
• One-step retrieval.
• Iterative retrieval.
• Graph retrieval.
• Agentic search.
• Long-context reading.
• Retrieval followed by independent verification.

Adaptive-RAG, Self-RAG, A-RAG, and RAGRouter-Bench all reflect this movement toward conditional retrieval.[19][26][54][55]

7. Treat reranking as an independent component

The nearest passage in embedding space is not always the most useful evidence.

Evaluate:

• Cross-encoder rerankers.
• Late-interaction models.
• LLM-based ranking.
• Diversity-aware reranking.
• Temporal reranking.
• Source-authority reranking.

Retrieval recall and final evidence quality should be reported separately.

8. Control context size

More context is not automatically better.

Excessive context can introduce:

• Duplicate evidence.
• Irrelevant passages.
• Conflicting document versions.
• Counterfactual claims.
• Prompt injection.
• High token cost.
• Evidence-position effects.
• Reduced attention to the strongest source.

Query-focused extraction, RECOMP-style compression, or structured evidence records can reduce this problem.[33]

Compression should preserve direct links to original text because an abstractive summary may remove qualifications or introduce unsupported claims.

9. Require grounded generation and abstention

The generator should be instructed to:

• Use retrieved evidence as its primary factual basis.
• Cite the evidence supporting each important claim.
• Separate sourced facts from inference.
• Identify conflicts among sources.
• Mention relevant publication or effective dates.
• State when evidence is insufficient.
• Refuse to invent missing details.
• Ignore operational instructions embedded in retrieved documents.

Sufficient Context and Chain-of-Note show why a system must distinguish between relevant evidence and sufficient evidence.[34][48]

10. Fine-tune for evidence use only when necessary

Fine-tuning may help when a model must learn:

• Domain vocabulary.
• A specialized answer structure.
• Citation placement.
• Distractor rejection.
• Evidence-grounded reasoning.
• Abstention.
• Retrieval decisions.
• Tool-selection policies.

RAFT, RA-DIT, and SimRAG offer different approaches.[29][30][49]

Fine-tuning should not be used merely to memorize documents that can be maintained in an updateable retrieval index.

11. Treat retrieved content as untrusted

Retrieved documents can contain malicious or misleading instructions.

A secure RAG system should include:

• Trusted-source policies.
• Provenance verification.
• Permission filtering before retrieval.
• Tenant isolation.
• Separation of retrieved text from system instructions.
• Prompt-injection detection.
• Corpus-poisoning detection.
• Source-diversity checks.
• Immutable ingestion logs.
• Corpus versioning.
• Adversarial evaluation.
• Independent corroboration for high-risk claims.

PoisonedRAG and SafeRAG show that retrieval infrastructure creates a genuine security surface.[42][43]

12. Version knowledge and test temporal correctness

A production record should ideally include fields such as:

document_id
document_version
published_at
effective_from
effective_until
ingested_at
source_authority
content_hash
supersedes_document_id

A semantically relevant passage can still be wrong if it represents an outdated law, product version, organizational policy, or scientific conclusion.

13. Evaluate the components separately

A final answer can fail because:

1. The required evidence was absent from the corpus.
2. The retriever failed to find it.
3. The reranker discarded it.
4. Context processing removed it.
5. The generator ignored it.
6. The generator contradicted it.
7. The citation pointed to the wrong passage.
8. The system should have abstained but answered.
9. A router selected the wrong retrieval paradigm.
10. A malicious document influenced generation.

End-to-end accuracy alone cannot distinguish among these failures.

Recommended RAG metrics

Retrieval metrics

• Recall@k.
• Precision@k.
• Mean Reciprocal Rank.
• nDCG@k.
• Evidence coverage.
• Retrieval diversity.
• Source-authority score.
• Temporal correctness.
• Duplicate-evidence rate.
• Multi-hop evidence recall.
• Permission-filter correctness.

Generation metrics

• Exact Match.
• Token-level F1.
• Semantic answer correctness.
• Faithfulness.
• Completeness.
• Contradiction rate.
• Unsupported-claim rate.
• Citation correctness.
• Citation completeness.
• Appropriate abstention.
• Conflict acknowledgment.

End-to-end metrics

• Answer accuracy when relevant evidence is retrieved.
• Accuracy under irrelevant retrieval.
• Performance on unanswerable questions.
• Multi-document reasoning accuracy.
• Robustness to conflicting documents.
• Robustness to outdated information.
• Robustness to corpus poisoning.
• Performance across languages.
• Performance across domains.
• Routing-decision accuracy.

Operational metrics

• Retrieval latency.
• Reranking latency.
• Time to first token.
• End-to-end p50 latency.
• End-to-end p95 latency.
• End-to-end p99 latency.
• Number of retrieval calls.
• Search depth.
• Tokens processed per answer.
• Cost per successful answer.
• Indexing throughput.
• Cache-hit rate.
• GPU use.
• Memory use.
• Corpus-update time.

A practical production architecture

A mature RAG system may use a staged architecture similar to the following:

User request
    ↓
Authentication and authorization
    ↓
Query classification
    ├── no retrieval
    ├── sparse retrieval
    ├── dense retrieval
    ├── hybrid retrieval
    ├── graph retrieval
    ├── long-context reading
    └── agentic multi-step search
    ↓
Metadata, date, tenant, and permission filtering
    ↓
Candidate retrieval
    ↓
Reranking and source diversification
    ↓
Context extraction or compression
    ↓
Grounded generation
    ↓
Citation and claim verification
    ↓
Abstention or final response
    ↓
Tracing, evaluation, and audit logging

This architecture reflects the field's transition from one fixed RAG pipeline toward controlled knowledge-access systems.

Final perspective

The development of RAG between 2020 and 2026 can be understood as six broad transitions:

The strongest RAG system is no longer simply the system with the largest vector database or the greatest number of retrieved chunks.

A reliable system must decide:

• Whether retrieval is needed.
• Which retrieval architecture fits the question and corpus.
• Whether the retrieved evidence is relevant and sufficient.
• Whether the sources are authoritative, current, and safe.
• How evidence should be ranked, compressed, and presented.
• Whether generated claims are actually supported.
• When the system should refuse to answer.

RAG is therefore evolving from a retrieval technique into a broader architecture for controlled, evidence-grounded access to external knowledge.

References

1. Lewis, P., Perez, E., Piktus, A., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.

2. Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M.-W. (2020). REALM: Retrieval-Augmented Language Model Pre-Training. ICML 2020.

3. Karpukhin, V., Oguz, B., Min, S., et al. (2020). Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020.

4. Izacard, G., and Grave, E. (2021). Leveraging Passage Retrieval with Generative Models for Open-Domain Question Answering. EACL 2021.

5. Petroni, F., Piktus, A., Fan, A., et al. (2021). KILT: A Benchmark for Knowledge Intensive Language Tasks. NAACL 2021.

6. Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., and Gurevych, I. (2021). BEIR: A Heterogeneous Benchmark for Zero-Shot Evaluation of Information Retrieval Models. NeurIPS 2021.

7. Shuster, K., Poff, S., Chen, M., Kiela, D., and Weston, J. (2021). Retrieval Augmentation Reduces Hallucination in Conversation. Findings of EMNLP 2021.

8. Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., and Zaharia, M. (2022). ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. NAACL 2022.

9. Borgeaud, S., Mensch, A., Hoffmann, J., et al. (2022). Improving Language Models by Retrieving from Trillions of Tokens. ICML 2022.

10. Izacard, G., Lewis, P., Lomeli, M., et al. (2022). Atlas: Few-Shot Learning with Retrieval-Augmented Language Models.

11. Izacard, G., Caron, M., Hosseini, L., et al. (2022). Unsupervised Dense Information Retrieval with Contrastive Learning.

12. Chen, W., Hu, H., Chen, X., Verga, P., and Cohen, W. W. (2022). MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text. EMNLP 2022.

13. Muennighoff, N., Tazi, N., Magne, L., and Reimers, N. (2022). MTEB: Massive Text Embedding Benchmark.

14. Gao, L., Ma, X., Lin, J., and Callan, J. (2023). Precise Zero-Shot Dense Retrieval without Relevance Labels. ACL 2023.

15. Shi, W., Min, S., Yasunaga, M., et al. (2024). REPLUG: Retrieval-Augmented Black-Box Language Models. NAACL 2024. First released as a 2023 preprint.

16. Jiang, Z., Xu, F. F., Gao, L., et al. (2023). Active Retrieval-Augmented Generation. EMNLP 2023.

17. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2023). Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. ACL 2023.

18. Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., and Chen, W. (2023). Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy. Findings of EMNLP 2023.

19. Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. (2024). Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR 2024. First released as a 2023 preprint.

20. Es, S., James, J., Espinosa-Anke, L., and Schockaert, S. (2024). RAGAS: Automated Evaluation of Retrieval Augmented Generation. EACL 2024 System Demonstrations. First released as a 2023 preprint.

21. Saad-Falcon, J., Khattab, O., Potts, C., and Zaharia, M. (2024). ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems. NAACL 2024.

22. Gao, T., Yen, H., Yu, J., and Chen, D. (2023). Enabling Large Language Models to Generate Text with Citations. EMNLP 2023.

23. Gao, Y., Xiong, Y., Gao, X., et al. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey.

24. Edge, D., Trinh, H., Cheng, N., et al. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization.

25. Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. D. (2024). RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. ICLR 2024.

26. Jeong, S., Baek, J., Cho, S., Hwang, S. J., and Park, J. C. (2024). Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. NAACL 2024.

27. Yan, S.-Q., Gu, J.-C., Zhu, Y., and Ling, Z.-H. (2024). Corrective Retrieval-Augmented Generation.

28. Su, W., Tang, Y., Ai, Q., Wu, Z., and Liu, Y. (2024). DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models. ACL 2024.

29. Zhang, T., Patil, S. G., Jain, N., et al. (2024). RAFT: Adapting Language Models to Domain-Specific RAG.

30. Lin, X. V., Chen, X., Chen, M., et al. (2024). RA-DIT: Retrieval-Augmented Dual Instruction Tuning. ICLR 2024.

31. Gutiérrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., and Su, Y. (2024). HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models.

32. Yu, Y., Ping, W., Liu, Z., et al. (2024). RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs. NeurIPS 2024.

33. Xu, F., Shi, W., Choi, E., and Yih, W.-t. (2024). RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation. ICLR 2024.

34. Yu, W., Zhang, H., Pan, X., et al. (2024). Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models. EMNLP 2024.

35. Gao, Y., Xiong, Y., Wang, M., and Wang, H. (2024). Modular RAG: Transforming RAG Systems into LEGO-Like Reconfigurable Frameworks.

36. Lyu, Y., Li, Z., Niu, S., et al. (2024). CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation.

37. Friel, R., Belyi, B., and Sanyal, A. (2024). RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems.

38. Tang, Y., Yang, Y., et al. (2024). MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries.

39. Ru, D., et al. (2024). RAGChecker: A Fine-Grained Framework for Diagnosing Retrieval-Augmented Generation.

40. Niu, C., et al. (2024). RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. ACL 2024.

41. Chen, J., Lin, H., Han, X., and Sun, L. (2024). Benchmarking Large Language Models in Retrieval-Augmented Generation. AAAI 2024.

42. Zou, W., et al. (2025). PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. USENIX Security 2025. First released as a 2024 preprint.

43. Liang, X., et al. (2025). SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Models. ACL 2025.

44. Singh, A., Ehtesham, A., Kumar, S., and Khoei, T. T. (2025). Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG.

45. Guo, Z., Xia, L., Yu, Y., Ao, T., and Huang, C. (2025). LightRAG: Simple and Fast Retrieval-Augmented Generation. Findings of EMNLP 2025. First released as a 2024 preprint.

46. Gutiérrez, B. J., et al. (2025). From RAG to Memory: Non-Parametric Continual Learning for Large Language Models.

47. Wang, Z., et al. (2025). Speculative RAG: Enhancing Retrieval-Augmented Generation through Drafting. ICLR 2025.

48. Jang, M., et al. (2025). Sufficient Context: A New Lens on Retrieval-Augmented Generation Systems. ICLR 2025.

49. Meng, Y., et al. (2025). SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains. NAACL 2025.

50. Jin, B., et al. (2025). Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning.

51. Zhang, J., et al. (2025). LongCite: Enabling LLMs to Generate Fine-Grained Citations in Long-Context QA. Findings of ACL 2025.

52. Li, Z., et al. (2025). LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs—No Silver Bullet for LC or RAG Routing. ICML 2025.

53. Cuconasu, F., et al. (2025). Enhancing Retrieval-Augmented Generation: A Study of Best Practices. COLING 2025.

54. Du, M., Xu, B., Zhu, C., Wang, S., Wang, P., Wang, X., and Mao, Z. (2026). A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces.

55. Wang, Z., Zhu, X., Lin, S., Xue, H., Guo, M., and Zhang, Y. (2026). RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing.

56. Fan, D., Xue, Z., Liu, S., and Tan, Q. (2026). Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems.

57. Hsiao, C.-H., et al. (2026). MegaRAG: Multimodal Knowledge Graph-Based Retrieval-Augmented Generation. ACL 2026.

58. Wang, X., Wang, Z., Gao, X., et al. (2024). Searching for Best Practices in Retrieval-Augmented Generation. EMNLP 2024.

59. Yang, Z., Qi, P., Zhang, S., et al. (2018). HotpotQA: A Dataset for Diverse, Explainable Multi-Hop Question Answering. EMNLP 2018.

60. Ho, X., Nguyen, A.-K. D., Sugawara, S., and Aizawa, A. (2020). Constructing a Multi-Hop QA Dataset for Comprehensive Evaluation of Reasoning Steps. COLING 2020. This paper introduced 2WikiMultiHopQA.

61. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2022). MuSiQue: Multihop Questions via Single-Hop Question Composition. Transactions of the Association for Computational Linguistics, 10, 539–554.

62. Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., and Hajishirzi, H. (2023). When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. ACL 2023. This paper introduced PopQA.

63. Kwiatkowski, T., Palomaki, J., Redfield, O., et al. (2019). Natural Questions: A Benchmark for Question Answering Research. Transactions of the Association for Computational Linguistics, 7, 452–466.

64. Rau, D., Déjean, H., Chirkova, N., et al. (2024). BERGEN: A Benchmarking Library for Retrieval-Augmented Generation. Findings of EMNLP 2024.

65. Chen, L., Razkenari, R., Zhou, Y., et al. (2026). Is GraphRAG Needed? From Basic RAG to Graph-/Agentic Solutions with Context Optimization.

Research coverage note: This article covers major general-purpose RAG architectures, retrieval methods, benchmarks, evaluation frameworks, and security studies published or publicly available through July 28, 2026. The 2026 section includes recent work whose conclusions may evolve after additional peer review and replication.