Top Research Papers on RAG

It is found that RAG-Fusion was able to provide accurate and comprehensive answers due to the generated queries contextualizing the original query from various perspectives, however, some answers strayed off topic when the generated queries' relevance to the original query is insufficient.

This survey provides a comprehensive exploration of Agentic RAG, beginning with its foundational principles and the evolution of RAG paradigms, and presents a detailed taxonomy of Agentic RAG architectures, highlights key applications in industries such as healthcare, finance, and education, and examines practical implementation strategies.

This talk describes the journey in building a knowledgeable AI assistant by harnessing LLM techniques, and describes a federated Retrieval-Augmented Generation system that integrates external information from both the web and knowledge graphs for trustworthy text generation on real-time topics like stocks and sports, as well as on torso-to-tail entities like local restaurants.

This paper proposes learning to Refine Query for Retrieval Augmented Generation (RQ-RAG), endeavoring to enhance the model by equipping it with capabilities for explicit rewriting, decomposition, and disambiguation, and demonstrates enhanced performance in handling complex, multi-hop QA datasets.

This paper proposes the ‘Blended RAG’ method of leveraging semantic search techniques, such as Dense Vector indexes and Sparse Encoder indexes, blended with hybrid query strategies, which achieves better retrieval results and sets new benchmarks for IR datasets like NQ and TREC-COVID datasets.

Speculative RAG is introduced - a framework that leverages a larger generalist LM to efficiently verify multiple RAG drafts produced in parallel by a smaller, distilled specialist LM, which accelerates RAG by delegating drafting to the smaller specialist LM, with the larger generalist LM performing a single verification pass over the drafts.

RAGGED, a framework for analyzing RAG configurations across various DBQA tasks, discovers distinct LM behaviors in response to varying context quantities, context qualities, and retrievers and provides a deeper analysis of these differences.

A novel dataset, MultiHop-RAG, which consists of a knowledge base, a large collection of multi-hop queries, their ground-truth answers, and the associated supporting evidence, and it is hoped MultiHop-RAG will be a valuable resource for the community in developing effective RAG systems, thereby facilitating greater adoption of LLMs in practice.

This study emphasizes that the RAG architecture’s ability to retrieve information by dynamically using the learnings obtained from large datasets of LLMs strengthens applications in the field of NLP.

This work conducts extensive empirical studies with novel attack methods, which demonstrate the vulnerability of RAG systems on leaking the private retrieval database and reveals that RAG can mitigate the leakage of the LLMs' training data.

A novel framework to evaluate the reliability of multi-modal RAG using two performance measures: the relevancy score (RS), assessing the relevance of retrieved entries to the query, and the correctness score (CS), evaluating the accuracy of the generated response.

A critical analysis of failure points in existing KG-based RAG methods is presented, identifying eight key areas of concern, including misinterpretation of question context, incorrect relation mapping, and ineffective ambiguity resolution, which aim to improve the reliability and effectiveness of KG-RAG systems.

LegalBench-RAG is introduced, the first benchmark specifically designed to evaluate the retrieval step of RAG pipelines within the legal space, and serves as a critical tool for companies and researchers focused on enhancing the accuracy and performance of RAG systems in the legal domain.

C-RAG is proposed, the first framework to certify generation risks for RAG models and it is proved that RAG achieves a lower conformal generation risk than that of a single LLM when the quality of the retrieval model and transformer is non-trivial.

The study demonstrates the effectiveness of the RAG system in generating relevant suggestions with a consistent accuracy of 93% and highlights the value of identifying and understanding knowledge gaps to guide future endeavours.

It is demonstrated that RAG systems that operate on databases with untrusted content are vulnerable to denial-of-service attacks the authors call jamming, and a new method based on black-box optimization is described and measured, including a new method based on black-box optimization.

Auto-RAG is introduced, an autonomous iterative retrieval model centered on the LLM's powerful decision-making capabilities that can autonomously adjust the number of iterations based on the difficulty of the questions and the utility of the retrieved knowledge, without requiring any human intervention.

This paper presents the first comprehensive study of RAG systems that incorporate fairness-aware rankings, focusing on both ranking fairness and attribution fairness - ensuring equitable exposure of sources cited in the final text.

Compared to model fine-tuning, Class-RAG demonstrates flexibility and transparency in decision-making, outperforms on classification and is more robust against adversarial attack, as evidenced by empirical studies.

This talk describes the journey in building a knowledgeable AI assistant by harnessing LLM techniques, and constructed a federated Retrieval-Augmented Generation (RAG) system that integrates external information from both the web and knowledge graphs in text generation.

The RAG-RewardBench is proposed, the first benchmark for evaluating RMs in RAG settings and reveals that existing trained RALMs show almost no improvement in preference alignment, highlighting the need for a shift towards preference-aligned training.

This work proposes a novel approach called Progressive Retrieval Augmented Generation (P-RAG), which not only effectively leverages the powerful language processing capabilities of LLMs but also progressively accumulates task-specific knowledge without ground-truth.

This paper constructs a large-scale and more comprehensive benchmark, and evaluates all the components of RAG systems in various RAG application scenarios, and refers to the CRUD actions that describe interactions between users and knowledge bases, and categorizes the range of RAG applications into four distinct types.

Experiments show that UniMS-RAG achieves state-of-the-art performance on the knowledge source selection and response generation task with itself as a retriever in a unified manner.

Document processing and query generation tasks in customs declaration scenarios face key challenges such as the complexity of multimodal data, adaptability to dynamic regulations, and ambiguity in query semantics. This study proposes a Retrieval-Augmented Generation system (ICCA-RAG) that addresses the core issues of processing complex customs documents and dynamically generating queries through multimodal document parsing, sparse-dense hybrid storage, and context-driven large language model generation. In terms of multimodal document parsing, the system supports comprehensive parsing of PDFs,...

This paper evaluates various advanced solutions proposed in recent literature, comparing their efficacy and discussing the trade-offs involved, and delves into the central architecture of RAG systems, encompassing retrieval components, generative components, and knowledge bases.

A novel knowledge graph-based RAG framework with a refined retrieval pipeline, robust chunking mechanism, and source traceability for enhanced diabetes-focused LLM is proposed, demonstrating effective performance in diabetes-focused LLM.

Stochastic RAG casts the retrieval process in RAG as a stochastic sampling without replacement process, and employs straight-through Gumbel-top-k that provides a differentiable approximation for sampling without replacement and enables effective end-to-end optimization for RAG.

GFM-RAG is a novel graph foundation model (GFM) for retrieval augmented generation powered by an innovative graph neural network that reasons over graph structure to capture complex query-knowledge relationships, making it the first graph foundation model applicable to unseen datasets for retrieval without any fine-tuning required.

Graphical Eigen Memories For Retrieval Augmented Generation (GEM-RAG), motivated by human memory encoding and retrieval, is introduced, which aims to improve over standard RAG methods by generating and encoding higher-level information and tagging the chunks by their utility to answer questions.

Multi-Agent Filtering Retrieval-Augmented Generation (MAIN-RAG) is proposed, a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents and introduces an adaptive filtering mechanism that dynamically adjusts the relevance filtering threshold based on score distributions.

Plan*RAG is introduced, a novel framework that enables structured multi-hop reasoning in retrieval-augmented generation (RAG) through test-time reasoning plan generation by isolating the reasoning plan as a directed acyclic graph outside the LM's working memory.

Experimental results indicate that Pistis-RAG improves alignment with human preferences relative to the baseline RAG system, showing a 6.06% increase in MMLU (English) and a 7.08% increase in C-EVAL (Chinese) accuracy metrics, highlighting Pistis-RAG's effectiveness in overcoming the limitations associated with traditional RAG approaches.

RAG Foundry is introduced, an open-source framework for augmenting large language models for RAG use cases and demonstrates the framework effectiveness by augmenting and fine-tuning Llama-3 and Phi-3 models with diverse RAG configurations, showcasing consistent improvements across three knowledge-intensive datasets.

This paper presents good practices to implement, optimize, and evaluate RAG for the Brazilian Portuguese language, focusing on the establishment of a simple pipeline for inference and experiments, and presents the complete architecture of the RAG.

The TREC 2024 RAG Track is proposed, which introduces a web-based user interface for an interactive arena allowing benchmarking pairwise RAG systems by crowdsourcing and opens-source the Ragnar\"ok framework and baselines to achieve a unified standard for future RAG systems.

The approach leverages a multi-armed bandit algorithm, which treats each retrieval method as a distinct ``arm'' and adapts the selection process by balancing exploration and exploitation, and introduces a dynamic reward function that balances accuracy and efficiency.

The RAG system utilizes a 2-step vector search using the vector search with cosine similarity metric on an HNSW index on the paper’s abstracts and the papers itself to pass only relevant information to LLM; this enables enhanced data retrieval and contextually aware text generation.

It is shown that retrieval-augmented fine-tuning of NMT mod-012 els under the RAGMT framework results in an average improvement of 2.03 BLEU scores over simple fine-tuning approaches on English 015 to German domain-specific translation.

ARM-RAG (Auxiliary Rationale Memory for Retrieval Augmented Generation), a system that learns from its successes without incurring high training costs is proposed, and it is demonstrated that the storage and subsequent retrieval of reasoning chains have a positive influence on performance in grade-school math problems.

This work proposes a novel LLM-centric approach that integrates IR systems with LLMs to support multi-round RAG through learning Inner Monologues, and introduces a Refiner that improves the outputs from the Retriever, effectively bridging the gap between the Reasoner and IR modules with varying capabilities and fostering multi-round communications.

The experimental results on multi-hop QA datasets show that DR-RAG can significantly improve the accuracy of the answers and achieve new progress in QA systems.

This paper endeavors to establish a straightforward yet potent benchmark for semantic search in Arabic within the framework of retrieval augmented generation (RAG) and examines the effectiveness of metrics and the dataset.

Poison-RAG manipulates item metadata, such as tags and descriptions, to influence recommendation outcomes, and indicates that popular items are more susceptible to attacks, whereas long-tail items are harder to manipulate.

Experiments on a real-world tourism dataset show that Spatial-RAG significantly improves spatial question answering, bridging the gap between LLMs and spatial intelligence.

This work proposes SAGE, a novel two-stage synthetic data generation paradigm that employs an attribute-based extraction and generation approach to preserve key contextual information from the original data and enhances the privacy properties of the synthetic data through an agent-based iterative refinement process.

It is demonstrated that LaB-RAG achieves better results across natural language and radiology language metrics compared with other retrieval-based RRG methods, while attaining competitive results compared to other fine-tuned vision-language RRG models.

This paper proposes RAG-end2end, an extension to RAG that can adapt to a domain- specific knowledge base by updating all components of the external knowledge base during training and introduces an auxiliary training signal to inject more domain-specific knowledge.

This project presents a novel system designed to enhance code generation by leveraging Retrieval-Augmented Generation, Grounding techniques, and Prompt Parameters, which represents a significant advancement in automating code generation from natural language descriptions.

Results show that RAG increases accuracy in some cases, but can still be misled when prompts directly contradict the model's pre-trained understanding, highlighting the complex nature of hallucinations and the need for more robust solutions to ensure LLM reliability in real-world applications.