The Power of Noise: Redefining Retrieval for RAG Systems
It is argued here that the retrieval component of RAG systems, be it dense or sparse, deserves increased attention from the research community, and accordingly, the first comprehensive and systematic examination of the retrieval strategy of RAG systems is conducted.
Abstract
Retrieval-Augmented Generation (RAG) has recently emerged as a method to\nextend beyond the pre-trained knowledge of Large Language Models by augmenting\nthe original prompt with relevant passages or documents retrieved by an\nInformation Retrieval (IR) system. RAG has become increasingly important for\nGenerative AI solutions, especially in enterprise settings or in any domain in\nwhich knowledge is constantly refreshed and cannot be memorized in the LLM. We\nargue here that the retrieval component of RAG systems, be it dense or sparse,\ndeserves increased attention from the research community, and accordingly, we\nconduct the first comprehensive and systematic examination of the retrieval\nstrategy of RAG systems. We focus, in particular, on the type of passages IR\nsystems within a RAG solution should retrieve. Our analysis considers multiple\nfactors, such as the relevance of the passages included in the prompt context,\ntheir position, and their number. One counter-intuitive finding of this work is\nthat the retriever's highest-scoring documents that are not directly relevant\nto the query (e.g., do not contain the answer) negatively impact the\neffectiveness of the LLM. Even more surprising, we discovered that adding\nrandom documents in the prompt improves the LLM accuracy by up to 35%. These\nresults highlight the need to investigate the appropriate strategies when\nintegrating retrieval with LLMs, thereby laying the groundwork for future\nresearch in this area.\n