In the world of Retrieval-augmented Generation (RAG), the retrieval process is paramount.
It's all about fetching the right documents based on a user's query.
But here's the catch: the traditional method of plugging in these documents in descending order of their vector similarity score might not be the best approach, especially when we're dealing with more than 10 documents.
The Core Issue
When we feed LLMs with a long context, they tend to overlook the documents placed in the middle.
So, contrary to what one might think, placing the least similar documents at the bottom isn't the best strategy.
So, we should put the least similar ones in the middle, not at the bottom.
π The "Lost-in-the-Middle" Phenomenon
A recent research paper shed light on this intriguing behavior of LLMs.
The findings?
1- The accuracy is better if the document containing the correct answer is near the top or bottom of the context.
Even the longer context models donβt solve this issue. Hence, they are not necessarily better at reasoning over longer input contexts yet.
Enter LangChain's innovative solution: the LongContextReorder.
Instead of sticking to the traditional order, LangChain shuffles the deck.
It places the most similar documents at the top, the next set at the bottom, and the least similar ones right in the middle.
The code?
LangChain LongContextReorder automatically does this.
- Create any retriever as usual
- Then get the relevant documents using the get_relevant_documents() method of that retriever
- Returns the documents in the descending order of their similarity score.
- Next, pass these documents to an instance of the LongContextReorder()
- Get the re-ordered docs where the least relevant ones are at the middle
In the fast-paced world of RAG, it's not just about retrieving the right documents but also presenting them in the most effective order.
The traditional method might seem logical, but as we've seen, it's not always the most efficient.
Thanks to LangChain's LongContextReorder, we now have a smarter way to present information to LLMs, ensuring better accuracy and performance.
So, the next time you're diving into the depths of RAG, remember: it's not just about what you present, but also how you present it.
Stay curious, and keep innovating!
If you like this, share it with others β»οΈ