2024

ITRF: Instruction-Tuned Reranker Fine-Tuning

LLM · Llama · NLP · RAG · Python

During my Thesis I researched ways on how to effectively train rerankers to score better results in retrievel augmented generation (RAG) applications.

Large Language Models (LLMs) have emerged as one of the most profound shifts in Natural Language Processing (NLP). While generally applicable they have their shortcomings: Generate content is often hallucinated, there is a lack of provenance and its knowledge is costly to update. Retrieval Augmented Generation (RAG) aims to relieve these issues by providing the LLM with relevant information.

In this work we establish a retrieval corpus, build a RAG pipeline, fine-tune a language model on task specific data in a parameter efficient way to extract knowledge from provided contexts and subsequently train a reranker to predict the probability a document is going to help the LLM generate the right answer on 25 datasets. We establish five distinct pipelines which we evaluate on faithfulness, answer relevancy, answer semantic similarity, context recall and context relevancy. We find that a out of the box pre-trained reranker in combination with our fine-tuned model works best on a range of scores, with a third-party LLM with our reranker provides the answer most closely to the ground truth. The code and models are available open source.