Flash Rerank with Document Compression.

[jagrut14]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import FlashrankRerank


db = Chroma(persist_directory=PERSIST_DIRECTORY, embedding_function=embedding_model,collection_name="rag-chroma")
retriever = db.as_retriever(search_kwargs={"k": 5})
#Reranking
compressor = FlashrankRerank()
retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)

Description

I want to use the Reranker function with my following code. Without it downloading the mode "ms-marco-MultiBERT-L-12" every time. Is there a way where I can locate this model locally and provide the path of the model to this function?

System Info

langchain==0.1.9

[dosubot[bot]]

Hey @jagrut14, great to see you diving into another challenge with LangChain! 🚀

Based on the provided context, it seems that the "ms-marco-MultiBERT-L-12" model is being used in the FlashrankRerank class of the LangChain codebase. However, there is no direct reference to downloading or initializing this model in the provided context. The model loading or initialization logic might be encapsulated within the LLMChain or the model creation utility create_model used in LangChain.

To specify a local path for a model, you would typically need to adjust the model initialization logic to check for a local model path before downloading it. This often involves passing a path to the model's constructor or initialization method, or setting a configuration option to use a local model path.

Unfortunately, without the specific implementation details of the model loading mechanism in LangChain, the precise code changes cannot be provided. You might need to explore the implementation of LLMChain, create_model, or wherever the model is being loaded in the LangChain library to make the necessary adjustments.

As for the FlashrankRerank class, it seems to be implemented as a subclass of BaseDocumentCompressor and is designed to compress documents using the Flashrank interface. The class includes a client attribute, which is an instance of the Ranker class from the flashrank package, used to perform the document reranking. The model attribute is an optional string specifying the model to use for reranking. If not provided, a default model named "ms-marco-MultiBERT-L-12" is used. However, there is no direct reference to a local model path in the FlashrankRerank class.

I hope this information helps. If you need further assistance, please provide more details about the model loading mechanism in LangChain or the specific part of the code where the "ms-marco-MultiBERT-L-12" model is being loaded.

Sources

• libs/langchain/langchain/chains/combine_documents/map_rerank.py
• libs/langchain/langchain/retrievers/document_compressors/flashrank_rerank.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[thiamteck]

hi @jagrut14 , I found from FlashRerank code at https://github.com/PrithivirajDamodaran/FlashRank/blob/main/flashrank/Config.py, it show the default cache folder is "/tmp". If it is running on Windows, most likely it is at C:/tmp/