Error When Using intfloat/multilingual-e5-large-instruct with llama_index

[Alnasser0]

Error When Using intfloat/multilingual-e5-large-instruct with llama_index

Description

Hi, I am trying to use intfloat/multilingual-e5-large-instruct, but all my approaches fail after following the llama_index documentation. I tried to use different versions of SentenceTransformer, but the same error remains.

I used the following code to initialize embeddings and return the embedding model and got the error after it

def init_embeddings():
    logger.info("Initializing embeddings")
    embed_model = InstructorEmbedding(model_name="intfloat/multilingual-e5-large-instruct")  # embed_batch_size?
    Settings.embed_model = embed_model
    ###### Test
    embeddings = embed_model.get_text_embedding("Hello World!")
    print(len(embeddings))
    print(embeddings[:5])
    ######
    logger.info("Embeddings initialized")
    return embed_model

---------------------------------------------------------------------------  
TypeError                                 Traceback (most recent call last)  
Cell In[16], line 65  
     63 # run the application  
     64 if __name__ == "__main__":  
---> 65     main()  

Cell In[16], line 10  
      8 logger.info(f"Data conversion to documents completed")  
      9 chunked_data = chunk(document_files)  
---> 10 embed_model = init_embeddings()  
     11 stored_index = create_index(chunked_data, embed_model)  
     12 print(stored_index)  

Cell In[6], line 3  
      1 def init_embeddings():  
      2     logger.info("Initializing embeddings")  
----> 3     embed_model = InstructorEmbedding(model_name="intfloat/multilingual-e5-large-instruct")  # embed_batch_size?  
      4     Settings.embed_model = embed_model  
      5     ###### test  

File c:\Users\Mahou\miniconda3\envs\bot4\lib\site-packages\llama_index\embeddings\instructor\base.py:48, in InstructorEmbedding.__init__(self, model_name, query_instruction, text_instruction, embed_batch_size, cache_folder, device, callback_manager)  
     30 def __init__(  
     31     self,  
     32     model_name: str = DEFAULT_INSTRUCT_MODEL,  
...
   (... truncated ...)  
    210         trust_remote_code=trust_remote_code,  
    211     )  

TypeError: _load_sbert_model() got an unexpected keyword argument 'token'

[dosubot[bot]]

Hey there, @Alnasser0! I'm here to help you with any bugs, questions, or contributions you have regarding the project. Let's tackle this issue together!

There is a similar closed issue regarding a TypeError with the InstructorEmbedding class due to an unexpected keyword argument 'token' ^[1]. The suggested solution for a related problem was to use an older version of the sentence-transformers package, specifically version 2.2.2, as the issue was identified with version 2.3+ ^[1]. You might want to try downgrading the sentence-transformers package to version 2.2.2 to see if it resolves your issue.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Alnasser0]

@dosu I tried pip install sentence-transformers==2.2.2, but it makes the import statement fail if I used version 2.2.2.

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 15
     13 from llama_index.core.retrievers import BaseRetriever
     14 from llama_index.core.schema import QueryBundle
---> 15 from InstructorEmbedding import INSTRUCTOR
     16 from llama_index.core.embeddings import BaseEmbedding
     17 from llama_index.core import get_response_synthesizer, Settings

File c:\Users\Mahou\miniconda3\envs\bot2\Lib\site-packages\InstructorEmbedding\__init__.py:1
----> 1 from .instructor import *

File c:\Users\Mahou\miniconda3\envs\bot2\Lib\site-packages\InstructorEmbedding\instructor.py:9
      7 from tqdm.autonotebook import trange
      8 from torch import Tensor, device
----> 9 from sentence_transformers import SentenceTransformer
     10 from sentence_transformers.models import Transformer
     11 from transformers import AutoConfig

File c:\Users\Mahou\miniconda3\envs\bot2\Lib\site-packages\sentence_transformers\__init__.py:3
      1 __version__ = "2.2.2"
      2 __MODEL_HUB_ORGANIZATION__ = 'sentence-transformers'
----> 3 from .datasets import SentencesDataset, ParallelSentencesDataset
      4 from .LoggingHandler import LoggingHandler
      5 from .SentenceTransformer import SentenceTransformer
...
---> 12 from huggingface_hub import HfApi, HfFolder, Repository, hf_hub_url, cached_download
     13 import torch
     14 from torch import nn, Tensor, device

ImportError: cannot import name 'cached_download' from 'huggingface_hub' (c:\Users\Mahou\miniconda3\envs\bot2\Lib\site-packages\huggingface_hub\__init__.py)
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...

[dosubot[bot]]

The error you're encountering is due to the cached_download function not being available in the version of the huggingface_hub package you have installed. This function is required by the sentence-transformers package version 2.2.2.

To resolve this issue, you can try updating the huggingface_hub package to a version that includes the cached_download function. Ensure that your huggingface_hub version is at least 0.19.0, as this version is compatible with sentence-transformers ^[1].

You can update the huggingface_hub package using the following command:

pip install huggingface-hub>=0.19.0

After updating, try running your code again to see if the import error is resolved.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Alnasser0]

@dosu I still have the same problem. I changed to HuggingFaceEmbedding, I hope it is fine to use with instruct models using query query_instruction and text_instruction

[dosubot[bot]]

Yes, you can use HuggingFaceEmbedding with instruct models, utilizing query_instruction and text_instruction. The InstructorEmbedding class in the LlamaIndex framework supports this by using functions to retrieve the appropriate instructions for a given model name. If the model is part of the INSTRUCTOR_MODELS, it uses default instructions for queries and texts. This setup ensures that the correct instructions are applied based on the model type, allowing effective integration with HuggingFaceEmbedding ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}