Mac M1 Pro — No module named transformers, Dependencies for InstructorEmbedding not found

[netsvetaev]

Macbook m1 Pro, Ventura 13. Python 3.10. When trying to start ingest.py, I get this error:

Loading documents from /Users/artur/localGPT/SOURCE_DOCUMENTS
Loaded 1 documents from /Users/artur/localGPT/SOURCE_DOCUMENTS
Split into 72 chunks of text
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/embeddings/huggingface.py", line 125, in __init__
    from InstructorEmbedding import INSTRUCTOR
  File "/Users/artur/instructor-embedding/InstructorEmbedding/__init__.py", line 1, in <module>
    from .instructor import *
  File "/Users/artur/instructor-embedding/InstructorEmbedding/instructor.py", line 9, in <module>
    from sentence_transformers import SentenceTransformer
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sentence_transformers/__init__.py", line 3, in <module>
    from .datasets import SentencesDataset, ParallelSentencesDataset
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sentence_transformers/datasets/__init__.py", line 3, in <module>
    from .ParallelSentencesDataset import ParallelSentencesDataset
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sentence_transformers/datasets/ParallelSentencesDataset.py", line 4, in <module>
    from .. import SentenceTransformer
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 11, in <module>
    import transformers
ModuleNotFoundError: No module named 'transformers'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/artur/localGPT/ingest.py", line 57, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/artur/localGPT/ingest.py", line 48, in main
    embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl",
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/embeddings/huggingface.py", line 131, in __init__
    raise ValueError("Dependencies for InstructorEmbedding not found.") from e
ValueError: Dependencies for InstructorEmbedding not found.

The thing is, I obviously have «transformers» at this python (3.10). And InstructorEmbedding seems fine.

BTW I can’t start the original PrivateGPT too (No module named ‚transformers’, Could not import sentence_transformers python package). Where to dig?

[PromtEngineer]

Are you using this within a virtual environment?

From the error message, it seems like your virtual env is not seeing the transformers package. Maybe create a new virtual env and just install the transformers package and see if you can load that in there.

[gabriead]

That should solve the issue: pip install transformers, pip install InstructorEmbedding

[Nabil-iium]

That should solve the issue: pip install transformers, pip install InstructorEmbedding

Faced same issue on windows. this solved the issue. Thanks

[mingjun1120]

During the deployment of my Streamlit web app, I am encountering the same issue (Refer to the diagram below). However, when I run the program on my local machine, it works as expected without any problems.

[IamAkku]

Facing same issue.

[adarshsaraf]

Have you found a solution for it?

[freebusy]

A similar problem was in Mac OS. Reinstalling the libraries helped me

pip install huggingface pip install transformers pip install InstructorEmbedding pip install torch pip install sentence_transformers

I also recommend watching the full error in the terminal (there is a more complete description then in the stream list window, and I wrote above this error that was still missing)

[techytim12]

Also try installing sentence-transformers==2.2.2