private-gpt icon indicating copy to clipboard operation
private-gpt copied to clipboard

Could not import sentence_transformers python package when running "python privateGPT.py"

Open RocketScience11 opened this issue 2 years ago • 3 comments

After ingesting the documents, when I run "python privateGPT.py", gives the following error:

Traceback (most recent call last): File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/langchain/embeddings/huggingface.py", line 46, in init import sentence_transformers File "/Users/pedrobeirao/privateGPT/sentence-transformers/sentence_transformers/init.py", line 3, in from .datasets import SentencesDataset, ParallelSentencesDataset File "/Users/pedrobeirao/privateGPT/sentence-transformers/sentence_transformers/datasets/init.py", line 1, in from .DenoisingAutoEncoderDataset import DenoisingAutoEncoderDataset File "/Users/pedrobeirao/privateGPT/sentence-transformers/sentence_transformers/datasets/DenoisingAutoEncoderDataset.py", line 5, in import nltk File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/nltk/init.py", line 153, in from nltk.translate import * File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/nltk/translate/init.py", line 24, in from nltk.translate.meteor_score import meteor_score as meteor File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/nltk/translate/meteor_score.py", line 13, in from nltk.corpus import WordNetCorpusReader, wordnet File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/nltk/corpus/init.py", line 64, in from nltk.corpus.reader import * File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/nltk/corpus/reader/init.py", line 106, in from nltk.corpus.reader.panlex_lite import * File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/nltk/corpus/reader/panlex_lite.py", line 15, in import sqlite3 File "/Users/pedrobeirao/mambaforge/lib/python3.10/sqlite3/init.py", line 57, in from sqlite3.dbapi2 import * File "/Users/pedrobeirao/mambaforge/lib/python3.10/sqlite3/dbapi2.py", line 27, in from _sqlite3 import * ImportError: dlopen(/Users/pedrobeirao/mambaforge/lib/python3.10/lib-dynload/_sqlite3.cpython-310-darwin.so, 0x0002): Symbol not found: _sqlite3_enable_load_extension Referenced from: /Users/pedrobeirao/mambaforge/lib/python3.10/lib-dynload/_sqlite3.cpython-310-darwin.so Expected in: /usr/lib/libsqlite3.dylib

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/pedrobeirao/privateGPT/privateGPT.py", line 76, in main() File "/Users/pedrobeirao/privateGPT/privateGPT.py", line 26, in main embeddings = HuggingFaceEmbeddings(model_name=embeddings_model_name) File "/Users/pedrobeirao/privateGPT/venv/lib/python3.10/site-packages/langchain/embeddings/huggingface.py", line 49, in init raise ValueError( ValueError: Could not import sentence_transformers python package. Please install it with pip install sentence_transformers.


MacBook Pro Apple M1 MacOs Monterrey

RocketScience11 avatar May 31 '23 10:05 RocketScience11

Can you try it; pip install sentence-transformers==2.2.1

yusufozben avatar May 31 '23 22:05 yusufozben

Installing pip install sentence-transformers==2.2.1 doesn't work, the problem is the missing symbol _sqlite3_enable_load_extension. You can fix it by re-configuring and recompiling python with the sqlite extensions: ./configure --enable-loadable-sqlite-extensions && make && sudo make install

See https://stackoverflow.com/questions/1210664/no-module-named-sqlite3

rostcheck avatar Jun 02 '23 23:06 rostcheck

Thanks! I cloned privateGPT and installed the requirements on a base conda environment instead of a dedicated environment as I did before, and it's now working.

On Sat, Jun 3, 2023 at 1:10 AM David Rostcheck @.***> wrote:

Installing pip install sentence-transformers==2.2.1 doesn't work, the problem is the missing symbol _sqlite3_enable_load_extension. You can fix it by re-configuring and recompiling python with the sqlite extensions: ./configure --enable-loadable-sqlite-extensions && make && sudo make install

See https://stackoverflow.com/questions/1210664/no-module-named-sqlite3

— Reply to this email directly, view it on GitHub https://github.com/imartinez/privateGPT/issues/570#issuecomment-1574424274, or unsubscribe https://github.com/notifications/unsubscribe-auth/AY4O2I5KV3OUD63DBCXQ7U3XJJXIDANCNFSM6AAAAAAYVHVLIE . You are receiving this because you authored the thread.Message ID: @.***>

RocketScience11 avatar Jun 04 '23 14:06 RocketScience11

Can you try it; pip install sentence-transformers==2.2.1

thank you so much, this solved my problem

mertsengil avatar Jul 24 '24 12:07 mertsengil