private-gpt

private-gpt copied to clipboard

Running into this error - python ingest.py

-Traceback (most recent call last): File "C:\Users\krstr\OneDrive\Desktop\privategpt\privateGPT\privateGPT\ingest.py", line 11, in from constants import CHROMA_SETTINGS File "C:\Users\krstr\OneDrive\Desktop\privategpt\privateGPT\privateGPT\constants.py", line 11, in CHROMA_SETTINGS = Settings( File "pydantic\env_settings.py", line 39, in pydantic.env_settings.BaseSettings.init File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for Settings persist_directory none is not an allowed value (type=type_error.none.not_allowed) -

I've installed the requirements and changed the .env file and followed the readme up to this point. Seeing some people solve but not answer what fixed the above errors. Help?

The error message suggests that the persist_directory variable in your .env file is set to None, which is not an allowed value according to the Settings class defined in constants.py. You will need to update the value of persist_directory in your .env file to a valid directory path.

Make sure that you have followed the instructions in the README file carefully, including setting up a virtual environment and installing the required dependencies. Also, double-check that your .env file is correctly formatted and contains all the required variables.

Once you have updated the value of persist_directory in your .env file, try running the ingest.py script again. If you still encounter issues, you may need to provide more information about your setup and the specific steps you took before encountering the error.

Thanks for the assistance however I'm running into the same issue. I tried changing the Persist_Directory=db to PERSIST_DIRECTORY=C:/Users/krstr/OneDrive/Desktop/privategpt/privateGPT/privateGPT/db:

Ingest.py gives the same values as before.

I am not running in a virtual environment at the moment.

Have you created the /db folder? I solved mine after creating the /db folder manually.

Yes I have, ranpo, under that but no change still. :( I do appreciate all the help.

Hi @krstrid, have you renamed the example.env file to .env? It looks like the Python script to ingest documents cannot find the env file with the required name.

Hi VirtualRamblas, yes I have. Not sure what's up and I'm not an expert coder. Just trying to follow along.

@krstrid Are you sure you're .env file is located at the root of the project? Did you add any spacing after or before the = when defining your env variables? Did you modify any of the code in the ingest file? In case it's none of these things, you might also have permissions issues to load the file.

I feel like there needs to be built it error handling for most common gotchas like this and specify in the error message exactly what it expects.

@krstrid Are you sure you're .env file is located at the root of the project? Did you add any spacing after or before the = when defining your env variables? Did you modify any of the code in the ingest file? In case it's none of these things, you might also have permissions issues to load the file.

Hi andreakiro, it is at the root of the project, there are no extra spacings for the env variables, it has been renamed .env from example.env and no code has been modified in the ingest file.

I shouldn't have permissions issues as Admin afaik.

PS C:\Users\krstr\OneDrive\Desktop\privategpt\privateGPT\privateGPT> python3 ingest.py Traceback (most recent call last): File "C:\Users\krstr\OneDrive\Desktop\privategpt\privateGPT\privateGPT\ingest.py", line 4, in from dotenv import load_dotenv ModuleNotFoundError: No module named 'dotenv' PS C:\Users\krstr\OneDrive\Desktop\privategpt\privateGPT\privateGPT>

Someone in another thread suggested to I try to run python3 ingest.py - It did give me this newer error than the python ingest.py command mentioned above this.

@krstrid You're missing the dotenv library, it is not in the requirements.txt for some reason. Try running:

# execute at project root
pip install -r requirements.txt
pip install python-dotenv
python ingest.py

If problem persist, can you please share your .env details here?

It says I have python-dotenv installed after I installed requirements. When I run python ingest.py I get the errors above. -

Here are the contents of my env file - PERSIST_DIRECTORY=db LLAMA_EMBEDDINGS_MODEL=C:\Users\krstr\OneDrive\Desktop\privateGPT\models\ggml-model-q4_0.bin MODEL_TYPE=GPT4All MODEL_PATH=C:\Users\krstr\OneDrive\Desktop\privateGPT\models\ggml-gpt4all-j-v1.3-groovy.bin MODEL_N_CTX=1000

There is a models folder I created and I put the models into that folder. It did not originate a db folder with ingest.py but I did create a db folder to no luck.

Appreciate all the help. I am using just the code provided along with instructions.

@krstrid can you try modifying constants.py, then running ingest and share the trace again.

# modifications in `constants.py` file:
L8: PERSIST_DIRECTORY = os.environ.get('PERSIST_DIRECTORY')
L9: print(PERSIST_DIRECTORY) # NEW!
$ python ingest.py

Hi @andreakiro, I have modified constants.py and get the same result - restarted from scratch and have done everything over again and this is what I get still -

Traceback (most recent call last): File "C:\Users\krstr\OneDrive\Desktop\privategpt\ingest.py", line 11, in from constants import CHROMA_SETTINGS File "C:\Users\krstr\OneDrive\Desktop\privategpt\constants.py", line 13, in CHROMA_SETTINGS = Settings( File "pydantic\env_settings.py", line 39, in pydantic.env_settings.BaseSettings.init File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for Settings persist_directory none is not an allowed value (type=type_error.none.not_allowed) -

This is what I get from a quick screenshot of my directory - I do have full Admin -

This is what I get when I run python3 ingest.py

Traceback (most recent call last): File "C:\Users\krstr\OneDrive\Desktop\privateGPT\ingest.py", line 4, in from dotenv import load_dotenv ModuleNotFoundError: No module named 'dotenv'

@krstrid Try the following and see what happens:

python3 -m pip install -r requirements.txt
python3 -m pip install python-dotenv
python3 ingest.py
python3 privateGPT.py

@andreakiro - I ran this both under a minconda3 instance and also a windows powershell instance.

It's giving me the original error on both

• Traceback (most recent call last): File "C:\Users\krstr\OneDrive\Desktop\privateGPT\ingest.py", line 11, in from constants import CHROMA_SETTINGS File "C:\Users\krstr\OneDrive\Desktop\privateGPT\constants.py", line 13, in CHROMA_SETTINGS = Settings( ^^^^^^^^^ File "pydantic\env_settings.py", line 39, in pydantic.env_settings.BaseSettings.init File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for Settings persist_directory none is not an allowed value (type=type_error.none.not_allowed) --

  The requirement is already satisified for python-dotenv for the command: python3 -m pip install python-dotenv

  I'm stumped, I feel like I'm close but close isn't good enough for this kind of thing haha (hope you're all in good spirits).

Update: Both ingest.py and privategpt.py have the same error, @andreakiro.

(base) C:\Users\krstr\OneDrive\Desktop\privateGPT>python3 ingest.py Traceback (most recent call last): File "C:\Users\krstr\OneDrive\Desktop\privateGPT\ingest.py", line 11, in from constants import CHROMA_SETTINGS File "C:\Users\krstr\OneDrive\Desktop\privateGPT\constants.py", line 13, in CHROMA_SETTINGS = Settings( ^^^^^^^^^ File "pydantic\env_settings.py", line 39, in pydantic.env_settings.BaseSettings.init File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for Settings persist_directory none is not an allowed value (type=type_error.none.not_allowed)

(base) C:\Users\krstr\OneDrive\Desktop\privateGPT>python3 privategpt.py Traceback (most recent call last): File "C:\Users\krstr\OneDrive\Desktop\privateGPT\privategpt.py", line 18, in from constants import CHROMA_SETTINGS File "C:\Users\krstr\OneDrive\Desktop\privateGPT\constants.py", line 13, in CHROMA_SETTINGS = Settings( ^^^^^^^^^ File "pydantic\env_settings.py", line 39, in pydantic.env_settings.BaseSettings.init File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for Settings persist_directory none is not an allowed value (type=type_error.none.not_allowed) --

The carats are under Settings(

FYI I solved the dotenv error by using this command:

python -m pip install python-dotenv

But now another error when running ingest.py:

Traceback (most recent call last):
  File "ingest.py", line 6, in <module>
    from langchain.document_loaders import TextLoader, PDFMinerLoader, CSVLoader
ModuleNotFoundError: No module named 'langchain'

I kinda wish someone (in my case with Windows) would do a full install of privategpt from where it is at with the current files uploaded to gh. Something isn't right but I don't know enough to fix it.

@Drakot - Thanks, @andreakiro also assisted a little earlier with a python3 command and I got past that error too. Thanks though!

Had the same problem few hours ago. Finally after some research i solved it. What happened at the very beginning was that when you did the "Pip install -r requirements" some requirements didn't install, it is actually mentioned somehow in the read.dm. Anyway, what you have to do is install langchain Pip install langchain should work, but when you try to run it it will launch a new error with chromadb, so that is the famous wheel problem that is in the read.dm. Sol: Download visual studio 2022 with the things related in the ReadMe

Ok, so my file was saving as .env.env when I ran the dir command instead of just .env. I changed by going to view file extensions then I renamed .env. I'm now getting different data but it seems like progress. Sorry everyone, you were right about that it just wasn't saving right in windows until I went into advanced properties and fixed it. Now I'm getting a heap of something else but it's progress. -

(base) PS C:\Users\krstr\Onedrive\desktop\privategpt> python ingest.py Loading documents from source_documents Loaded 1 documents from source_documents Split into 90 chunks of text (max. 500 tokens each) Traceback (most recent call last): File "C:\Users\krstr\miniconda3\lib\site-packages\langchain\embeddings\llamacpp.py", line 78, in validate_environment values["client"] = Llama( File "C:\Users\krstr\miniconda3\lib\site-packages\llama_cpp\llama.py", line 155, in init self.ctx = llama_cpp.llama_init_from_file( File "C:\Users\krstr\miniconda3\lib\site-packages\llama_cpp\llama_cpp.py", line 182, in llama_init_from_file return _lib.llama_init_from_file(path_model, params) OSError: [WinError -529697949] Windows Error 0xe06d7363

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\krstr\Onedrive\desktop\privategpt\ingest.py", line 65, in main() File "C:\Users\krstr\Onedrive\desktop\privategpt\ingest.py", line 56, in main llama = LlamaCppEmbeddings(model_path=llama_embeddings_model, n_ctx=model_n_ctx) File "pydantic\main.py", line 339, in pydantic.main.BaseModel.init File "pydantic\main.py", line 1102, in pydantic.main.validate_model File "C:\Users\krstr\miniconda3\lib\site-packages\langchain\embeddings\llamacpp.py", line 98, in validate_environment raise NameError(f"Could not load Llama model from path: {model_path}") NameError: Could not load Llama model from path: C:\Users\krstr\OneDrive\Desktop\privateGPT\models Exception ignored in: <function Llama.del at 0x000002A8B6718700> Traceback (most recent call last): File "C:\Users\krstr\miniconda3\lib\site-packages\llama_cpp\llama.py", line 978, in del if self.ctx is not None: AttributeError: 'Llama' object has no attribute 'ctx'

A quick redo of the absolute filepaths and now it finally is working. Thank you all for your assistance! So yes ultimately it was the folder being renamed to .env (appeared as that way in Windows GUI but not in dir as .env.env) and the filepaths being in the right spot. Now it is running python ingest.py and ingesting from the source_documents folder. Phew.

I have the smae error,can you tell me how you solved it? Thanks in advance.

My main error was not copying over the .env file right. When in GUI it appears as .env but in dir showed up as .env.env. When in your directory use the command dir.

There you'll see the files. I had a file called env.env. So I copied it by doing = cp .env.env .env --

Now you should be able to manipulate the values in .env. I then deleted env.env.

I then followed all the steps and it worked. Hope that helps. If not post an error, but admittedly I'm a bit of a 'script-kiddy' so someone else might be of more assistance than I.

thanks。 I have sloved it.

Solved No module named 'dotenv' by removing python 3.11 from Path and leaving 3.10 at the top of the list. Even with 3.10 over 3.11, installing requirements from CMD, it would install it for 3.11 instead of 3.10.