run_localGPT.py: chromadb.errors.NoIndexException: Index not found, please create an instance before querying

[johnkmayo]

I've been able to run the ingest.py and it seems to work. a chroma-collections.parquet and chroma-embeddings.parquet are created in the same folder as ingest.py.

When I run run_localGPT.py, it generates an "chromadb.errors.NoIndexException: Index not found, please create an instance before querying" error on line 66: res = qa(query).

Should the ingest.py be generating a third file with an index or is something else wrong?

[PromtEngineer]

It should also be generating an index folder with .pkl files. Do you have those?

[johnkmayo]

Yes. There is an index folder with three .pkl files. Does that mean the ingest.py process is running fine and the problem is in the run_localGPT.py?

[johnkmayo]

I'm still stuck on the "chromadb.errors.NoIndexException: Index not found, please create an instance before querying" error on line 66 of run_localGPT.py: res = qa(query).

The code seems to be set up to look in the same place that ingest.py is creating the .pkl files. I'd apprecaite some help getting past this problem.

[johnkmayo]

FYI, I pulled the latest code and tried again. The parquet files are now in the DB folder and the .pkl files are in the DB/index folder. I'm not sure why this isn't working for me.

[dr-nyt]

Did you manage to solve this?

[johnkmayo]

No. I have not been able to get the problem resolved. I still get the "chromadb.errors.NoIndexException: Index not found, please create an instance before querying" error with the current cut of the code.

[johnkmayo]

I tried again today. I made sure I had the latest code. I cleared out the DB folder. The ingest.py ran fine and populated the DB folder. When I run run_localGPT.py, it generates an "chromadb.errors.NoIndexException: Index not found, please create an instance before querying" error on line 86: res = qa(query).

[sindre-bouvet]

I fixed this by updating the constants.py file:

import os
from dotenv import load_dotenv
from chromadb.config import Settings

load_dotenv()

current_dir = os.getcwd()
ROOT_DIRECTORY = os.path.dirname(os.path.realpath(current_dir))

# Define the folder for storing database
SOURCE_DIRECTORY = f"{ROOT_DIRECTORY}/{os.environ.get('SOURCE_DIRECTORY')}"

PERSIST_DIRECTORY = f"{ROOT_DIRECTORY}/{os.environ.get('PERSIST_DIRECTORY')}"

# Define the Chroma settings
CHROMA_SETTINGS = Settings(
        chroma_db_impl='duckdb+parquet',
        persist_directory=PERSIST_DIRECTORY,
        anonymized_telemetry=False
)

Note: I am running the code in a notebook, on Azure.

[johnkmayo]

I'm running locally on Windows. I can't get the dotenv working. I changed the constants to have hard coded paths and I'm still getting the error.

What puzzles me is why the ingest.py can write to the folder but the run_localGPT.py can't seem to read from it.

[johnkmayo]

I finally got the code to execute. It looks like run_localGPT.py didn't like the path because of a non-alphanumeric character while ingest.py didn't have a problem with the path. Kind of odd. But at least I'm past that problem.

[johnkmayo]

I moved my data files into a folder which didn't have the non-alphanumeric character and then changed the ROOT_DIRECTORY to point to that folder.

On Mon, Jun 19, 2023 at 4:54 PM Muhammad Farooq @.***> wrote:

@johnkmayo https://github.com/johnkmayo Can you please share how you managed to run it? Did you give hard coded path?

— Reply to this email directly, view it on GitHub https://github.com/PromtEngineer/localGPT/issues/13#issuecomment-1597800966, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZODI36YH2DTL6ZGKEXDLHTXMDDAZANCNFSM6AAAAAAYS7A2ZA . You are receiving this because you were mentioned.Message ID: @.***>