The query (chat) process takes forever

[chenle02]

Describe the bug and how to reproduce it I have included about 1K documents and indigested them for form a database chroma-embeddings.parquet of size 1.4G. When I enter a query and hit entery, the terminal seems to freeze. I has been a few minutes now, there is still no answer coming out. Is there a way to speed up this process? Is this due to my database is too large?

By the way, my machine has 36 cores and a decent GPU.

Thanks a lot for any suggestions.

EDIT: I have one short training md file and a simple query takes about 4 minutes to get processed.

[melroy89]

36 cores?? or 36 threads? What is your CPU spec exactly?

EDIT: Ow oke it's 18 cores..

[chenle02]

It is Intel i9-10980XE (36) @ 4.600GHZ

[InfamyStudio]

Did you make any headway on improving speeds @chenle02 - this is also what we want to achieve!