private-gpt
private-gpt copied to clipboard
zylon-ai
Ingest going on forever
Is it normal that it takes more than 3 hours (and still running) to ingest a file of 200 Kb, which has 1500 lines of text and max 350 characters per line? How long does ingest take for you? I have an Intel® Core™ i9-8950HK CPU @ 2.90GHz × 12 with 32 Gig RAM.
I had a 100mb file take a little over 7 hours. a simple question took about 15 minutes to answer and it didn't even answer it well.
It took me 25 minutes to load the sample state of the union file (38kb), with the avg speed of 141ms per token. Ran out of RAM trying to query it (16gigs wasn't enough). The project looks awesome, just not for my current setup haha
Can't seem to do this on my Macbook Air M1. It took more than a day and still not completed for the given example file.
Same here...how long does the ingesting takes...I am just testing ingesting the sample "state of union.txt" file...
Same. Is this because of the computer or the models?
both the implementation and the pc. see, a cpu is not the best way of running such mathematical functions, also you need a decent gpu to get the best out of it. try to run it on a gpu (chdck my repos, have a Jupyter file for it).
Ingest got a LOT faster with the use of the new embeddings model #224
Note: this is a breaking change, any existing database will stop working with the new changes. You'll need to re-ingest your docs. It is recommended as the process is faster and the results are better.
