Iván Martínez

Also take a look at this: LangChain has support for several [PDF Loaders](https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/pdf.html#) You can change the TextLoader in ingest.py and use any PDF loader as described in that link....

It depends on your hardware. On a M1 the state_of_the_union took around 1h for me

> The best thing to speed up ingestion would be to abandon the idea of using LLaMA for embeddings. Just like using full GPT-3 davinci to generate embeddings is costlier...

Bear in mind the ingest process will take quite some time, given it creates the embeddings locally. You could switch to lighter models or online embeddings for larger files. My...

Your error is quite explicit: "error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools". Did you try that? If that doesn't work, apparently...

@coderwithsense please update the branch with latest changes on main so it can be merged. Shouldn't be hard to integrate. Thanks!

A couple thoughts: First of all, this is amazing! I really like the idea. But I think we could explore the idea a little bit more. After reading this https://github.com/imartinez/privateGPT/issues/54...

In general terms @msrivas-7 , and trying to manage expectations, I'd like to test it thoroughly before merging it. Also an update to the readme would be required. I do...

Being 100% transparent, I used that text because it was the one used in [LangChain documentation](https://python.langchain.com/en/latest/modules/indexes/getting_started.html) for indexes, which are a core component of this POC. I just reused it,...

Great work @su77ungr @alxspiker ! Feel free to PR your changes to this repo if you feel like it, or just share your results here. My initial tests with qdrant...