zylon-ai
Mention CASALIOY in ReadMe - we fixed **50+ issues of this repo**
Add note inside README
references CASALIOY (air-gapped toolkit)
Why your community would benefit? We took some time and effort to supercharge this base idea of a private LLM client and solved dozens of your repo's issues also. Choosing different routes in things like vector storage, ingestion approach and GUI for good measure. Some but not all features:
- [x] Speed improvements through qdrant and change in the base model
- [x] Blazing fast parallelized ingestion and local HF Sentence Transformer
- [x] GUI built with streamlit to let your family chat in the browser (easy to embed)
Benchmarks (just an excerpt)
| tests° | CASALIOY | privateGPT |
|---|---|---|
| Ingestion Speed (2000+ documents)* | <8 minutes | >10 hours |
| Ingestion Speed(SOTU.txt)* | <1 second | 10 seconds |
| LLM speed | Queued | Queued |
°tested on Ubuntu 22.04 LTS (i5-9600k, GTX 1050ti) examples for large ingestion here
Post Thank you for considering integrating this. However, we understand if you choose not to, and hope it can at least help resolve some of your repo's issues without altering your boilerplate.
could you please explain a bit more on wat this is? so with the changes you are recommending, the model would speed up indigestion of documents?
