Eric Buehler
Eric Buehler
Yes, the prefix cache is a bit complicated and is spread out over multiple files. Here's an outline, for your and future reference, of how it works: - During the...
Hi @joshpopelka20! Thanks for the questions. Currently, it is a bit limited, but #350 will make it more powerful by enabling subsequences. The paper you linked is similar to what...
The disk is much slower, but the sequences there would be ones low on the LRU order, so it shouldn't incur a repetitive cost. There would be 3 tiers for...
Yeah, I'll look into that! I'm still working on #350, and I made some progress so now it is *almost* working. After I merge that, I'll apply the changes to...
Hi @abhisek-roy! This error means that you are compiling with the `mkl` feature but did not install MKL. You can find an installation guide on Windows here: [Intel oneAPI Base...
Thanks for reviewing! @LaurentMazare can you please take a look perhaps?
Hello @psyv282j9d! This sounds like a great feature which I would love to add. I have begun work on tracking memory usage in #392. I will look into applying the...
@psyv282j9d I just added importance matrix calculations (see the [docs](https://github.com/EricLBuehler/mistral.rs/blob/master/docs/IMATRIX.md))! Closing this now, please feel free to reopen!
@kinchahoy are you having this issue? I cannot reproduce it on my Mac - everything works.
@kinchahoy could you please let me know what your hardware (chip, memory, etc) is?