setzer22

I wasn't very fond of the names either, but during the porting phase I prioritized matching the C/C++ code as closely as possible over writing idiomatic Rust in many ways....

Hi @faassen! Thanks, I'm aware of the issue. You have to specify the `--release` profile as well for `cargo run`. That is, `cargo run --release`. It just doesn't build in...

Wow, nice catch! @philpax :smile: I wouldn't even know where to begin with this one. > It can be fixed in this repo's copy of ggml, or an upstream fix...

Fixed in #32

Thanks for reporting @CheatCod! We are aware of the issue. Apparently, some models have tokens in the embedded vocabulary that use invalid UTF-8 codepoints. This is an error, but not...

Yup, I'd say turning it into `info` would be nice to avoid log spam, since the issue seems to be quite common.

Not sure what to do about this. We often get reports about this issue, so leaving it open sounds like a good idea to avoid duplicates. But also, there's very...

Hi! Thanks for your post and for all the help so far :smile: > Turns out we think on extremely similar wavelengths Glad to hear I'm not the only one...

Just something to follow w.r.t to GPTQ quantization :eyes: `https://github.com/ggerganov/llama.cpp/issues/9`

> I've been testing on Windows with my patches applied and it seems to work fine. It's probably not _as_ fast as it could be, but it's plenty fast enough!...