anzz1

Could this be related to the phenomenon of the so-called [glitch tokens](https://www.lesswrong.com/posts/8viQEp8KBg2QSW4Yc/solidgoldmagikarp-iii-glitch-token-archaeology) ? The research on those has been focused on GPT-3 and I've yet to find any information as...

> Perhaps you are right. I have noticed that a lot of text related to Chinese has been incorrect until recently. That might be related, since the LLaMA model is...

Yes, by design all the "verbosity" is output to standard error (stderr) and the actual output in standard output (stdout) so simply running: `main -m ./models/llama-13B-ggml/ggml-model-q4_0.bin -p "Hi Bob!" >...

> quantize.py could probably be removed if we manage to make quantize.cpp just a bit more user-friendly. That is true, quantize.py is an wholly unnecessary step. > Or make llama.cpp...

The use of F16C code even when F16C was not enabled has been fixed as of https://github.com/ggerganov/llama.cpp/commit/a6bdc47cba23713a22ade47dd65b6afeb8009ff4 When building for Windows and building with MSVC, you can either uncheck the...

I somewhat agree with: - Typecastings **in most cases** should be explicit to denote conscious choices, however I understand that there can be a preference of not using them when...

> ... but not in this specific case. To prove it, I added a test that will loop through all possible floats (except NaNs and infinities) and verifies that the...

@Green-Sky should i port these changes to the format in #497

> @anzz1 sure, go ahead. will have some time later to check. Done.

Python versioning is painful in general, and especially so in Windows. Exactly why I made this https://github.com/anzz1/python-windows/ Just extract that to C:\python and click install.reg and you'll have a working...