anzz1

Since the alpaca.cpp project currently does not exhibit this issue, and based on when these reports started appearing, the problem most likely is traced back to the tokenizer change and...

@Piezoid Thanks for pointing me towards here from the other discussion. I'll be checking your branch out and testing it. I'm also rooting for you finishing the trace tool at...

@sr-hm @botatooo The cause of this is that the point-alpaca model in question has a added "[PAD]" token so the resulting model contains 32001 tokens, but the vocab size was...

Reading the [data release](https://github.com/tatsu-lab/stanford_alpaca#data-release) closely, > During inference (eg for the web demo), we use the user instruction with an empty input field (second option). How it's currently implemented is...

Yes, keep the core lean, portable, fast and free of dependencies while having the option of building things on top of it, as modules. This could be achieved with a...

> ... > Again if the case is that more processes is what is wanted and an ability to share the state between them, a more general approach would be...

> You mean something like importing https://github.com/alitrack/mman-win32 ? Nope, quite the opposite, steering clear of any non-portable code, imported libraries or dependencies *inside* the main program, and have any functionality...

> @anzz1 This PR is not about multiprocessing or sharing memory but rather accelerating the loading of the model via a memory mapped file (see #91 for more details). Though...

the .pth files == ggml-f16 which both contain the full information. if you have a quantized .pt or ggml-q4_0 / q4_1 , the full information is already lost so you...

> @anzz1 Thank you for your comment. However, what if you want to study the effect of finetuning on quantized models? Or simply want to look at the distribution of...