Wouter Tichelaar

This is my first ever pull request, so please feel free to give me feedback on anything I could improve upon.

> @DifferentialityDevelopment thanks for your quick work on getting a PR open, I pulled your changes to llama.cpp and rebuilt, then tried the new template. I'm seeing some issues with...

> Hi, I think you should also modify file utils.hpp > > ``` > llama_params["stop"].push_back(""); // chatml > llama_params["stop"].push_back(""); // gemma > llama_params["stop"].push_back(""); // llama 3 ``` > > to...

I'm not 100% sure but I think I know why the one test might be failing tests/test_chat_template.cpp line 79 begins with "" ie the bos token though I noticed that...

> Yes, you need to remove the BOS text from the reference string Done!

> Hi @DifferentialityDevelopment - per the [model card on hugging face](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), `` is a stop token, but it seems like it might not be the only one? If you look...

> I think i read somewhere (so take this with a grain of salt, because I don't remember where) that it's an artifact of how they did the instruct-tuning on...

> `` is the EOS token, so you don't need to include it in the list of stop words. In short, server will stop generation if it receives EOS token....

It's due to a different pull request that got merged I think https://github.com/ggerganov/llama.cpp/commit/b97bc3966e852adb626c90be64fd48282800f504#diff-ad8b15a29dd7c625dd2688de421972baaa73494a72d7210d679efc5f2ec0d888 llama_token_is_eog is supposed to return true for as far as I'm aware ` bool llama_token_is_eog(const struct llama_model...

This sounds like an excellent and much needed addition to main. Did you add a flag for specifying the system roles message?