HanishKVC

@ngxson do have a look at the new PR https://github.com/ggerganov/llama.cpp/pull/6834 which I have uploaded, it uses a simple json to load the expected/supported handshake-template as well as flag to control...

> @ngxson do have a look at the new PR > > #6834 > > which I have uploaded, it uses a simple json to load the expected/supported handshake-template as...

> It seems to me that the chatml formatting code in main.cpp is mostly good, sans the anti-prompt issue. Instead of trying to reuse `llama_chat_apply_template` is it possible to just...

> My idea is simply call `llama_chat_apply_template` twice: with and without the last user message. Then, I can find the diff between 2 output strings and feed it into inference....

> #6795 should work with all templates **except** for templates that does not have support for system prompt (or llama2 with `` for system message). That's why in #6810 I...

I am more of a casual recent enquirer (not a regular user) of LLMs, so I havent really looked deep into them or for that matter even llama.cpp. But my...

Now if you are talking about maintaining the context of all the interactions starting from system prompt to all the subsequent user + assistant dialogs/handshakes, wont each of them still...

> Now if you are talking about maintaining the context of all the interactions starting from system prompt to all the subsequent user + assistant dialogs/handshakes, wont each of them...

Also advantage of possibility-2 based kind of flow instead of a all in chat-apply-template, would be that, if there is text in the user's message which mimicks the underlying model's...

Small correction wrt possiblity-2 should be chat-hs-role-prefix and chat-hs-role-suffix, so that new roles can be added in future, using the same calls and infrastructure. Not sure why I suggested seperate...