Xuan Son Nguyen

As I understand, your Possibility-2 uses prefix/postfix system which is discussed in #5922 . The idea lead to nowhere so I though that using postfix/prefix was a bad idea. After...

> After a second thought (and also which what you suggested), my idea would be to firstly refactor the chat template system to support the postfix/prefix. This will save a...

> Why not use a switch? You cannot use switch if inside the `if` statement you do some logics (for example `str_contains`). In other words, `switch` will be compiled into...

> Or does C++ suffer from the same issue as C in a switch where it would need to be a integer or a character Yes, see: https://stackoverflow.com/questions/650162/why-cant-the-switch-statement-be-applied-to-strings > could...

`llama.h` is C-style header file, so `std::map` cannot be placed there. We can place it in `llama.cpp` if needed. But anw it is code style and not the real implementation,...

@dranger003 Probably that's because GritLM uses 2 prompt templates, one is used only for text generation and one only for embedding. Can you try embedding with the template specified by...

This model uses T5 architecture so we firstly need to support T5, see this issue: https://github.com/ggerganov/llama.cpp/issues/5763 Beside that, this model doesn't have a tokenizer (since time-series signal is converted directly...

At first lance, I'm not sure if it's a good idea to move the implementation completely into a separated JSON. While the good point is that it allows users to...

I've just have a look in detail for this PR. The idea seems ok (i.e. using input_prefix/input_suffix/antiprompt), but I still find the implementation is quite complicated IMO: - I still...

I understand the high level idea but sorry I really don't have time to look at the detailed implementation. While it's a good idea, IMO the chat template infrastructure should...