janhq
epic: format generated code in code format (not plain text)
Motivation
Seems all the other LLMs do this now, so would be very useful comfort to have in Jan!
hi @xx88xx, sorry, is this what you mean?
The copy button will help copy code in the same format that is being displayed 🙏
I also sometimes run into this issue. Does anyone know if different models maybe use different syntax for things like code blocks? Like how we have multiple different prompt templates/tokens?
Hi @Van-QA ! Thanks for the response. Since posting this, noticed it works in the more popular models. I agree with @mr-september sometimes it works as in the picture you attached, but often especially on some of the more bespoke models it shows up unformatted. @mr-september might be onto something with the different syntax.
thank you, can you let us know more details of the failed model / step to reproduce? @xx88xx
With some further testing/hindsight, I think I only experienced this when using Llama3/fine-tunes released in the first week or so, where there were issues with the tokens or something, e.g.
- https://github.com/ggerganov/llama.cpp/issues/6770
- https://www.reddit.com/r/LocalLLaMA/comments/1cltac3/part3_cause_to_issue_found_possible_bug_llama3/
Recently I haven't ran into it again. So it's likely some kind of syntax/formatting-specific thing, as previously suspected. No idea if this was also the case for @xx88xx?
