feat(template): read jinja templates from gguf files

[mudler]

Description

This PR adds automatic detection and parsing of jinja templates in gguf files. If we fail to identify a variant and we do not have already a specific template, it injects the jinja templates which is part of the model metadata if one is found.

Alternatively, it is possible to enable jinja templates manually in the model config file, in the template config section with jinja_template: true.

Notes for Reviewers

This is extracted from #3722 as it refactors message templating in a way that is more re-usable by other endpoints.

Signed commits

• [ ] Yes, I signed my commits.

[netlify[bot]]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	56f6ab4187b46a38b20485c34f6a904c3bc2ab97
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/67557dade0632500089601d4
😎 Deploy Preview	https://deploy-preview-4332--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

WIP as need to still add mapping between the transformer tokenizer and the templates (see TODO note in code comments)

[mudler]

basic support should work (tested with llama3 prompt), probably is not going to cover all cases as gonja has limitations, but, since this kicks-in when no other template was defined it is safe to merge without drawbacks.