keras-nlp
keras-nlp copied to clipboard
Add Mistral 0.2 models as possible presets
Is your feature request related to a problem? Please describe.
We can currently load Mistral 7b models with keras_nlp.models.MistralCausalLM.from_preset("mistral_7b_en") (or mistral_instruct_7b_en).
I noticed those are the version 0.1 of the models. The versions 0.2 have significantly improved while using the same code base.
Describe the solution you'd like
It would be nice to offer the 0.2 variants of both base and instruct models as possible presets.
Describe alternatives you've considered
Just using the 0.1 versions or other libraries… However Keras offers nice integration with JAX + sharding!
Additional context
I want to use Keras models as backbones to train VLM’s and Mistral is a very strong variant at 7B.