keras-nlp icon indicating copy to clipboard operation
keras-nlp copied to clipboard

Add Mistral 0.2 models as possible presets

Open borisdayma opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe.

We can currently load Mistral 7b models with keras_nlp.models.MistralCausalLM.from_preset("mistral_7b_en") (or mistral_instruct_7b_en). I noticed those are the version 0.1 of the models. The versions 0.2 have significantly improved while using the same code base.

Describe the solution you'd like

It would be nice to offer the 0.2 variants of both base and instruct models as possible presets.

Describe alternatives you've considered

Just using the 0.1 versions or other libraries… However Keras offers nice integration with JAX + sharding!

Additional context

I want to use Keras models as backbones to train VLM’s and Mistral is a very strong variant at 7B.

borisdayma avatar Mar 15 '24 17:03 borisdayma