[Feature Request] Please add support for Mistral-7b-instruct-v0.2

[jazir555]

Feature request

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2

Mistral Instruct .2 has been released. When you can, could you please update the model in GPT4ALL?

Motivation

A new version of the Mistral 7B model has been uploaded.

Your contribution

Unfortunately nothing I know how to do :(

Edit: Somewhat related, I'm having difficulty increasing the token amount for Mistral .1 (the uncensored model) to allow for longer responses. In the settings for the model, I greatly increased the amount of tokens it's allowed to generate, but it seems to have had no effect on the output length.

[Hsn37]

You can use any GGUF files with gpt4all.

Download the 0.2 GGUF file from this link.

When initializing your model, simply pass the path to this checkpoint.

[DistantThunder]

You can use any GGUF files with gpt4all.

Download the 0.2 GGUF file from this link.

When initializing your model, simply pass the path to this checkpoint.

Although the quantizations are not supported for GPU accelerated inference right?

I'm trying to use Q5_K_M and gets "model or quant has no GPU support" (AMD 7900XTX, Linux).

[cebtenzzre]

Although the quantizations are not supported for GPU accelerated inference right?

I'm trying to use Q5_K_M and gets "model or quant has no GPU support" (AMD 7900XTX, Linux).

Only Q4_0 and Q4_1 quants are supported with Vulkan atm, and Q4_1 is not recommended for LLaMA-2 models such as Mistral.