Juarez Bochi

That would be a great feature. The protocol allows this without string manipulation by the client, but only in v3. We would need to support it first (#22). ``` QUERY,...

@Shoshin23 Thanks for reporting this. I suspect you are still using an older version of `mlx` because I can't find this error message in the source. Now that https://github.com/ml-explore/mlx/pull/350 has...

@awni , do you still plan to make any changes to this? Or should we wait for https://github.com/ml-explore/mlx/pull/426 ?

Awesome! Thank you

What model are you using, @gladjoyhub ? Looks like it doesn't have `tokenizer.ggml.tokens` or `tokenizer.ggml.merges`, so the tokenizer cannot be initialized. I only tested this code with tinyllama.

> Would love some pointers to output in GGUF. I am trying to get a mlx lora finetuned model conversion to gguf for use in Ollama. Is there a example...

> @jbochi this is working now for Mistral and TinyLlama with native quantization. Let's merge it after we merge [ml-explore/mlx#426](https://github.com/ml-explore/mlx/pull/426) Amazing! Thanks for making it happen!

Hello, I've converted all the machine learning translation models to huggingface's transformers / safetensors. You can find them in this collection: https://huggingface.co/collections/jbochi/madlad-400-65491e6a78726cac9a4b84b7 ```python from transformers import T5ForConditionalGeneration, T5Tokenizer, GenerationConfig model...

@atillabasaran I can't reproduce the error. What version of `transformers`, `tokenizers`, and `sentencepiece` do you have? I tested with these: ``` transformers==4.35.0 sentencepiece==0.1.99 tokenizers==0.14.1 ```

No problem! I'm glad it worked. To choose the target language, just prepend the input text with `""`, where `xx` is the BCP-47 language tag. For example: ``` from transformers...