mlx-examples
mlx-examples copied to clipboard
ml-explore
Examples in the MLX framework
When using the latest LoRA training, the volatility of Loss became larger. When I further analyzed the cause, I suspected that it might have been introduced by commit #528 due...
I would like to add the chronos model(s). Looking for feedback or suggestion from the maintainers. - paper: https://arxiv.org/abs/2403.07815 - model: https://huggingface.co/amazon/chronos-t5-large
The LLM community seems to be moving towards the standard chat template, with many models starting to use the default chat template. Enabling the default chat template for mlx-lm may...
address https://huggingface.co/mlx-community/Mixtral-8x7B-Instruct-v0.1-hf-4bit-mlx/discussions/1
- Use SDPA for Mixtral - Use no repeat for phi - Store keys in fp16 for phi, phixtral Also closes #526
posted this in the discord (https://discord.gg/pEPVK6gGfW) but thanks to the awesome work of the [unsloth](https://github.com/unslothai/unsloth) team, they've identified some bugs in gemma implementations across the ecosystem: https://unsloth.ai/blog/gemma-bugs i think these...
Hi Team, Just solved the[ training problem](https://github.com/ml-explore/mlx-examples/issues/615) last night, but this time I failed to evaluate the the trained lora adapter with the following command ``` python3 -m mlx_lm.lora --model...
Updates the mlx_lm LoRA module's YAML configuration to include sections like the following: ```yaml schedule: join: boundaries: [101] schedules: - { name: linear_schedule, arguments: [0.0, 1e-5, 100] } - {...
I'm in the folder `./mlx-examples/llms` and run the fine tune command as the instruction (https://github.com/ml-explore/mlx-examples/blob/5a52899405ee911ce07ca3132e781ce149acd6b5/llms/mlx_lm/LORA.md). But there is the no attribute 'state' issue as following, ``` (mlx) I542464@DY4GPKX1J0 llms %...
Extract the `load_with_lora` function for easy use in other files.