Help with using LoRA adapter weights on a converted Qwen2.5 model in MLX

[Developer-Jacob]

Hi team, I’m currently using a Qwen2.5 model that was originally written in Python, which I’ve converted to run with MLX and integrated into a Swift app. I’d like to use LoRA with it. When I try to load adapter_model.safetensors using LoRATrain.loadLoRAWeights, I get the following error:

unhandledKeys(base: "Qwen2Model", keys: ["base_model"])

Do I need to convert the adapter_model.safetensors file into an MLX-compatible format in order to use LoRA? If so, could you advise on how to properly convert it? For reference, the model appears to have been converted, but running LoRATrain.convert(...) didn’t seem to work as expected.

Thanks a lot in advance!

[davidkoski]

This means that there is a key base_model in the weights and there is no such key in the swift code. I am not sure what this key is for as I don't see anything matching it on the python side either.

You have a few options:

• disable the validation when loading the weights

  • specifically this call: https://github.com/ml-explore/mlx-swift-examples/blob/main/Libraries/MLXLMCommon/Load.swift#L90

• change the keys in the safetensors file

• use sanitize() to rewrite (or discard) the keys on load

  • for example: https://github.com/ml-explore/mlx-swift-examples/blob/main/Libraries/MLXLLM/Models/PhiMoE.swift#L230

[Developer-Jacob]

Thanks for the explanation!

Just to confirm — I'm trying to use LoRA adapter weights (adapter_model.safetensors) that were trained in Python (e.g., using Hugging Face's PEFT) with an MLX-converted Qwen2 model.

Is there any official tool or guide to convert the safetensors file into a format MLX expects (like .npz)?
Or should I manually extract the tensors and remap the keys to match MLX's model structure?

Any tips or example scripts would be really helpful!

[davidkoski]

safetensors is the native format for MLX, so that is fine. I wonder if PEFT produces the same structure of weights for the adaptors? @awni do you know about this?

You should be able to do LORA training with either mlx-lm (python) or mlx-swift-examples -- both support LORA and will produce the right keys for the weights.

[Developer-Jacob]

I think I may have caused some confusion — when I said “Python”, I was actually referring to PyTorch, not mlx-lm.

[awni]

I wonder if PEFT produces the same structure of weights for the adaptors?

No it doesn't.

You should be able to do LORA training with either mlx-lm (python) or mlx-swift-examples -- both support LORA and will produce the right keys for the weights.

Exactly.

[Developer-Jacob]

Got it — I trained the adapter using PEFT originally, so that explains the key mismatch issue.

Out of curiosity, do you have any plans to support converting PEFT-style LoRA adapters into a format that's compatible with MLX?

That would be super helpful for people coming from the PyTorch ecosystem.