mlx-examples icon indicating copy to clipboard operation
mlx-examples copied to clipboard
verified publisher icon ml-explore

Support for Nvidia Nemotron and NVLM 1.0

Open vlbosch opened this issue 1 year ago • 1 comments

Nvidia recently released a few Nemotron models, like Llama-3_1-Nemotron-51B-Instruct. Converting it with MLX gives the following error: "Model type nemotron-nas not supported." Can the Nemotron-family be added, since they're bases on llama3.1 which is supported?

Furthermore, today they released NVLM-72b-D. I'm still downloading this model, but I suppose this one also needs to be added?

vlbosch avatar Oct 01 '24 14:10 vlbosch

Regarding the multimodal vision model, I'd recommend filing an issue on https://github.com/Blaizzy/mlx-vlm

Regarding the nemotron 51B, we could add it. I will mark this as a feature request.

Just FYI you can already use the nemotron 72B: nvidia/Llama-3.1-Nemotron-70B-Reward-HF since it has a supported model type.

awni avatar Oct 02 '24 20:10 awni