Support for ERNIE-4.5-VL-28B-A3B-Thinking in MLX-VLM

[ethan7zhanghx]

We’re part of the ERNIE team at Baidu. Many of our users have been asking for an MLX-compatible version of the ERNIE-4.5-VL-28B-A3B-Thinking so they can run it locally on Apple Silicon through MLX-VLM.

At the moment, we’re exploring this direction , but we’re not yet familiar with the integration approach for porting VLM into MLX-VLM.

we would like to understand: • whether MLX-VLM or community contributors are open to adding support for this model • what information or model details we can provide to make the port easier

In particular, we expect that once ERNIE-4.5-VL-28B-A3B-Thinking is integrated, the same approach would apply to ERNIE-4.5-VL-28B-A3B and ERNIE-4.5-VL-424B-A47B, as they share the same architecture.

We’d be glad to collaborate closely if any community contributors are interested in leading the porting work, and we can provide model details, architecture specs, and validation support as needed.

Thanks for considering this request!

[Blaizzy]

Hey Ethan

Thanks for reaching out!

Prince here, I'm the creator and maintainer of MLX-VLM.

I have been looking into your latest awesome releases and am happy to adding support to not only your VLMs but your newest Omni model too :)

All I need to get started is transformers or Torch source code.

You can also add me to slack or discord for faster communication and so we can collaborate on future releases.

Email: [email protected]

[ethan7zhanghx]

Hey Ethan

Thanks for reaching out!

Prince here, I'm the creator and maintainer of MLX-VLM.

I have been looking into your latest awesome releases and am happy to adding support to not only your VLMs but your newest Omni model too :)

All I need to get started is transformers or Torch source code.

You can also add me to slack or discord for faster communication and so we can collaborate on future releases.

Email: [email protected]

Thanks so much for the quick reply!

I’ve already sent you a Slack invite — hope it reaches you smoothly.

Regarding the “Omni” model you mentioned, I believe you’re referring to ERNIE 5.0. This model isn’t open-sourced yet, but if we plan to open it in the future, we’ll be very happy to sync with you so we can collaborate on MLX support together.

For the VLM models, all Transformers / PyTorch source files for our VLM are already fully open-sourced on Hugging Face, including the architecture definitions and processing code.You can find everything here: https://huggingface.co/baidu/ERNIE-4.5-VL-28B-A3B-Thinking

If anything is missing or unclear, we’re happy to clarify or provide additional details.

Really appreciate your interest in supporting our models — looking forward to working together!