MLX engine support

Open onderceylan opened this issue 1 year ago • 1 comments

Hi there! First of all, amazing project - I'm a big fan! Thanks for the hard work.

About the request, I was blown away with the local inference speed via Apple Silicon (M1,M2,M3) on recent LM Studio release using https://github.com/lmstudio-ai/mlx-engine.

Is this something we can bring to the MLC AI community and the WebLLM project?

Oct 11 '24 19:10 onderceylan

Probably no, mlc on its own should replace mlx in terms of model inference

Aug 14 '25 07:08 SuperKenVery