outlines icon indicating copy to clipboard operation
outlines copied to clipboard

Consideration of support for mlx-vlm

Open michaellee1 opened this issue 4 months ago • 0 comments

Presentation of the new feature

Similarly to mlx-lm, mlx-vlm allows local Metal execution of MLX vision models. The project is used quite extensively and is growing quickly.

Just like with mlx-vlm, just using the Torch 'mps' backend is impractical due to hugely inefficient execution, so everyone running a vision model locally on Metal uses this library.

Where does it fit in Outlines?

This fits as a new integration, just like mlx-lm. I believe due to the high degree of similarity to the mlx-lm implementation, mlx-vlm will be less difficult to integrate than a new framework!

michaellee1 avatar Oct 02 '24 19:10 michaellee1