outlines
outlines copied to clipboard
Consideration of support for mlx-vlm
Presentation of the new feature
Similarly to mlx-lm, mlx-vlm allows local Metal execution of MLX vision models. The project is used quite extensively and is growing quickly.
Just like with mlx-vlm, just using the Torch 'mps' backend is impractical due to hugely inefficient execution, so everyone running a vision model locally on Metal uses this library.
Where does it fit in Outlines?
This fits as a new integration, just like mlx-lm. I believe due to the high degree of similarity to the mlx-lm implementation, mlx-vlm will be less difficult to integrate than a new framework!