Support for Structured Output for OpenAI API

[4davelo]

Hi,

First: Amazing work getting MLX-VLM out of the ground, a pleasure to play with VLMs on the Apple ecosystem.

Feature request: would be great to be able to force the VLM to respond in structured way. I tried using the text_format on the client.responses.parse OpenAI endpoint only to discover that the parameter was ignored and the VLM did not comply to the pydantic structure making it difficult to consistently parse the output.

Keep up the great work!

David

[Blaizzy]

Hey @4davelo!

Thank you very much!

Yes, very few VLMs can do structured outputs so we leave that to the developer using to handle.

If you have a good idea on how to make it work reliably across all models, feel free to submit a PR :)