Rémi Louf
Rémi Louf
I think it would make sense, vLLM benefits from structured generation and Outlines can re-focus on its main goals.
SGLang's code was copied from Outlines', they [just decided to import Outlines instead](https://github.com/sgl-project/sglang/issues/43#issuecomment-1935236868) and implemented the change. See also [this blog post](http://blog.dottxt.co/coalescence.html) that was published prior to theirs and explains...
The `seq_id` can probably be replaced with a hash of the token ids if that’s really the blocker. But that’s beside the point, even if we needed to pass `seq_id`,...
Happy to help!
We will likely need to wait until https://github.com/vllm-project/vllm/pull/2468 is merged on the vLLM side, which will relax the pydantic version.
It should soon be ok, vLLM merged https://github.com/vllm-project/vllm/pull/2531 that drops dependency on Pydantic v1.
vLLM released 0.3.0 which bumps the Pydantic version to >= 2.0. We'll cut a new release in Outlines today or tomorrow.
I think the high-level MFVI API is pretty much what we'd want it to look like
That’s the one. @albcab is there more?
Thanks! We're looking for something simple like your classification example (but only with the inference part). It's essentially to signal potential users that they can use Blackjax with GPJax easily....