Rémi Louf
Rémi Louf
Is this still the case on the latest release?
> Just for the record, this is still an issue for me, meaning that the newest version of Outlines I can use is v0.0.46, as all newer versions cause me...
#1386 integrates the last `outlines-core` update, and we haven't seen any performance of memory issue with it. I can take a look at the example you're struggling with @saattrupdan. @denadai2...
Fixing https://github.com/dottxt-ai/outlines-core/issues/178 should get rid of the vLLM performance issues.
This is supported in the v1 branch. You can either install outlines from it, or wait until we release it https://github.com/dottxt-ai/outlines/tree/v1.0
V1 has been released with support for hosted instances
@cpfiffer
I agree on all points. You can split this in smaller issues, and people can keep sharing their grievances here :)
As long as major inference libraries import `outlines` we shouldn't include any library by default. _However_ when the structured generation logic is in a separate library I am ready to...
Do you mind moving the resolution of this problem to a new issue? 🙂