Rémi Louf
Rémi Louf
Sorry I somehow missed that issue, I will take a look shortly!
A few API tweaks are necessary to implement this properly. First, the code that transforms a regex into an index should be decomposed in: 1. A function that takes a...
We could store that in addition to the sequence weights (which can be, but are not necessarily, the log-probability of the sequence).
Great! It is fairly involved and there are many important design decisions that need to be made, and we need to handle computation of the KV cache after concatenating text...
> would like to get a sense of the scope of work involved so that I don't make promises I can't keep. It is fairly involved, interleaving function calls should...
Yes, that's the standard behavior of vLLM. We could possibly update the integration so it behaves similarly to the rest of Outlines. If someone wants to open a PR I'd...
It actually uses Outlines under the hood
Thank you for opening an issue! > Vision LLMs are also just LLMs that produce probabilities/logits over tokens, so my understanding is that they should also be compatible with outlines....
I think that for now it would be much faster / easier to add a logits processor in [this file](https://github.com/outlines-dev/outlines/blob/main/outlines/integrations/transformers.py) for multimodal models. I am soon going to change the...
> This is not intended to be merged, I was just wondering whether you think this is a promising direction to look into I think this is the right general...