Andrew Lapp

> > @RobinPicard are you still interested in implementing this? > > I can look at adapting it to the change made this week end That's great news! Please let...

@RobinPicard per my comment in the linked issue, it appears that transformers beam search submits an unused sequence group to logits processors during the final generation step. Is this still...

Glad that it's not blocking, please let me know if you run into any other issues or have any questions!

>I don't get how step 2 would work. Are the unhealed tokens passed down to the original Guide just the user prompt? >Another design I'm considering for better separation of...

> I don't get what the unaligned token means in this context. Sorry I wasn't clear here. For example, if the prompt is "hello wo" and we truncate it to...

Sorry for the delayed response. >but I thought the problem is that "rld" may not exist as a token in the vocabulary so it would not be found in the...

@rlouf what are your thoughts on replacing `outlines/integrations/transformers.py` with a set of logits processors classes (similar to `outlines/integrations/llamacpp.py`) and updating `examples/transformers_integration.py` accordingly? This will ensure consistency between different inference engines...

> @lapp0 I'm quite surprised that batching doesn't work for you, as I've been using this with batches just fine. The batch ID doesn’t need to be used in these...

> @lapp0 What I don't understand is maybe exactly what you mean by "replacing the transformers integration with logits processors". > > Logits processors have a different type signature than...

> @lapp0 But the transformers model classes don't accept a logits_processor argument. Are you talking about the outlines wrapper class here? Here's a dummy example that shows a logits processor...