Luca Beurer-Kellner

This is how caching is actually implemented with OpenAI. However, with `sample` I think caching does not apply, since it is typically not seeded. With the `seed` parameter, it could...

State handling looks good now thanks. I just wanted to test with `"meta-llama/Llama-2-7b-chat-hf"`, however, it seems the start/end tag extraction does not work very well there. First of all, some...

The problem with the Llama template seems to be that it does not allow all tag interleavings as expressible with our current {:role} syntax. If users specify e.g. two system...

Thanks for suggesting this, it it is definitely a good idea. I will keep this issue to track progress/work on this.

Marking this as a good first issue. A query request builder would just construct an LMQL string, that is then passed to the LMQL compiler to produce a callable LMQL...

I just pushed support for "openai/gpt-4-1106-preview" to `main`, which should now work out of the box. For other models that raise a similar issue, you can now also specify that...

Thanks for reporting this. Any more concrete way to reproduce this would be much appreciated. I have seen this kind of issue before and it typically is caused by unexpected...

Unfortunately, current chat templates are very different between models, which makes it hard to support them all under a unified abstraction. However, what is always possible is to simply include...

Thanks for reporting this. `@dataclass` constraints are still in preview and a work-in-progress, so this is valuable feedback. Thanks also for the links.

On the first one I don't get warnings on logit bias limitations, but the second one is due to the integer constraining. This is an API limitation from the OpenAI...