Luca Beurer-Kellner

Hi there, thanks for creating an issue for this. Can you point to resources specifying the concrete API of AnyScale models. Like what kind of parameters and control over the...

Unfortunately, as documented by AnyScale at https://d3qavlo5goipcw.cloudfront.net/guides/migrate-from-openai, the OpenAI-like API they provide does not implement the features required for supporting LMQL. However, we will keep Anyscale on the list for...

Thanks for raising this. As explained in PR #25, there are some issue on the OpenAI end, which are hard for us to solve, although we are weighing off several...

Hi there Andre, can you recommend any resources on how torch.compile improves inference speed, with e.g. `transformers`. In general I am definitely not opposed to adding it.

Marking this as a good first issue. The feature can be added to https://github.com/eth-sri/lmql/blob/main/src/lmql/models/lmtp/backends/transformers_model.py, where an optional `lmql serve-model` argument can be set, such that compilation is done before model...

Hi Vivien. Your request sounds reasonable, I can see how the current setup may cause issues in some environments. `NO_CACHE` is currently only meant to force the runtime to not...

Thanks for reporting this. It seems to be a bug, I was able to reproduce it. The extra level of indirection via `yes_no_reasoning` seems to cause an issue in the...

Thanks for reporting this, the resources and pointers. Types are currently in preview, so we are definitely not fully there yet, with respect to reliability and features. Indeed non-deterministic branching...

Hi there :) `api_base` is reserved for Azure OpenAI configuration only. To change the general endpoint, you can just specify `endpoint=`. This should probably be aligned, such that api_base can...

Thanks for reporting this. Can you go into more detail what you mean about "running the function separately"? There is a limitation, where Python's `inspect` will not be able to...