Luca Beurer-Kellner

Hi Gregor, thanks for the suggestions. Additional backends are always welcome. We recently added first support for replicate.com which does allow you to run open models in the cloud (@charles-dyfis-net...

I think the behavior of `ast.parse` is actually correct here. In Python, `""""a"""` is valid, whereas `"""a""""` is not valid. This is because after reading `"""` a parser's scanner will...

Can you share more about your environment, e.g. can you provide the version of `transformers` in use. LMQL does not import 'Sequence' from 'collections' anywhere in its code base, so...

Looks like we are a bit too loose with our dependency requirement on bumpy, and an old version is incompatible. Could you try installing lmql with: ``` pip install lmql[hf]...

Experimenting more with `lmql serve-model` I found that another issue may be causing problems here. I can advise to experiment with the solution shown in the colab above, and I...

Petal *does* look very interesting and it seems it even offers full access to an LM hidden state. This should make it compatible with LMQL's constrained decoding. I will have...

It looks like Ray Serve does offer a relatively flexible access to the model. Most importantly, for support, we need access to the next-token distribution and a way to shift...

We intend to add more model backends over time. For Anthropic models, it looks like the API is still very restricted: https://docs.anthropic.com/claude/reference/complete_post. More specifically, it does not offer a way...

In addition to what @reuank said, you can also specify the default model for the playground on launch. For instance: ```bash LMQL_DEFAULT_MODEL='local:gpt2' lmql playground ``` Like, queries without `from` clause...

This sounds like there may be an issue with you `api_base` being an invalid SSL certificate. Can you confirm that `api_base` is the correct URL.