Luca Beurer-Kellner comments

Results 149 comments of


                                            Luca Beurer-Kellner

[Question] Support constraining via context free grammar for dsls

We have some plans, but there is no concrete ETA currently. I will keep the issue to track the status. Do you have any concrete use cases in mind?

There isn't a distinction between model_name/identifier and model_path

Currently the string you use on the client side must match the server-side string exactly, so you cannot customize this. In the case where this includes paths, it indeed is...

Compiler: Subscript in f-string is interpreted as template variable

The current workaround is to factor out `dictionary[name]` into a separate variable, i.e. write the following instead: ``` value = dictionary[name] ' # {value}\n' # after putting this line in,...

[support Qwen models] AssertionError: Cannot intersect further patterns if '*' has already been handled.

Hi there, we have not tested LMQL with Qwen models yet, so this may be an issue with supporting its tokenizer. I will have to investigate a bit further.

[support Qwen models] AssertionError: Cannot intersect further patterns if '*' has already been handled.

You can fix the assertion by adding `if p2 == "*": break` after this line: https://github.com/eth-sri/lmql/blob/ab02526ddf9883aff4acda4e76c5a2a1cc136bf1/src/lmql/ops/follow_map.py#L241 However, I could not get the model to do inference on my machine, since...

Syntax Highlighter 0.0.3 (improvements)

Awesome work, this 0.0.2 just got merged via #246 and published on the Marketplace. Thanks again. Keeping this open for ideas around future improvements.

Docker for serving local model using lmql serve-model

Hi there, I haven't gotten around to document that, sorry about that. There is actually also a script that makes running Dockerfile.serve behave just like a local `lmql serve-model` command...

Minimal Syntax

0.0.6.5 added inline constraints, one line syntax (lmql.F) and the simplified syntax.

Allow distribution in the middle of a prompt

Are you looking just for a `VAR in [a,b,c,..]` constraint or are you interested in normalised probabilities across all possible values?

hope additional backend inference support for Exllama.

Yes, ExLlama seems very impressive. Do you know what kind of low-level interface they offer. LMQL requires full distribution access for masking and logprobs.