lmql
lmql copied to clipboard
eth-sri
A language for constraint-guided and efficient LLM programming.
I appreciate your effort for making LMQL. I've played with it a bit and found many of the concepts it does are similar to what Guidance does. But I'm not...
I'm using `serve_model_process = subprocess.Popen(["python" , "-m" "lmql.model.serve", args.model, "--cuda"])` to serve a model in the background of a python script. However, when I call `serve_model_process.terminate()` it doesn't properly terminate...
Is there a plan to incorporate key/value caching to improve generation efficiency significantly? See e.g. [Guidance's acceleration](https://github.com/microsoft/guidance#guidance-acceleration-notebook).
As a user, I would like a Google Colab notebook that demonstrates how to use LMQL with local models.
similar to INT(N)，I want to constraint the variable to be standard time format, like this: YYYY-MM-DDTHH:MM:SS is there a constraint implement this function?
In the following query, I am still required to write "\\n" instead of "\n" in order to be able to compile the query in the Python package (note that this...
The following query returns "What did the fish say when" instead of the expected "What did the fish say". ```python argmax(max_len=80) """A list of good dad jokes. A indicates the...
I have the following query: ```python sample(temperature=0.8, openai_chunksize=32, max_len=64, chatty_openai=True) "The movie review in positive sentiment is: '[OUTPUT]" FROM "openai/text-ada-001" ``` The OUTPUT variable in this case is thus constrained...
It would be very helpful to be able to have more advanced stopping conditions in STOPS_AT/STOPS_BEFORE. One use case for stopping conditions with lists instead of strings, is that: ```python...