neurallambda

Same problem, I've left a comment on a related ticket: https://github.com/guidance-ai/guidance/issues/454#issuecomment-1878149397

@MINGYUK good to know thanks, unfortunately llama.cpp doesn't work with `gptq`

I would personally prefer more control in my hands, and less opinions baked in (eg per my suggestion to have a server *library*, where I just have to feed in...

Thanks for the fast response! I guess I'm confused at the docs then: https://docs.lmql.ai/en/stable/language/constraints.html > To do so, LMQL supports the STOPS_AT constraint. It takes two arguments, the first is...

Regarding my `aside:...` in there, I added an issue that addresses the failure of args to flow through to model instantiation: https://github.com/eth-sri/lmql/issues/230

Hm, I left a process running all day after changing to `n=1` in [that PR](https://github.com/eth-sri/lmql/pull/229), and looking at the logs they're full of loading and unloading the model each time...

I would also expect it to stay on the last used model. And to make matters worse, the logs suggest it's loading and unloading twice per generation.

Cool cool, I just made a discord for the group, there's some other interest that will likely trickle in as well. https://discord.gg/qSB6j76PX I'm only just started getting this "out there",...

I'm interested in this solution too, as I was having the same parserllm issues as in: https://github.com/r2d4/parserllm/issues/4 Also, the [outlines](https://github.com/normal-computing/outlines) project may interest you. They precompile valid continuations, and then...

For some reason it sounds like you don't have a `uniteai.yml` file. If you have added `uniteai` to your editor, upon first running it should create this file and ask...