Sketch guided constrained deciding for black box LLMs

[AriMKatz]

https://twitter.com/SaiboGeng/status/1749490603111387643?t=MRfYngCpJRB7zfZCpW43MA&s=19

Any thoughts about adding that?

[merrymercy]

This sounds very interesting. You can already implement the low-level logic with the current sglang. When this method becomes stable and mainstream, we can provide it as a default option in sglang.

[AriMKatz]

Cool @merrymercy. Also this tweet mentions that LMQL uses OAI's logit-bias API. is that something else that may be useful? I can open an issue

[github-actions[bot]]

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.