sglang
sglang copied to clipboard
[Feature] Support llguidance for constrained decoding
Motivation
This pull request integrates llguidance backend to extend guided decoding capabilities for sglang llguidance backend supports regex, json and grammar (lark or ebnf)
We have just released a large JSON Schema benchmark and a paper. Of particular interest might be isolated mask-generation benchmarks - comparing LLGuidance, Outlines, XGrammar and llama.cpp grammars.
Modifications
- Added llguidance constrained backend to support masked decoding & json/ebnf/regex testcases.
- We have plan to further support fast-forward decoding in the near future
Checklist
- [x] Format your code according to the Code Formatting with Pre-Commit.
- [x] Add unit tests as outlined in the Running Unit Tests.
- [ ] Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.