sglang icon indicating copy to clipboard operation
sglang copied to clipboard

[Feature] Support llguidance for constrained decoding

Open JC1DA opened this issue 3 weeks ago • 17 comments

Motivation

This pull request integrates llguidance backend to extend guided decoding capabilities for sglang llguidance backend supports regex, json and grammar (lark or ebnf)

We have just released a large JSON Schema benchmark and a paper. Of particular interest might be isolated mask-generation benchmarks - comparing LLGuidance, Outlines, XGrammar and llama.cpp grammars.

image

Modifications

  • Added llguidance constrained backend to support masked decoding & json/ebnf/regex testcases.
  • We have plan to further support fast-forward decoding in the near future

Checklist

JC1DA avatar Feb 04 '25 22:02 JC1DA