mlc-ai
Unsupported json feature in xgrammar
I'm also trying to use vLLM to support guided decoding of my request. currently it's default guided backend set to xgrammar (which I thought having better performance). however, in my case, it falls back to 'outlines' because it detected some json feature are not supported by xgrammar. see https://github.com/vllm-project/vllm/blob/e92694b6fe264a85371317295bca6643508034ef/vllm/model_executor/guided_decoding/utils.py#L6
I'm wondering if this limitation still exists? my target json schema is like { "type": "object", "additionalProperties": false, "properties": { "POINT": { "oneOf": [ { "type": "integer", "minimum": 0, "maximum": 500 }, { "$ref": "#/$defs/Location", } ] } } "$defs": { "Location": { "type": "array", "items": { "type": "integer", "minimum": 0, "maximum": 1000 }, "minItems": 2, "maxItems": 2 } } }
I'm wondering why xgrammar did not support array and integer min and max range?
Hi @gryffindor-rr, thanks for asking about that. Currently, the min and max integer range is supported (we are enhancing it in #182 to better support negative integer range). The num of items is still ongoing. Please stay tuned!
We will sync with vLLM team to enable more features of json schema with XGrammar backend.
We will sync with vLLM team to enable more features of json schema with XGrammar backend.
I work on vLLM and try to watch the activity on this repo so I catch updates. Let me know if something is working that we haven't enabled! I'm about to turn regex on since it seems like we should be good in 0.1.12.
