JC1DA

Cannot download feature extraction models

4

Hi, It seems that the google drive URL does not exist anymore and the OneDrive URL requires authentication. Can you take a look and reupload those models somewhere? Thanks a...

[Feature] Support llguidance for constrained decoding

17

## Motivation This pull request integrates [llguidance backend](https://github.com/guidance-ai/llguidance) to extend guided decoding capabilities for sglang llguidance backend supports regex, json and grammar (lark or ebnf) We have just released a...

[Frontend][Core] Add Guidance backend for guided decoding

14

This pull request extends guided decoding capabilities * Add [guidance](https://github.com/guidance-ai/guidance) backend `guidance` backend supports `regex`, `choice`, `json` and `grammar`. relevant: https://github.com/vllm-project/vllm/issues/5245 ## Usage * JSON Generation ```python from pydantic import...

Handle invalid utf-8 bytes in engine class instead of model class

4

This PR removes unnecessary invalid utf-8 bytes handling code in Model class as we can better handle it in Engine class.