endolith
Duplicate of https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python/issues/275
Maybe this is actually the reason for https://github.com/runpod-workers/worker-vllm/issues/233
I deployed `worker-vllm` v2.9.4, which uses the vLLM library v0.10.0. My goal was to test the functionality of the old `guided_json` API before the introduction of the `structured_outputs` API in...
> Maybe this is actually the reason for [#233](https://github.com/runpod-workers/worker-vllm/issues/233) No, I got Structured Outputs working now, but it still doesn't enforce order, whether using ````` GUIDED_DECODING_BACKEND=outlines ````` or ````` GUIDED_DECODING_BACKEND=lm-format-enforcer...
> no no no, why do you want to support an ancient version of python... See #1539
This was written by AI but seems to work, with the possible exception of the `structural_tag`. I need to check more carefully. It also doesn't enforce field order in json,...
> with the possible exception of the `structural_tag`. I think this was just a misunderstanding of the AI verifying the outputs? `structural_tag` only constrains text within the start and end...
Actually, it seems Structured Outputs are already supported and working (somewhat...) when using the OpenAI API, and I confused that with the RunPod Job API. I thought this was fixing...
Changed it to map the offline inference interface directly instead of mixing OpenAI API into Runpod API (and rebased onto https://github.com/runpod-workers/worker-vllm/pull/244)
Same problem for me but you can work around it by entering the number entry field