text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Structured output doesn't work with open ai endpoint

Open Stealthwriter opened this issue 9 months ago • 1 comments
trafficstars

System Info

response_format doesn't work with open ai endpoint, please add it

Information

  • [x] Docker
  • [ ] The CLI directly

Tasks

  • [x] An officially supported command
  • [ ] My own modifications

Reproduction

just run a normal server and send a request with a structured response to the open ai endpoint

Expected behavior

it should return a structured output like what I define

Stealthwriter avatar Jan 27 '25 20:01 Stealthwriter