Misleading/wrong openapi schema in REST API docs for structured output

[oscarjohansson94]

System Info

Running image ghcr.io/predibase/lorax:0.9.0 in kubernetes.

Information

• [X] Docker
• [ ] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

Using the /generate endpoint.

When reading the swagger at: REST API, the property schema is defined as "schema": "{\"type\": \"string\", \"title\": \"response\"}",

This makes me believe that this parameter would be responsible for enforcing structured output, as described in Structured Output. However, when setting it to any schema (using pydantics .model_json_schema() or by passing a json schema directly) the schema is not enforced on the generated text. This is the only parameter in the swagger for this endpoints that I would relate to structured output.

However, if you pass the parameter "response_format", structured output is enabled. An example would be

{
  "inputs": "prompt",
  "parameters": {
    "adapter_id": "/some/adapter",
    "adapter_source": "local",
    "response_format": {
      "type": "json_object",
      "schema": {
        "$defs": ...,
        "additionalProperties": false, 
        "properties": ...,
        "title": "ExampleObject",
        "type": "object"}
    }
}

This is not present in the swagger for /generate or any other documentation that I could find for the endpoint. I only tested it because of the example in Structured Output, where a Python Client is used.

Expected behavior

I expected the documentation to more clear on how to enable structured output in the /generate endpoint.