REST API should set content-type header for responses

[colinhicks]

Describe the bug Prior to the Jetty->Vert.x migration, the REST API included Content-Type headers in responses. This header is no longer returned. Downstream applications may leverage the header to interpret or route the response.

To Reproduce Steps to reproduce the behavior, include:

1. Submit a request to the /query endpoint in a way that allows you to see response headers, e.g. by passing -v to curl.
2. Observe that the response headers only indicate the protocol and Transfer-Encoding.

Expected behavior The Content-Type header should be included in responses, with the appropriate value. For example Content-Type: application/vnd.ksqlapi.delimited.v1

Additional context I noticed that EndpointResponse instances returned by API handlers generally don't set headers when a 200 is returned: https://github.com/confluentinc/ksql/blob/cd5254f9214fe0faadb98d505664f866805aedf0/ksqldb-rest-model/src/main/java/io/confluent/ksql/rest/EndpointResponse.java#L47-L49. It seems the handlers could instead also pass the appropriate content type.

[agavra]

@colinhicks can you help me understand the priority of this?

[colinhicks]

This was reported by a user in the Slack community. They reported the change was breaking their application after upgrading from an older Confluent Platform version.

It's also a best practice and increasingly useful as we add new options like the protobuf response type for pull queries.

[msestak]

Hi,

I'm running 7.2.0 version from confluent distribution and trying to set HSTS. This is version:

curl -s https://hostname:8088/info | jq
{
  "KsqlServerInfo": {
    "version": "7.2.0",
    "kafkaClusterId": "hq7lZrgHTg-s-5G7j3eacA",
    "ksqlServiceId": "default_",
    "serverStatus": "RUNNING"
  }
}

HSTS is set in:

cat /etc/ksqldb/ksql-server.properties
... 
response.http.headers.config="add Cache-Control: no-cache, no-store, must-revalidate", add X-XSS-Protection: 1; mode=block, add Strict-Transport-Security: max-age=31536000; includeSubDomains, add X-Content-Type-Options: nosniff

but is ignored:

curl https://hostname:8088/info -D -
HTTP/2 200
content-type: application/json
content-length: 132

{"KsqlServerInfo":{"version":"7.2.0","kafkaClusterId":"hq7lZrgHTg-s-5G7j3eacA","ksqlServiceId":"default_","serverStatus":"RUNNING"}}

This is security issue for us because HSTS is not enabled. Other Kafka services (REST, Schema registry and Kafka connect) respect and use response.http.headers.config value. It should return something like this. Example from Kafka REST:

curl https://hostname:8081/subjects -D -
HTTP/2 200
date: Thu, 04 Aug 2022 10:11:30 GMT
cache-control: no-cache, no-store, must-revalidate
x-xss-protection: 1; mode=block
strict-transport-security: max-age=31536000; includeSubDomains
x-content-type-options: nosniff
content-length: 2
content-type: application/vnd.schemaregistry.v1+json
vary: Accept-Encoding, User-Agent