when sending key in a topic message it gets converted to a string with escaped quotes

[ontehfritz]

curl -X POST -H "Content-Type: application/vnd.kafka.json.v2+json" \ -H "Accept: application/vnd.kafka.v2+json" \ --data '{"records":[{"key":1, "value":{"name":"event","content":"testing_curl"}}]}' "http://localhost:8082/topics/test-topic" key value: "1" <--- converted 1 to string

curl -X POST -H "Content-Type: application/vnd.kafka.json.v2+json" \ -H "Accept: application/vnd.kafka.v2+json" \ --data '{"records":[{"key":"1", "value":{"name":"event","content":"testing_curl"}}]}' "http://localhost:8082/topics/test-topic" key value: "\"1\"" <--- converted string to escaped string with quotes

This only happens through the rest proxy. TCP clients work fine without escaped quotes and just the string value. How can you get the rest proxy not escape the string value?

[Diom]

I also have this issue. String keys are encapsulated in quotes. I cannot find a mechanism to change this.

Edit: The issue also occurs for the value field. Given a numeric value it is translated to a string version of e.g. "1". Given a string value it is translated to a string encapsulated in quotes e.g. ""1"".

Edit: Does NOT happen when the values are part of an object: Could not handle key=["1"], value=[{"A":1}] (here the key should be displayed a simply 1).

I think that the issue may be here where the Base64 decode is happening.

[OlivierVandenNest]

you should set content type to Content-Type: application/vnd.kafka.binary.v2+json, keys will be exactly what you give them

the other one Content-Type: application/vnd.kafka.json.v2+json will have escaped quotes such that it properly deserializes

[kayshn]

You can do this by sending a Base64 encoded key and value attributes while setting the content type header to application/vnd.kafka.binary.v2+json. See below for an example:

curl -X POST -H "Content-Type: application/vnd.kafka.binary.v2+json" \ -H "Accept: application/vnd.kafka.v2+json" \ --data '{"records":[{ "key": "dGVzdC1rZXktdmFsdWU=", "value":"S2Fma2E="}]}' "http://localhost:8082/topics/your-topic-name" This will create a message in the topic with raw values (decoded) and the key will not have the double quotes around the value in Kafka.

[tiphdousset]

Hello, I've the same issue as described by ontehfritz --> the rest proxy is converting my key (String) into an escaped string with quotes.

I've tried the solution suggested by Olivier and Kayshn (to set Content-Type: application/vnd.kafka.binary.v2+json) but it doesn't solve my issue --> it simply doesn't succeed anymore to produce my message into the kafka topic.

Any other suggestion?