REST proxy change request JSON format

[mithunmanohar]

This is more of a question. We are trying to connect an external system to Kafka REST proxy. But that system cannot post the messages in the format specified by REST proxy like: { "records": [ { "key": "recordKey", "value": "recordValue" } ] }

Is it possible to modify the REST proxy config to make it accept simple JSON message like { "key": "recordKey", "value": "recordValue" } ?

[ruslanruslanruslan]

Same problem. External systems don't support this crazy format. Why don't you allow free json messages?

[domibay-hugo]

I can only make a supposition but it seems that the JSON structure was meant for batching the data records like:

{"records": [
{"key": "recordKey1", "value": "recordValue1"}, 
{"key": "recordKey2", "value": "recordValue2"},
{"key": "recordKey3", "value": "recordValue3"}
]}

Batching can be desirable to mitigate the protocol overhead of HTTP and improve the data throughput.

To submit the data with 1 HTTP Request -> 1 Topic Record will multiply the protocol overhead with the growth of the data volume and lead to scalability issues much sooner.

[alpacino12]

I'm too facing the same issue , not able to consume data from external team in this format. Is there any resolution provided for it?