node-red-contrib-kafka-manager icon indicating copy to clipboard operation
node-red-contrib-kafka-manager copied to clipboard

Kafka producer v0.6.2 producing null kafka messages from stringified json

Open DsmythMI opened this issue 1 year ago • 3 comments

"Kafka producer" in v0.6.2 producing "null" kafka messages from stringified json payload. Reverting to v0.5.1 resolves the issue
flow kafka_nulls

DsmythMI avatar Apr 19 '23 15:04 DsmythMI

I did do some major uplift of payload adding parameters that may be having wrong default behavior on upgrade of code. Around compression and fixed some potential problems.

What is not clear to me from what is sent and stated, the kafka node is at the end point of flow. Debug nodes imply this is occurring before the kafka node thus how could it be causing the issue.

If it is a problem with default values this is easily identified by editing the node and checking the values are as expected and saving if they needed to be modified.

peterprib avatar Apr 20 '23 08:04 peterprib

Hi There, The "Kafka producer" node is pretty much configured with the defaults, the node json is attached (see file).

kafka_producer_node_json.txt

The "stringify json" node stringify_function_node.txt

sends the msg.payload containing a string

msg_payload.txt

The "kafka producer" node in v0.5.1 seems to then correctly create a message in the kafka topic with the json string. Whereas at v0.6.2 a "null" message is created in the topic

DsmythMI avatar Apr 21 '23 11:04 DsmythMI

The way topic is set

const msgTopic = node.topicSlash2dot && msg.topic ? msg.topic.replace('/', '.') : msg.topic
const sendMsg = {
  topic: msgTopic || node.topic || '',

which implies the message being sent has the property topic and may be the issue. Is this property set? As re-arranged a bit, not sure if I made incoming msg topic take precedence over set topic and this could be the issue. Can remember thinkng about adding a option to always set to node.topic.

Note, produce has a option to stringify a payload which is JSON. Not sure when I put it in.

peterprib avatar Apr 22 '23 00:04 peterprib