canal icon indicating copy to clipboard operation
canal copied to clipboard

canal.mq.enableDynamicQueuePartition NullPointerException

Open elegance opened this issue 9 months ago • 1 comments

  • [ x] I have searched the issues of this repository and believe that this is not a duplicate.
  • [ x] I have checked the FAQ of this repository and believe that this is not a duplicate.

environment

  • canal version canal-1.1.8
  • mysql version

Issue Description

I want to automatically get the number of queues for a topic, so I set canal.mq.enableDynamicQueuePartition=true, but when receiving the binlog, a NullPointerException is generated when sending a message

Steps to reproduce

just set canal.mq.enableDynamicQueuePartition=true in instance.properties, don't set canal.mq.partitionsNum

Expected behaviour

Automatically obtain the number of queues corresponding to the topic

Actual behaviour

NullPointerException is thrown continuously

java.lang.NullPointerException: null
        at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:242)
        at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:173)
        at com.alibaba.otter.canal.connector.core.spi.ProxyCanalMQProducer.send(ProxyCanalMQProducer.java:51)

elegance avatar Mar 10 '25 07:03 elegance

rocketmq 模式下,这个 "canal.mq.enableDynamicQueuePartition=true" 参数应该是自动获取目标topic 的队列数量,但是未发送过消息,缓存中没有队列数,就会报空指针错误,除非设置 canal.mq.partitionsNum ,这样会语义不明,前面明确了动态获取。另外手工设置队列数可能不对,会导致第一次发送的消息未正确hash到预期的队列,继而影响顺序。 @agapple

elegance avatar Mar 21 '25 02:03 elegance