kombu icon indicating copy to clipboard operation
kombu copied to clipboard

The producer.publish method does not handle the replyCode and replyText of AMQP. Resulting in message loss but without any error log! As a result, I lost some celery tasks.

Open zhaobingshuang opened this issue 5 months ago • 3 comments

If the message size sent by the producer.publish method exceeds the max_message_size (assumed to be 20 bytes) configured in rabbitmq.conf, the RabbitMQ server will generate an error log as follows. However, kombu will not throw an exception or print the error log. We will be unable to determine which messages have been lost. Occasionally, when using Celery, the problem of task loss is encountered. Eventually, it was discovered that this was caused by the aforementioned reasons.

2025-07-18 03:12:03.043934+00:00 [info] <0.1693.0> accepting AMQP connection <0.1693.0> (172.17.0.1:50460 -> 172.17.0.2:5672)

2025-07-18 03:12:03.047726+00:00 [info] <0.1693.0> connection <0.1693.0> (172.17.0.1:50460 -> 172.17.0.2:5672): user 'rabbit' authenticated and granted access to vhost '/'

2025-07-18 03:12:03.053160+00:00 [error] <0.1700.0> Channel error on connection <0.1693.0> (172.17.0.1:50460 -> 172.17.0.2:5672, vhost: '/', user: 'rabbit'), channel 1:

2025-07-18 03:12:03.053160+00:00 [error] <0.1700.0> operation basic.publish caused a channel exception precondition_failed: message size 21 is larger than configured max size 20

2025-07-18 03:12:03.080468+00:00 [warning] <0.1693.0> closing AMQP connection <0.1693.0> (172.17.0.1:50460 -> 172.17.0.2:5672, vhost: '/', user: 'rabbit'):

2025-07-18 03:12:03.080468+00:00 [warning] <0.1693.0> client unexpectedly closed TCP connection

zhaobingshuang avatar Jul 18 '25 03:07 zhaobingshuang

may be we can improve that?

auvipy avatar Jul 19 '25 08:07 auvipy

Can we check the replyCode and replyText of AMQP when the producer.publish method is called, and throw an exception?

zhaobingshuang avatar Jul 21 '25 08:07 zhaobingshuang

yes you can come with a pr which do that

auvipy avatar Jul 21 '25 11:07 auvipy