MQTTX
MQTTX copied to clipboard
emqx
Connection dropped and auto-reconnect not working after publish
Describe the bug Sending a larger stringified content results in connection disconnect and auto-reconnect fails on every retry.
Expected behavior Sending of regular sized JSON content should be possible. The connection should not abort or at least it should auto-reconnect again
Actual Behavior Connection gets dropped, reconnect fails, message is not sent.
Log:
2023-05-09 22:50:02 2023-05-09 21:50:02 [13] ERROR /nanomq/nng/src/sp/transport/mqtt/broker_tcp.c:681 tcptran_pipe_recv_cb: size error 0x95
2023-05-09 22:50:02
2023-05-09 22:50:02 2023-05-09 21:50:02 [13] WARN /nanomq/nng/src/sp/transport/mqtt/broker_tcp.c:872 tcptran_pipe_recv_cb: tcptran_pipe_recv_cb: recv error rv: 149
2023-05-09 22:50:02
To Reproduce If possible, include actual reproduction test code here. Minimal C test cases are preferred.
Start docker container as mentioned on docker site. Use MQTTX or similar to send a message
UPDATE: Send with QOS 1 to reproduce
{
"msg": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxnnnnnxxxxxxxxxxxxxxxxxxxxx"
}
** Environment Details **
- NanoMQ version: 0.17 (latest docker image)
- Operating system and version: macOS (M1)
- Compiler and language used: typescript with recommended mqtt.js lib and also MQTTX
- testing scenario
Client SDK If possible include the mqtt sdk you used to connect to nanomq Minimal C test cases are perferred.
mqtt.js javascript lib and also via MQTTX UI
Additional context Add any other context about the problem here.
After playing around with the config, it seems to be an issue with default settings. Relates to emqx/nanomq#1126
I am glad you figured it out by yourself. Yes, 1024 bytes is too small for most scenarios. I will modify the default conf file now.
Is it expected, that auto-reconnect is not possible, if you exceed the size of a single message? As far as I've seen, the broker should only cancel the single message and not drop the whole connection. At least re-connect should be possible. If you have IoT-devices, it might hurt a lot, if they are going offline and never come back.
Nope. Auto-reconnect fully relies on the client itself. The broker has no power over it. So normal case should be: Client sends a packet exceed max limitation--- broker disconnect the client --- client reconnect again.
If the client cannot reconnect, I would suggest you look up to SDK code
are you sure @JaylinYu ? I'm on Version: v1.9.2 The fields in the connection settings are available, the UI shows the banners "Reconnecting" and you can see in broker log the re-connects. This seems to work as expected
Whoops, I neglect this new feature of MQTTX. it is likely MQTTX stops reconnecting. will dig further
just tried with mosquitto cli, it can reconnect automatically:
mosquitto_pub -h localhost -p 1883 -t topic_1 -l -d -q 1
Client wioudauisdh sending PUBLISH (d0, q1, r0, m1, 'topic_1', ... (7 bytes))
Client wioudauisdh sending CONNECT
Client wioudauisdh received CONNACK (0)
Client wioudauisdh sending PUBLISH (d1, q1, r0, m1, 'topic_1', ... (7 bytes))
Client wioudauisdh sending CONNECT
Client wioudauisdh received CONNACK (0)
Client wioudauisdh sending PUBLISH (d1, q1, r0, m1, 'topic_1', ... (7 bytes))
123123123Client wioudauisdh sending CONNECT
Client wioudauisdh received CONNACK (0)
Client wioudauisdh sending PUBLISH (d1, q1, r0, m1, 'topic_1', ... (7 bytes))
The reconnecting issue should be a bug of MQTTX, will inform them.
Hi, @sebastianwessel . Thank you for using MQTTX and NanoMQ, we appreciate your feedback. Regarding the issue you've experienced, I would like to confirm - have you enabled the auto-reconnect feature in your MQTTX config page? And if so, have you also set the right reconnection period? Please check these settings and let us know.
You can check this by clicking on the edit button in the top right corner 👇
@ysfscream Yes, I have same settings as you mentioned in screenshot. As soon as I send a message with too large content, the connection breaks, MQTTX goes in some re-connection loop. It is also not possible to stop somehow this loop - you need to close & re-open MQTTX in this case.
The auto-reconnect seems to work correctly in general - no matter which broker, but breaks at least on nanoMQ on too large messages. Tbh: I didn't test, what happens, if mosquitto or any other broker receives a message which is larger than expected/allowed.
I also used the mqtt.lib and played around a bit with timings, but this didn't help. As soon as there is a single message published, which is too large, it breaks the connection and the re-connect possibilities.
Might be an issue, that mqtt.lib caches the message in an internal queue and tries to send the message again and again as soon as the connection is open. So might be the case, that the flow looks like open connection -> (user) publish invalid message -> connection closed by broker -> auto-reconnect -> connection open -> (mqtt.js queue) publish invalid message -> connection closed by broker .....
This would make sense, because if you stop the program and re-start, the connection can be established and everything works well until you send an invalid message again. Might be, that in case the message is too large, a negative ack from broker with proper error code is expected here, instead of closing the connection or the connection is closed too fast, so the error message is not received by client. This means, there is no chance to know, that the message is invalid and needs to be removed from queue.
My program sends multiple messages during startup, the first ones are working ok (they are smaller) and then when the larger messages should be published it breaks.
@ysfscream I've tried to send the same message with QOS 0 In this case, the connection gets also closed on invalid messages, but the auto-reconnect works at least in MQTTX as expected.
If I publish the message with QOS 1, the auto-reconnect issue occurs.
So, it looks like that my assumption about the client-message-queue is the correct direction.