emqx
[Feature] Add the option to automatically collapse large messages
Motivation
Large messages (read about ~100MB) are fetched and displayed as is by MQTTX. Not only is this not practical but it also makes the application lag and even crash.
Detailed design
There should be a limit for the maximum length for messages being displayed. If that limit was exceeded then the message should be automatically collapsed.
Thank you for bringing this to our attention. We hadn't considered the display issue with large messages before. I will take your suggestion into account and work to implement it as soon as possible.
@TommyDuc Hi, before we optimize this, could I ask you two questions?
- What is the scenario that requires such a large size of data?
- What is the current performance and show of MQTT X when receiving and sending this large amount of data?
We want to be able to reproduce this when optimizing fully. Thanks
@ysfscream
- The scenario is that the MQTT broker is used to transfer files (read firmware updates). We know that MQTT was not meant to transfer files but it does support messages up to 256MB :) and so it was quick and cheap way to do it.
- For uploading I haven't tried it but when file-messages, which are often retained, then opening MQTT X on the file topic causes it to freeze for multiple minutes or even crash.
OK. Thanks for your details. We will optimize this data display, but you can wait for the large file transfer function of EMQX, which will work very well for you!
It's updated to https://github.com/emqx/MQTTX/releases/tag/v1.10.0, welcome to try and thanks again.