Rate limit sending queued up messages from bridge?

[beville]

We've recently seen a scenario where the mosquitto bridge had queued up 20000 (a limit we set up the config) outgoing QoS1 messages. This is on a mobile device with limited/spotty connections. When the bridge was finally able to connect to to the remote broker, it overwhelmed it with a huge tidal wave of messages from the queue.

Does there exist a way to rate limit the outflow of queued up messages?

[ckrey]

Try max_inflight_messages set to e.g. relatively small number (10 or 20)

https://mosquitto.org/man/mosquitto-conf-5.html

[beville]

@ckrey Because we have multiple local mqtt clients sending messages (with sensor data that we don't want to lose) to the single mosquitto bridge, we want the bridge to hold onto all the messages until the cell connection is restored.

Unfortunately, max_inflight_messages is just a hard limit on the count of messages, and doesn't seem to affect the rate at which the messages are sent once the connection is made.

We were hoping to delegate the awareness of remote broker connectivity and message queuing to mosquitto. I suppose if the rate limiting doesn't exist yet, we will have to look into either patching mosquitto, or rolling our own bridge.

@ralight Off hand, how complex would it be to add this feature?

[cimplart]

I join in with this issue. We need to limit the rate at which mosquitto bridge dumps locally retained topics to an external broker.

[ckrey]

@beville would you like to retest the problem with a mosquitto.config on the bridge like:

MQTTv311

max_inflight_messages 10
max_queued_messages 20000
connection bridge
address <external-broker>:<port>
topic # out 1

Queue up a high number of messages while the external broker is not connected. When the connection is restored, the mosquitto bridge sends out (PUBLISH) 10 messages and then stops until it receives the first PUBACK. The it continues to PUBLISH the next queued messages and waits...

max_inflight_messages does affect the rate at which the messages are send out.

MQTTv50

If you use MQTTv5 by adding the following line to your bridge configuration

bridge_protocol_version mqttv50

The rate limit used will not be the max_inflight_messages value defined in the bridge configuration, but is negotiated in the CONNECT/CONNACK handshake between external broker and your bridge and will be limited to the minimum of both

Issue solved?

If I understand you right, this is the rate limit feature you are looking for.