MQTT keepalive less than 60s and chrome 88 intensive timer throttling

[dirkkessler]

• it would be great to have some documentation regarding best practices in regards to keepalive and chrome intensive timer throttling
• it is not uncommon for our application tab to be put in the background by our users since they can also spend a fair amount of time in other web applications
• our keepalive is less than 60s since we need to know fairly quickly when the connection with the broker has been severed since we require timely message delivery
• chome 88 only executes timers every minute during intensive timer throttling which means our keepalive does not happen within the configured time frame which in turn causes disconnects
• please advise, thanks

https://docs.google.com/document/d/105Hlbcb1mTV06bT9aDpwBotY13eZiKV5Hl4PW5LlyeE/edit https://developer.chrome.com/blog/timer-throttling-in-chrome-88/

AB#9481421

[Eric24]

Just to add a bit of detail to this, it is absolutely an issue. It only happens during long periods of idle time (i.e. no message traffic), where the only traffic is pingreq/pingresp. The default keepalive setting of 60 seconds causes the pingresp to be "missed", so the connection is closed (perhaps this could be fixed internally to mqtt.js, to look for the pingresp message before closing the connection, which I'm assuming is driven by some kind of timer event).

But the quick fix is to extend the keepalive timer; I used 120 seconds (but it's possible that 90 seconds or something else between 60 and 120 seconds would also work).

Once Chrome starts its "intensive timer throttling", is a noticeable delay develops between pingreq and pingresp (which is usually sub-second with our broker), but with the longer keepalive interval, the pingresp message is "seen" before the connection is closed.

[kroutal]

Hi, Paho-mqtt js is not impacted by this issue. I noticed that paho-mqtt js library reschedules the ping on pingresp . paho-mqtt.js It seems that reshecheduling the pings on "packetreceive" solves this issue

client.on("packetreceive", function() {
    client._shiftPingInterval();
});

On another thing would be to solve the reconnection which will not occur on reconnectperiod because of the throttling.

One thing that could be done would be to shedule the reconnection on the close event

client.on("close", function() {
    //schedule client._reconnect();
});

[carloscolombo]

In my case, as a work around, I offloaded the keepalive timer logic to a Web Worker. Web Workers are not throttled by the Chromium's Intensive Timer Throttling policy. I did the following:

Examples are in TS

1. Set the keepalive timer to 0 to opt out of the library's internal timer that is being throttled.

this.client = mqtt.connect(myUrl, {
    [...],
    keepalive: 0,
    [...]
})

2. Timer logic that will be used by Web Worker

/**
 * This Web Worker logic will notify our application to ping our MQTT connection to keep it alive.
 *
 */
function keepAlive() {
    const ctx: Worker = self as any;
    setInterval(() => {
        ctx.postMessage('keep-alive'); // will post to our application, doesn't need to be named 'keep-alive' can be anything
    }, 15000); // every 15 seconds
}

3. Create Web Worker to use the timer logic above. In my case I turned the function above as a Blob. You don't have to, you can use the standard URL way of creating a Web Worker.

/**
 * Creates a web worker via blob (instead of file)
 *
 */
function createWebWorker(func: Function) {
    return new Worker(URL.createObjectURL(new Blob(["(" + func.toString() + ")()"], { type: 'text/javascript' })));
}

4. Now in the connect event listener, create the Web Worker and add the listener to pingreq our connection.

this.client.on('connect', () => {
        // create web worker using our "keepAlive()" function
        this.keepAliveWorker = createWebWorker(keepAlive);

        // event listener for our Web Worker
        this.keepAliveWorker.addEventListener('message', (event) => {
            // ping our mqtt connection to keep it alive
            (this.client as any)._sendPacket({ cmd: 'pingreq' });
        })
});

5. Clean up Web Worker on disconnects

this.client.on('close', () => {
        // clean up keep alive web worker scheduler
        this.keepAliveWorker.terminate();
});

Hopefully this can help someone else as it did for me. Please point out any flaws in my logic to help myself and others that my run into this issue.

[pimenas]

@carloscolombo I see the following flaws in your solution

• By setting keepalive to 0, on the mqtt connect options, you also disable the server side checking. According to spec

If the Keep Alive value is non-zero and the Server does not receive an MQTT Control Packet from the Client within one and a half times the Keep Alive time period, it MUST close the Network Connection to the Client as if the network had failed

• I don't see any checking that the ping response was received. Maybe instead of _sendPacket you could call _checkPing (and also set pingResp to true initially).

[sunmd]

I also encountered this problem, thanks for the reminder. I seem that setting the pingResp on "packetreceive" solves this issue.

 this.client.on("packetreceive", () => {
     this.client.pingResp = true;
 });

[erichaus]

@kroutal could you explain what that's actually doing?

    client._shiftPingInterval();

[kroutal]

@hoIIer

This is the current workaround I'm using:

....
this.mqttClient.on('packetsend', this.onPacketSend());
...
private onPacketReceived(mqttClient: MqttClient): OnPacketCallback {
  return (packet: Packet) => {
    //Used to fix chrome issue 
    if (packet.cmd == 'pingresp') {
      (mqttClient as any)._shiftPingInterval();
    }
  };
}

Each time a ping message is sent by the client. The server will send back a ping response and trigger the 'packetsend' callback. Since this is done through a websocket this will not be impacted by the throtling. Within the callback the call to _shiftPingInterval() will restart the ping timer. The throtling will only start after the second tick of the timer.

[github-actions[bot]]

This is an automated message to let you know that this issue has gone 365 days without any activity. In order to ensure that we work on issues that still matter, this issue will be closed in 14 days.

If this issue is still important, you can simply comment with a "bump" to keep it open.

Thank you for your contribution.

[github-actions[bot]]

This issue was automatically closed due to inactivity.

[robertsLando]

Hi everyone. I started helping maintaining this library. It would be super helpful if someone of you could write a solution to this on docs and/or even better try an implementation of one of the proposed solutions. We already have a var named IS_BROWSER that we use to detect browser env, that in combination to a specific connection option could be used to add the fix needed on browser

[swapnonil-b]

@robertsLando Perhaps we could use worker-timers https://www.npmjs.com/package/worker-timers when IS_BROWSER is true. Timers from this package are meant to be run in webworkers and thus be immune to the throttling from browsers.

[swapnonil-b]

Also in terms of @carloscolombo 's solution, we can also use the worker-timers setInterval to handle things. Keeping most of the steps the same, but instead of spawning the web worker ourselves, we can just do this on the connect callback.

import { setInterval, clearInterval } from 'worker-timers';

...

this.client.on('connect', () => {
      // do your other stuff

     this.timer  = setInterval(() => {
          (this.client as any)._sendPacket({ cmd: 'pingreq' });
        }, 15 * 1000);
});

you would need to handle the clearInterval on the disconnect.

[robertsLando]

Problem is we are actually using reInterval package to handle ping scheduling. We should drop retimer or think about adding workers support on that module when running on browser. Would you mind to submit a PR?

[robertsLando]

I have created an issue there: https://github.com/mcollina/retimer/issues/10. I thinnk if we add the support there then we can import the setInterval and setTimeout functions directly from that package instead of adding worker-timers dep also on mqttjs

[robertsLando]

Ok retimer@4 has been released with that change. Let me do a bump+release here 🚀

[RainfoxAri]

@robertsLando Any update? I didn't see it in the commit history and also not in the release notes, that's why i'm asking. :)

[robertsLando]

I tried to use retimer latest but many tests fail then and I cannot find out why...

[oliverunger]

Can you give an update. When will this problem be fixed?

[robertsLando]

I don't have an ETA for this, would you like to submit a PR?

[jellybins]

who can tell me this issue has been fixed in which release version？

[robertsLando]

@jellybins I submitted a possible fix 4 months ago in version 5.0.0. I also tried to use the retimer solution without success

[oliverunger]

@jellybins I switched to eclipse paho javascript and that works great.

[robertsLando]

Yeah that's cool last commit was 4 years ago on paho mqtt: https://github.com/eclipse/paho.mqtt.javascript

[oliverunger]

For me: works but old > doesn't work but fancy

[robertsLando]

@oliverunger I'm still waiting for someone to give me feedbacks about this, It could be it is working now...

[jellybins]

@oliverunger I'm still waiting for someone to give me feedbacks about this, It could be it is working now...

#1361 in this issue ，some one tested the version 5.0.3 still have same problem

[robertsLando]

latest is 5.3.1, also mine was just a supposition that error was related to this, I need someone that finds a reliable way to reprouce this issue so I can fix it

[jellybins]

latest is 5.3.1, also mine was just a supposition that error was related to this, I need someone that finds a reliable way to reprouce this issue so I can fix it

version 4.x （such as the last 4.3.7） can reliable reproduce this issue ；and i could not upgrade to version 5.x because using vue2

[jellybins]

just use this, use the newest google chrome browser and just connect, no need to subscribe , when connected open the preview console and change the browser tab to background, then about every 5 minutes you'll get a disconnect console msg; Ps: i have tried to change the mqtt version to the newest ,but it went wrong;

[robertsLando]

and i could not upgrade to version 5.x because using vue2

What? All my projects are using Vue 2 and I use mqttjs without any problems...

Ps: i have tried to change the mqtt version to the newest ,but it went wrong;

So at the end you have been able to try latest version and it still fails?