goaccess icon indicating copy to clipboard operation
goaccess copied to clipboard

Realtime Stats WebSocket Gets Disconnected After 1 Minute of Inactivity

Open Elepover opened this issue 5 years ago • 7 comments

Symptom

Every after exactly 1 exactly minute of inactivity, the WebSocket gets disconnected like this:

image

On servers with a large amount of requests, the problem may not appear for the stats are always being updated quickly and therefore the WebSocket is always active. But on small servers like this one, there is totally possible to have a 1-minute-no-request period.

There have already been an issue discussing about this problem: #1491, I'll talk about that in the Suggested Solution part.

Analysis

It's not GoAccess's fault.

It may be due to the limitation of Firefox, Chrome and possibly other browsers. They kill WebSocket connections that have been inactive for 1 minute. Reference: This StackOverflow Post

Or, it may be nginx's fault for the proxy_read_timeout field.

Suggested Solution

As is described in #1491, extending proxy_read_timeout does work, but it's not the perfect solution since there may be an even longer "no-request period". And extending the period is somehow "bypassing a security feature".

Here are some suggested solutions or "workarounds":

  • Implement a "heartbeat" mechanism on the backend to keep the WebSocket connection alive. (Which has already been mentioned in #1491)
  • Try automatically reconnecting the WebSocket on the frontend.

Edited for some typos :)

Elepover avatar Feb 02 '20 07:02 Elepover

Thanks for reporting this and pointing that out. Like you said (+ here), we could use pings to resolve this or just reconnect when you need to use the socket again. I'm more incline for sending a ping/pong frame.

I can look into this.

allinurl avatar Feb 04 '20 00:02 allinurl

I ran into the same issue with a low traffic domain and first thought that it was me doing something wrong.

Chi-Yu avatar Apr 20 '21 17:04 Chi-Yu

Ran into the same issue, ended up adding

window.GoAccess.wsKeepAlive = function(socket) {
    // jumpstart the "loop"
    keepAlive();

    // send data after 30s
    function keepAlive() {
        setTimeout(onKeepAlive, 30000);
    }

    // what a great time to keep alive
    function onKeepAlive() {
        // we got disconnected
        // we could reconnect or do nothing
        // for me it was enough to refresh the page
        if (socket.readyState == WebSocket.CLOSED) {
            return window.location.refresh();
        }

        // any data would suffice probably
        socket.send('keepalive');

        // continue the loop
        keepAlive();
    }
};

And then calling adding the keep alive after https://github.com/allinurl/goaccess/blob/master/resources/js/app.js#L92

@allinurl Want a pull request with this logic ? I'm thinking of adding a config value for the delay and the disconnect action.

Khez avatar Mar 29 '22 13:03 Khez

@Khez Thanks for sharing that. Just to make sure, did that solve the disconnect issue on your end?

allinurl avatar Apr 04 '22 22:04 allinurl

@allinurl I did see some disconnects, but it might have been my reverse proxy setup closing the connections.

Refreshing is a bit intrusive to the end user, a reconnect would make more sens, but I'm uncertain how data consistency is ensured thorugh goaccess - gwsocket

Edit worth mentioning: my initial setup was a PoC on a monitoring server, having 0 requests when no admin was around. I was usually getting disconnects below the 1 minute mark. With the above solution I held a 1 hour long socket connection with just the keepalive.

Need a PR? 😄

Khez avatar Apr 05 '22 01:04 Khez

@allinurl I had the same problem and it was completely solved with the code in the PR I just submitted (#2339). Any comments about it are welcome!

mario-donnarumma avatar Jun 15 '22 10:06 mario-donnarumma

Thanks a lot @mario-donnarumma. #2339 should address this issue and will be deployed in the upcoming release.

allinurl avatar Jun 15 '22 13:06 allinurl