mobile client unstable when RocketChat client behind http/2 proxy

[jalbstmeijer]

Description:

I have been trying to fix a unstable connection from my Android mobile clients to my self hosted RocketChat server. All seems to work fine, but very frequently the client hangs (not all users with the same frequency) in an updating state, whereby only clearing the cache and sometimes killing the client makes it work again.

https://github.com/RocketChat/Rocket.Chat.ReactNative/issues/3169

Using the official RocketChat server Docker image. I tried different proxy docker images in front of it for SSL (in that order);

Traefik1 Traefik2 jwilder/nginx-proxy with the acme-companion

Till I came across https://github.com/RocketChat/Rocket.Chat/issues/15028

Making a direct curl http/2 connection to RocketChat on port 3000 works, but trying the same against Traefik/nginx results like in that issue in "Not a valid websocket request"

Forcing curl with --http1.1, makes it work also against the proxies.

Removing http2 from the "listen" line in the nginx config fixes it too.

My mobile clients are working fine now.

Environment Information:

• Rocket.Chat Server Version: 4.5.4
• Rocket.Chat App Version: 4.26

Steps to reproduce:

1. use Rocket.Chat mobile App a few days
2. find it hangs now and then on updating
3. change nginx proxy to not support http/2 and see that not happen again

Expected behavior:

no hanging on updating

Actual behavior:

hangs frequently on updating

[jalbstmeijer]

As it can be reproduced with curl, you would expect a fix is needed in the proxy config or at the RocketChat side.

But from the Rocket.Chat.ReactNative side, it seems a nice test scenario to make it handle such connection issues better or at least report something seems wrong at the server side instead of just hanging.

[feriarvis]

i got same problem

[jalbstmeijer]

Yes, for some reason only a limited amount of users is effected, otherwise this stuck on "updating" would get more attention. I stayed with nginx for now. But still occasionally (2 times a week) I need to clear my cache when my client hangs on "updating".

On Traefik 1/2 it's worse, then I get the issue much more frequently.

[feriferdinan]

Is there any evidence that the issue is on the mobile or on the server?

[jalbstmeijer]

@feriferdinan, no there is no evidence. The cause/trigger must be something on the server side (self hosted, using default docker images), otherwise much more users would complain.

The only thing that might be exotic in my configuration is that Docker is running in LXD on AWS.

I still have on my todo to try it outside LXD too. But after ruling out all possible proxies first as possible causes, Traefik1, Traefik2, nginx .. I gave it a rest.

The reason I post this as a mobile client issue, is that that is the only part showing the problem and just gets stuck in that "updating" state. So that behavior already seems to me a client bug on it's own. It should timeout, retry or at least stop and give a usable error.

I asked before in #3169 if I could provide more debug information, or if there is a mobile client debug version that will provide more information on the reason for hanging in updating state. But did not get any feedback.

[Feijai]

i got same problem too, is there any possible that user have lots of message (maybe over 1500+ message one day ) to stuck at updating ?

[jalbstmeijer]

Not sure which update fixed it.

But we haven't seen the issue for a couple of weeks now. We where even able to go back to Traefik instead of ningx.

server: 4.8.6 android client: 4.32.0.35745