cloudflared icon indicating copy to clipboard operation
cloudflared copied to clipboard

🐛 Long lived websocket seems to disconnect consistently

Open AquiGorka opened this issue 1 year ago • 7 comments

Describe the bug and To Reproduce A service is being exposed via a cloudflared tunnel (http endpoint). Clients connect to the service via the exposed tunnel which then upgrades the connection to a websocket. The clients consistently disconnect - and a successful reconnect happens since our clients can reconnect.

Expected behavior This only happens when the client connects via the tunnel, if no tunnel is being used the disconnect behavior does not happen.

Environment and versions

  • OS: Service: Linux and Macos
  • Architecture: arm64
  • Version: cloudflared version 2024.6.0 (built 2024-06-03T18:29:11Z)

Logs and errors When the socket disconnects we do get this output: Read error: websocket: close 1006 (abnormal closure): unexpected EOF

AquiGorka avatar Jul 01 '24 19:07 AquiGorka

This is actually not a bug, rather a limitation of the current production websocket impl, you can contact support and ask them to enable the Websocket handoff feature which can increase the lifetime of a websocket connection by an order of magnitude. the drawback of this is lesser visibility (you are unable to view total bandwidth of websockets in the zone)

flpydsk avatar Jul 29 '24 10:07 flpydsk

"When Cloudflare releases new code to its global network, we may restart servers, which terminates WebSockets connections."

Is this the reason? Any plans this will change? (soon ?)

Unless its something on my end sometimes its reconnecting every other minute which kinda makes it hard to make it unnoticed

rmedavm avatar Aug 01 '24 03:08 rmedavm

I don't work at cf and can't comment about any plans that may or may not exist but you can ask support to enable websocket handover

flpydsk avatar Aug 01 '24 03:08 flpydsk

"When Cloudflare releases new code to its global network, we may restart servers, which terminates WebSockets connections."

Is this the reason? Any plans this will change? (soon ?)

Unless its something on my end sometimes its reconnecting every other minute which kinda makes it hard to make it unnoticed

every other minute is quite often, Id think there is something else going on, possibly config somewhere. consider asking support or posting in discord/forums

flpydsk avatar Aug 01 '24 03:08 flpydsk

Its mostly fine but yes some days theres very "hot" 10 minutes of reconnects. Also find it hard to believe this is expected but only does it through the tunnel and at specific times of the day so

rmedavm avatar Aug 01 '24 04:08 rmedavm

I think we're facing this problem as well.

justjake avatar Mar 01 '25 23:03 justjake

We have the same issue, started after switching to Cloudflare websocket / tunnel.

This is actually not a bug, rather a limitation of the current production websocket impl, you can contact support and ask them to enable the Websocket handoff feature which can increase the lifetime of a websocket connection by an order of magnitude. the drawback of this is lesser visibility (you are unable to view total bandwidth of websockets in the zone)

We reached out to CF support about this, and activating the feature made no difference.

Also increased idle timeout duration on tunnel web settings, but no difference.

Unfortunatly, we'll not use it until Cloudflare provides a solution.

xmessier59 avatar Jun 17 '25 18:06 xmessier59