Issue: recv() failed (104: Connection reset by peer) while proxying/upgrading WebSocket in NGINX with Socket.IO

[Naveen-Mallow]

Describe the bug

We're encountering the following NGINX errors while using it as a reverse proxy to a Node.js server running Socket.IO. These errors occur during WebSocket connections.

1. While proxying upgraded WebSocket connection:

*10088819 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: <internal-ip>, server: , request: "GET /socket.io/?EIO=4&transport=websocket HTTP/1.1", upstream: "http://127.0.0.1:8080/socket.io/?EIO=4&transport=websocket", host: "<hostname>"

2. While reading response header from upstream:

*13983419 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: <internal-ip>, server: , request: "GET /socket.io/?EIO=4&transport=websocket HTTP/1.1", upstream: "http://127.0.0.1:8080/socket.io/?EIO=4&transport=websocket", host: "<hostname>"

Environment

• Node.js: 18.x
• Socket.IO: 4.7.4
• NGINX: 1.28.0
• Deployment: NGINX as a reverse proxy to a Node.js server
• Transport: WebSocket (transport=websocket)
• Clients: Mobile app (Socket.IO client)

What We’ve Checked

• These errors consistently occur randomly.
• Socket.IO server logs do not show any corresponding exceptions or disconnect messages.
• Other HTTP (non-WebSocket) traffic through NGINX works without issues.
• No known memory or CPU bottlenecks on the Node.js server.
• No aggressive timeouts or buffer limits are configured in NGINX.

Current NGINX Configuration

location /socket.io/ {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host;

    proxy_pass http://nodes;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    proxy_connect_timeout 75s;
    proxy_send_timeout 75s;
    proxy_read_timeout 75s;
                
    proxy_buffering off;
    proxy_request_buffering off;
}

Questions / Guidance Needed

1. We’re seeking help to understand and troubleshoot this behavior. Specifically:
2. What are the common causes of recv() failed (104: Connection reset by peer) in a Socket.IO + NGINX setup, particularly during WebSocket upgrades?
3. Is it possible for the Socket.IO server to terminate a connection without logging it?
4. How can we distinguish whether the disconnect is triggered by the client, network, NGINX, or Socket.IO server?
5. What debugging strategies or additional logging/tracing can help us isolate the root cause?

Additional Context

• Issue is strictly limited to WebSocket transport.
• NGINX and Node.js server are hosted on the same machine (127.0.0.1 backend).
• Client devices are stable and do not indicate network issues at their end.
• Please let us know what diagnostics or traces from the Socket.IO server we can enable to better trace or isolate this.

Thanks in advance for your help!