code-server icon indicating copy to clipboard operation
code-server copied to clipboard

[Bug]: intermittent reconnection

Open mouyong opened this issue 2 years ago • 4 comments

Is there an existing issue for this?

  • [x] I have searched the existing issues

OS/Web Information

  • Web Browser: Chrome
  • Local OS: Windows 11
  • Remote OS: Ubuntu 20.04
  • Remote Architecture: x86_64
  • code-server --version: v4.5.1

image

image

Steps to Reproduce

  1. insrtall the latest version of code-server
  2. open an project and wating sometimes.
  3. open the code-server develope tool, and you can see some log like the below example

image

Expected

can use code-server normal.

Actual

intermittent reconnection

received socket close event (wasClean: false, code: 1006, reason: ).

Logs

No response

Screenshot/Video

No response

Does this issue happen in VS Code or GitHub Codespaces?

  • [X] I cannot reproduce this in VS Code.
  • [X] I cannot reproduce this in GitHub Codespaces.

Are you accessing code-server over HTTPS?

  • [X] I am using HTTPS.

Notes

use nginx reverse proxy code-server, and nginx configure https.

mouyong avatar Jul 26 '22 01:07 mouyong

I'm seeing this as well after the update 4.5.1, 4.4.0 wasn't exhibiting this behavior with NGinx as the reverse proxy.

shawnweeks avatar Jul 26 '22 18:07 shawnweeks

I've been having the same issue since 4.5.0

diegosucaria avatar Jul 27 '22 13:07 diegosucaria

Yes, I've sam issue after update to 4.5.0 and 4.5.1

dualface avatar Jul 28 '22 05:07 dualface

Yes, i have the same problem with version 4.5.1 too.

Screen Shot 2022-07-30 at 8 02 52 PM

Help ?

jefftuschool avatar Jul 31 '22 03:07 jefftuschool

Running code-server 4.5.1 on macOS + Brave, I cannot reproduce:

image

Tried running behind Caddy with

# Caddyfile
http://localhost:8082/ide/* {
  uri strip_prefix /ide
  reverse_proxy 127.0.0.1:8080
}
image

but I'm not seeing that CloseEvent either.

Is this only happening with NGINX?

jsjoeio avatar Aug 01 '22 19:08 jsjoeio

@jsjoeio I think you need to run in HTTPs mode to get rid of those errors. I'm not seeing this on 4.5.1 when it's behind Caddy. Here is what I see in the nginx access.log and error.log when this reconnect happens. Not sure if it helps or not.

==> error.log <==
2022/08/01 22:34:11 [error] 1121#0: *62 connect() failed (111: Connection refused) while connecting to upstream, client: 10.0.99.11, server: dev-1.example.org, request: "GET /stable-30d9c6cd9483b2cc586687151bcbcd635f373630?type=ExtensionHost&reconnectionToken=0be3e07b-ccd4-48a7-8cdf-fbb316f364b9&reconnection=true&skipWebSocketFrames=false HTTP/1.1", upstream: "http://[::1]:8080/stable-30d9c6cd9483b2cc586687151bcbcd635f373630?type=ExtensionHost&reconnectionToken=0be3e07b-ccd4-48a7-8cdf-fbb316f364b9&reconnection=true&skipWebSocketFrames=false", host: "dev-1.example.org"

==> access.log <==
10.0.99.11 - - [01/Aug/2022:22:34:21 +0000] "GET /stable-30d9c6cd9483b2cc586687151bcbcd635f373630?type=Management&reconnectionToken=d6f0d7b1-1cc9-4ab1-bac4-9566a95b9db4&reconnection=true&skipWebSocketFrames=false HTTP/1.1" 101 16016 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:103.0) Gecko/20100101 Firefox/103.0" "-"

shawnweeks avatar Aug 01 '22 22:08 shawnweeks

It all in HTTPs mode. Screen Shot 2022-08-01 at 7 49 37 PM

jefftuschool avatar Aug 02 '22 02:08 jefftuschool

it keeps reconnecting cause screen flickering, any ideas? thanks!

Screen Shot 2022-08-03 at 11 25 40 AM

jefftuschool avatar Aug 03 '22 18:08 jefftuschool

I've been doing some testing and it appears that what's causing this is a change in behavior to the websocket keep alive between 4.4.0 and 4.5.1. If I add proxy_read_timeout 86400; to my Nginx configuration the issue goes away. By default Nginx has a 60 second read timeout which worked just fine on 4.4.0. Below is my working configuration but we need to figure out why that's no longer enough. I just don't know enough about how VSCode works to try and track this down.

server {
    listen 443 ssl;
    server_name dev-1.example.com;
    ssl_certificate /etc/pki/tls/certs/dev-1.cer;
    ssl_certificate_key /etc/pki/tls/private/dev-1.key;
    ssl_protocols TLSv1.2 TLSv1.3;

    location / {
      proxy_pass http://localhost:8080/;
      proxy_set_header Host $host;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection upgrade;
      proxy_read_timeout 86400;
      proxy_set_header Accept-Encoding gzip;
    }
}

shawnweeks avatar Aug 04 '22 00:08 shawnweeks

I changed timeout from 60 to 86400. Same results, still not working.

Screen Shot 2022-08-03 at 7 37 53 PM

jefftuschool avatar Aug 04 '22 02:08 jefftuschool

@jefftuschool Did you bounce nginx after making that change, I've only ever configured nginx from the shell so I'm not sure how your server is setup? Since making that change I haven't had a single reconnect on 4.5.1.

shawnweeks avatar Aug 04 '22 11:08 shawnweeks

I can confirm that setting proxy_read_timeout 86400; solves the issue

diegosucaria avatar Aug 04 '22 14:08 diegosucaria

I put the value 86400 and reboot the server. Yes, the problem is solved.

Thank you guys !!

jefftuschool avatar Aug 05 '22 03:08 jefftuschool

I've been doing some testing and it appears that what's causing this is a change in behavior to the websocket keep alive between 4.4.0 and 4.5.1. If I add proxy_read_timeout 86400; to my Nginx configuration the issue goes away. By default Nginx has a 60 second read timeout which worked just fine on 4.4.0. Below is my working configuration but we need to figure out why that's no longer enough. I just don't know enough about how VSCode works to try and track this down.

server {
    listen 443 ssl;
    server_name dev-1.example.com;
    ssl_certificate /etc/pki/tls/certs/dev-1.cer;
    ssl_certificate_key /etc/pki/tls/private/dev-1.key;
    ssl_protocols TLSv1.2 TLSv1.3;

    location / {
      proxy_pass http://localhost:8080/;
      proxy_set_header Host $host;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection upgrade;
      proxy_read_timeout 86400;
      proxy_set_header Accept-Encoding gzip;
    }
}

Thank you !

proxy_read_timeout 86400; it's working 💯

dualface avatar Aug 05 '22 10:08 dualface

There might be shorter values that work, like I said I'm not sure what causing the behavior change since other reverse proxies like Caddy don't seem to do this.

shawnweeks avatar Aug 05 '22 12:08 shawnweeks