terminal icon indicating copy to clipboard operation
terminal copied to clipboard

Canary Build!!! Many attempts to SSH to another machine will result in a hung request.

Open akulbe opened this issue 1 year ago • 5 comments

Windows Terminal version

1.22.2351.0

Windows build number

10.0.22635.0

Other Software

OpenSSH_9.8p1, OpenSSL 3.2.2 4 Jun 2024

Steps to reproduce

I have an alias for a host defined in ~/.ssh/config called "workbits"

Attempt to SSH to that alias:

ssh workbits

Expected Behavior

for ssh session to the "workbits" machine to be established

Actual Behavior

The session hangs.

akulbe avatar Aug 23 '24 14:08 akulbe

␛[1t␛[?1004h␛[?9001h␛[1t␛[18;56;0;1;259;1_␛[I␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_PowerShell␣7.4.5␍␊ ␛]0;Administrator:␣PowerShell␛\␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;1;259;1_␛[18;56;0;0;2;1_␛[18;56;0;0;0;1_␛]0;Administrator:␣PowerShell␛\What␣kind␣of␣pants␣do␣ghosts␣wear?␣Boo␣jeans.␛]0;Administrator:␣PowerShell␛\␛[0m␛[0m␍␊ ␍␊ ␛[0m␛[0mLoading␣personal␣and␣system␣profiles␣took␣1192ms.␍␊ ␛[38;2;255;238;88m␣␣␛[0m␛[38;2;255;255;255m␣Aaron␣Kulbe@smallkraken␣␛[0m␛[0m␛[38;2;145;221;255;49m␛[7m␛[0m␛[48;2;145;221;255m␛[38;2;16;14;35m␣~␣␛[0m␛[38;2;145;221;255m␛[0m␣␛[83;31;115;1;0;1_␛[?25l␛[5;35H␛[0m␛[93ms␛[0m␛[97;2;3msh␣workbits␛[0m␛[39;49m␛[0m␛[5;36H␛[?25h␛[83;31;115;0;0;1_␛[83;31;115;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mss␛[0m␛[97;2;3mh␣workbits␛[0m␛[39;49m␛[0m␛[5;37H␛[?25h␛[72;35;104;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[97;2;3m␣workbits␛[0m␛[39;49m␛[0m␛[5;38H␛[?25h␛[83;31;115;0;0;1_␛[32;57;32;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[97;2;3mworkbits␛[0m␛[39;49m␛[0m␛[5;39H␛[?25h␛[72;35;104;0;0;1_␛[32;57;32;0;0;1_␛[87;17;119;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mw␛[0m␛[97;2;3morkbits␛[0m␛[39;49m␛[0m␛[5;40H␛[?25h␛[79;24;111;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mwo␛[0m␛[97;2;3mrkbits␛[0m␛[39;49m␛[0m␛[5;41H␛[?25h␛[87;17;119;0;0;1_␛[82;19;114;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mwor␛[0m␛[97;2;3mkbits␛[0m␛[39;49m␛[0m␛[5;42H␛[?25h␛[79;24;111;0;0;1_␛[82;19;114;0;0;1_␛[75;37;107;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mwork␛[0m␛[97;2;3mbits␛[0m␛[39;49m␛[0m␛[5;43H␛[?25h␛[75;37;107;0;0;1_␛[66;48;98;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mworkb␛[0m␛[97;2;3mits␛[0m␛[39;49m␛[0m␛[5;44H␛[?25h␛[66;48;98;0;0;1_␛[73;23;105;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mworkbi␛[0m␛[97;2;3mts␛[0m␛[39;49m␛[0m␛[5;45H␛[?25h␛[73;23;105;0;0;1_␛[84;20;116;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mworkbit␛[0m␛[97;2;3ms␛[0m␛[39;49m␛[0m␛[5;46H␛[?25h␛[84;20;116;0;0;1_␛[83;31;115;1;0;1_␛[?25l␛[5;35H␛[0m␛[93mssh␛[0m␛[39;49m␣␛[0m␛[37mworkbits␛[39;49m␛[0m␛[5;47H␛[?25h␛[83;31;115;0;0;1_␛[13;28;13;1;0;1_␍␊ ␛[0m␛[0m␛[13;28;13;0;0;1_␛[O␛[I␛[O

akulbe avatar Aug 23 '24 14:08 akulbe

terminal-trace.zip

akulbe avatar Aug 23 '24 14:08 akulbe

WT_SSH_Hanging

akulbe avatar Aug 23 '24 14:08 akulbe

@akulbe could you also throw in the output from ssh -vv workbits?

DHowett avatar Aug 23 '24 14:08 DHowett

FYI - this is making me crazy. Sometimes it'll happen, sometimes it won't.

OpenSSH_9.8p1, OpenSSL 3.2.2 4 Jun 2024 debug1: Reading configuration data /c/Users/Aaron Kulbe/.ssh/config debug1: Reading configuration data /c/Users/Aaron Kulbe/.ssh/ks_ssh_config debug1: Reading configuration data /c/Users/Aaron Kulbe/.ssh/milan_ssh_config debug1: /c/Users/Aaron Kulbe/.ssh/config line 12: Applying options for * debug1: /c/Users/Aaron Kulbe/.ssh/config line 255: Applying options for workbits debug1: Reading configuration data /etc/ssh/ssh_config debug2: resolve_canonicalize: hostname 172.21.252.108 is address debug1: Connecting to 172.21.252.108 [172.21.252.108] port 22. debug1: Connection established. debug1: identity file /c/Users/Aaron Kulbe/.ssh/homelab_ed25519 type 3 debug1: identity file /c/Users/Aaron Kulbe/.ssh/homelab_ed25519-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_9.8 debug1: Remote protocol version 2.0, remote software version OpenSSH_9.6 debug1: compat_banner: match: OpenSSH_9.6 pat OpenSSH* compat 0x04000000 debug2: fd 4 setting O_NONBLOCK debug1: Authenticating to 172.21.252.108:22 as 'akulbe' debug1: load_hostkeys: fopen /c/Users/Aaron Kulbe/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug2: local client KEXINIT proposal debug2: KEX algorithms: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,[email protected] debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256 debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,[email protected],zlib debug2: compression stoc: none,[email protected],zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug2: peer server KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-s,[email protected] debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 debug2: ciphers ctos: [email protected],[email protected],aes256-ctr,[email protected],aes128-ctr debug2: ciphers stoc: [email protected],[email protected],aes256-ctr,[email protected],aes128-ctr debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 debug2: compression ctos: none,[email protected] debug2: compression stoc: none,[email protected] debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ssh-ed25519 debug1: kex: server->client cipher: [email protected] MAC: compression: none debug1: kex: client->server cipher: [email protected] MAC: compression: none debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: SSH2_MSG_KEX_ECDH_REPLY received debug1: Server host key: ssh-ed25519 SHA256:giyLn2MXwq4qNDHvDRh7b3Rcktq9ZAHJUvDPNHqV2i4 debug1: load_hostkeys: fopen /c/Users/Aaron Kulbe/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: Host '172.21.252.108' is known and matches the ED25519 host key. debug1: Found key in /c/Users/Aaron Kulbe/.ssh/known_hosts:574 debug1: ssh_packet_send2_wrapped: resetting send seqnr 3 debug2: ssh_set_newkeys: mode 1 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: Sending SSH2_MSG_EXT_INFO debug1: expecting SSH2_MSG_NEWKEYS debug1: ssh_packet_read_poll2: resetting read seqnr 3 debug1: SSH2_MSG_NEWKEYS received debug2: ssh_set_newkeys: mode 0 debug1: rekey in after 134217728 blocks debug2: KEX algorithms: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,[email protected] debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256 debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,[email protected],zlib debug2: compression stoc: none,[email protected],zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: SSH2_MSG_EXT_INFO received debug1: kex_ext_info_client_parse: server-sig-algs=<ecdsa-sha2-nistp256,[email protected],ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,[email protected],rsa-sha2-256,rsa-sha2-512> debug1: kex_ext_info_check_ver: [email protected]=<0> debug1: kex_ext_info_check_ver: [email protected]=<0> debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: SSH2_MSG_EXT_INFO received debug1: kex_ext_info_client_parse: server-sig-algs=<ecdsa-sha2-nistp256,[email protected],ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,[email protected],rsa-sha2-256,rsa-sha2-512> debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Will attempt key: /c/Users/Aaron Kulbe/.ssh/homelab_ed25519 ED25519 SHA256:EGr1oCOTdNmX948nzktoeApQTJH7ynqdvz1JXmVY3is explicit debug2: pubkey_prepare: done debug1: Offering public key: /c/Users/Aaron Kulbe/.ssh/homelab_ed25519 ED25519 SHA256:EGr1oCOTdNmX948nzktoeApQTJH7ynqdvz1JXmVY3is explicit debug2: we sent a publickey packet, wait for reply debug1: Server accepts key: /c/Users/Aaron Kulbe/.ssh/homelab_ed25519 ED25519 SHA256:EGr1oCOTdNmX948nzktoeApQTJH7ynqdvz1JXmVY3is explicit Authenticated to 172.21.252.108 ([172.21.252.108]:22) using "publickey". debug1: channel 0: new session [client-session] (inactive timeout: 0) debug2: channel 0: send open debug1: Requesting [email protected] debug1: Entering interactive session. debug1: pledge: filesystem debug1: client_input_global_request: rtype [email protected] want_reply 0 debug1: client_input_hostkeys: searching /c/Users/Aaron Kulbe/.ssh/known_hosts for 172.21.252.108 / (none) debug1: client_input_hostkeys: searching /c/Users/Aaron Kulbe/.ssh/known_hosts2 for 172.21.252.108 / (none) debug1: client_input_hostkeys: hostkeys file /c/Users/Aaron Kulbe/.ssh/known_hosts2 does not exist debug1: client_input_hostkeys: host key found matching a different name/address, skipping UserKnownHostsFile update debug1: Remote: /home/akulbe/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding debug1: Remote: /home/akulbe/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding debug2: channel_input_open_confirmation: channel 0: callback start debug2: fd 4 setting TCP_NODELAY debug2: client_session2_setup: id 0 debug2: channel 0: request pty-req confirm 1 debug2: channel 0: request shell confirm 1 debug1: pledge: fork debug2: channel_input_open_confirmation: channel 0: callback done debug2: channel 0: open confirm rwindow 0 rmax 32768 debug2: channel_input_status_confirm: type 99 id 0 debug2: PTY allocation request accepted on channel 0 debug2: channel 0: rcvd adjust 2097152 debug2: channel_input_status_confirm: type 99 id 0 debug2: shell request accepted on channel 0 Web console: https://workbits.lvblab.com:9090/ or https://172.21.252.108:9090/

Last login: Thu Aug 22 09:05:56 2024 from 172.20.252.10 Welcome to fish, the friendly interactive shell Type help for instructions on how to use fish WT_SSH_Hanging_OR_NOT

akulbe avatar Aug 23 '24 14:08 akulbe

Ugh, sorry to ask this... but do you think you can get a ssh -vv output with the issue actually happening?

(I'm gonna close this out for now, but please hit me up with more details if you've got it!)

DHowett avatar Sep 04 '24 21:09 DHowett