Win32-OpenSSH GSSAPI Authentication from Windows to Linux failed

trafficstars

Troubleshooting steps https://github.com/PowerShell/Win32-OpenSSH/wiki/Troubleshooting-Steps

Terminal issue? please go through wiki https://github.com/PowerShell/Win32-OpenSSH/wiki/TTY-PTY-support-in-Windows-OpenSSH

Please answer the following

"OpenSSH for Windows" version OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2

Server OperatingSystem Ubuntu 20.04.3 LTS OpenSSH_8.2p1 Ubuntu-4ubuntu0.3, OpenSSL 1.1.1f

Client OperatingSystem Windows Server 2019 Standard OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2

What is failing The corporate environment is running on a Windows Domain Controller and we are trying to get Windows Client computer to SSH to Linux server to execute linux programs remotely. Both client and servers are domain joined. Ubuntu server runs sssd and PAM for authentication domain accounts and the following GSSAPI options are turned on:

GSSAPIAuthentication yes GSSAPIKeyExchange yes

What we are trying to achieve with GSSAPI auth is SSO through pam stack for passwordless automation and file share mount (pam_mount). Pub-Key auth works but it does not integrate with sssd so PAM modules are triggered without AD credential, therefore fail to mount the SMB file share. We are hoping that GSSAPI can provide passwordless auth while passing keytab to the server for mounting (via mount.cifs -o sec=krb5)

However, ssh seems to attempt authentication via gssapi but without success. Since it is using Microsoft SSPI, I don't have a way to dig further into the what went wrong.

On the AD side, we didn't turn on the encryption for the service account we are using to connect since it isn't on by default. On the server machine we also didn't turn on "Trust this computer for delegation" option. Would you recommend to retry with these two options turned on?

Expected output for client logs: debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 52 debug1: Authentication succeeded (gssapi-with-mic).

Actual output debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 51

Full client and server logs will be provided below

Dec 28 '21 19:12 pengsiya

Client log (windows):

OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2 debug3: Failed to open file:C:/Users/testuser/.ssh/config error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2 debug2: resolving "clientdns" port 2222 debug2: ssh_connect_direct debug1: Connecting to clientdns [10.1.27.48] port 2222. debug1: Connection established. debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_rsa type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_rsa-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_dsa type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_dsa-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ecdsa type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ecdsa-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519 error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ed25519 type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ed25519-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_xmss type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_xmss-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug1: Authenticating to clientdns:2222 as 'domain\testuser' debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent debug3: receive packet: type 20 debug1: SSH2_MSG_KEXINIT received debug2: local client KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,[email protected],zlib debug2: compression stoc: none,[email protected],zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug2: peer server KEXINIT proposal debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,[email protected] debug2: compression stoc: none,[email protected] debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: [email protected] MAC: compression: none debug1: kex: client->server cipher: [email protected] MAC: compression: none debug3: send packet: type 30 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug3: receive packet: type 31 debug1: Server host key: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug3: put_host_port: [10.1.27.48]:2222 debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: checking without port identifier debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\testuser/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from clientdns debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\testuser/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from 10.1.27.48 debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: Host 'clientdns' is known and matches the ECDSA host key. debug1: Found key in C:\Users\testuser/.ssh/known_hosts:1 debug1: found matching key w/out port debug3: send packet: type 21 debug2: set_newkeys: mode 1 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: receive packet: type 21 debug1: SSH2_MSG_NEWKEYS received debug2: set_newkeys: mode 0 debug1: rekey in after 134217728 blocks debug3: unable to connect to pipe \\.\pipe\openssh-ssh-agent, error: 2 debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory debug1: Will attempt key: C:\Users\testuser/.ssh/id_rsa debug1: Will attempt key: C:\Users\testuser/.ssh/id_dsa debug1: Will attempt key: C:\Users\testuser/.ssh/id_ecdsa debug1: Will attempt key: C:\Users\testuser/.ssh/id_ed25519 debug1: Will attempt key: C:\Users\testuser/.ssh/id_xmss debug2: pubkey_prepare: done debug3: send packet: type 5 debug3: receive packet: type 7 debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]> debug3: receive packet: type 6 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 53 debug3: input_userauth_banner Authorized uses only. All activities performed on this system will be monitored. debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: C:\Users\testuser/.ssh/id_rsa debug3: no such identity: C:\Users\testuser/.ssh/id_rsa: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_dsa debug3: no such identity: C:\Users\testuser/.ssh/id_dsa: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_ecdsa debug3: no such identity: C:\Users\testuser/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_ed25519 debug3: no such identity: C:\Users\testuser/.ssh/id_ed25519: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_xmss debug3: no such identity: C:\Users\testuser/.ssh/id_xmss: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password debug3: failed to open file:C:/dev/tty error:3 debug1: read_passphrase: can't open /dev/tty: No such file or directory domain\testuser@clientdns's password:

Dec 28 '21 19:12 pengsiya

Server Log:

debug2: load_server_config: filename /etc/ssh/sshd_config debug2: load_server_config: done config len = 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config len 626 debug2: /etc/ssh/sshd_config line 13: new include /etc/ssh/sshd_config.d/.conf debug2: /etc/ssh/sshd_config line 13: including /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: filename /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: done config len = 106 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys debug3: checking syntax for 'Match User svcQremote' debug3: /etc/ssh/sshd_config:29 setting LogLevel INFO debug3: /etc/ssh/sshd_config:34 setting LoginGraceTime 60 debug3: /etc/ssh/sshd_config:36 setting PermitRootLogin no debug3: /etc/ssh/sshd_config:39 setting MaxAuthTries 4 debug3: /etc/ssh/sshd_config:40 setting MaxSessions 10 debug3: /etc/ssh/sshd_config:54 setting HostbasedAuthentication no debug3: /etc/ssh/sshd_config:59 setting IgnoreRhosts yes debug3: /etc/ssh/sshd_config:63 setting PermitEmptyPasswords no debug3: /etc/ssh/sshd_config:67 setting ChallengeResponseAuthentication no debug3: /etc/ssh/sshd_config:76 setting GSSAPIAuthentication yes debug3: /etc/ssh/sshd_config:79 setting GSSAPIKeyExchange yes debug3: /etc/ssh/sshd_config:90 setting UsePAM yes debug3: /etc/ssh/sshd_config:96 setting X11Forwarding no debug3: /etc/ssh/sshd_config:100 setting PrintMotd no debug3: /etc/ssh/sshd_config:103 setting PermitUserEnvironment no debug3: /etc/ssh/sshd_config:106 setting ClientAliveInterval 300 debug3: /etc/ssh/sshd_config:107 setting ClientAliveCountMax 3 debug3: /etc/ssh/sshd_config:111 setting MaxStartups 10:30:60 debug3: /etc/ssh/sshd_config:118 setting Banner /etc/issue.net debug3: /etc/ssh/sshd_config:121 setting AcceptEnv LANG LC_ debug3: /etc/ssh/sshd_config:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: /etc/ssh/sshd_config:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: rexec_argv[0]='/usr/sbin/sshd' debug1: rexec_argv[1]='-ddd' debug1: rexec_argv[2]='-p' debug1: rexec_argv[3]='2222' debug3: oom_adjust_setup debug1: Set /proc/self/oom_score_adj from 0 to -1000 debug2: fd 3 setting O_NONBLOCK debug1: Bind to port 2222 on 0.0.0.0. Server listening on 0.0.0.0 port 2222. debug2: fd 4 setting O_NONBLOCK debug3: sock_set_v6only: set socket 4 IPV6_V6ONLY debug1: Bind to port 2222 on ::. Server listening on :: port 2222. debug3: fd 5 is not O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug3: send_rexec_state: entering fd = 8 config len 626 debug3: ssh_msg_send: type 0 debug3: send_rexec_state: done debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 debug3: recv_rexec_state: entering fd = 5 debug3: ssh_msg_recv entering debug3: recv_rexec_state: done debug2: parse_server_config_depth: config rexec len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys debug3: checking syntax for 'Match User svcQremote' debug3: rexec:29 setting LogLevel INFO debug3: rexec:34 setting LoginGraceTime 60 debug3: rexec:36 setting PermitRootLogin no debug3: rexec:39 setting MaxAuthTries 4 debug3: rexec:40 setting MaxSessions 10 debug3: rexec:54 setting HostbasedAuthentication no debug3: rexec:59 setting IgnoreRhosts yes debug3: rexec:63 setting PermitEmptyPasswords no debug3: rexec:67 setting ChallengeResponseAuthentication no debug3: rexec:76 setting GSSAPIAuthentication yes debug3: rexec:79 setting GSSAPIKeyExchange yes debug3: rexec:90 setting UsePAM yes debug3: rexec:96 setting X11Forwarding no debug3: rexec:100 setting PrintMotd no debug3: rexec:103 setting PermitUserEnvironment no debug3: rexec:106 setting ClientAliveInterval 300 debug3: rexec:107 setting ClientAliveCountMax 3 debug3: rexec:111 setting MaxStartups 10:30:60 debug3: rexec:118 setting Banner /etc/issue.net debug3: rexec:121 setting AcceptEnv LANG LC_* debug3: rexec:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: rexec:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: inetd sockets after dupping: 3, 3 Connection from 10.1.35.100 port 20520 on 10.1.27.48 port 2222 rdomain "" debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_8.1 debug1: match: OpenSSH_for_Windows_8.1 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug3: ssh_sandbox_init: preparing seccomp filter sandbox debug2: Network child is on pid 82176 debug3: preauth child monitor started debug3: privsep user:group 112:65534 [preauth] debug1: permanently_set_uid: 112/65534 [preauth] debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth] debug3: ssh_sandbox_child: attaching seccomp filter program [preauth] debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT sent [preauth] debug3: receive packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT received [preauth] debug2: local server KEXINIT proposal [preauth] debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 [preauth] debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,[email protected] [preauth] debug2: compression stoc: none,[email protected] [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug2: peer client KEXINIT proposal [preauth] debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c [preauth] debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa [preauth] debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,[email protected],zlib [preauth] debug2: compression stoc: none,[email protected],zlib [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug1: kex: algorithm: curve25519-sha256 [preauth] debug1: kex: host key algorithm: ecdsa-sha2-nistp256 [preauth] debug1: kex: client->server cipher: [email protected] MAC: compression: none [preauth] debug1: kex: server->client cipher: [email protected] MAC: compression: none [preauth] debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth] debug3: receive packet: type 30 [preauth] debug3: mm_sshkey_sign entering [preauth] debug3: mm_request_send entering: type 6 [preauth] debug3: mm_sshkey_sign: waiting for MONITOR_ANS_SIGN [preauth] debug3: mm_request_receive_expect entering: type 7 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 6 debug3: mm_answer_sign debug3: mm_answer_sign: KEX signature 0x5650e7b4dbe0(100) debug3: mm_request_send entering: type 7 debug2: monitor_read: 6 used once, disabling now debug3: send packet: type 31 [preauth] debug3: send packet: type 21 [preauth] debug2: set_newkeys: mode 1 [preauth] debug1: rekey out after 134217728 blocks [preauth] debug1: SSH2_MSG_NEWKEYS sent [preauth] debug1: Sending SSH2_MSG_EXT_INFO [preauth] debug3: send packet: type 7 [preauth] debug1: expecting SSH2_MSG_NEWKEYS [preauth] debug3: receive packet: type 21 [preauth] debug1: SSH2_MSG_NEWKEYS received [preauth] debug2: set_newkeys: mode 0 [preauth] debug1: rekey in after 134217728 blocks [preauth] debug1: KEX done [preauth] debug3: receive packet: type 5 [preauth] debug3: send packet: type 6 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user domain\\testuser service ssh-connection method none [preauth] debug1: attempt 0 failures 0 [preauth] debug3: mm_getpwnamallow entering [preauth] debug3: mm_request_send entering: type 8 [preauth] debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] debug3: mm_request_receive_expect entering: type 9 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 8 debug3: mm_answer_pwnamallow debug2: parse_server_config_depth: config reprocess config len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: checking match for 'User svcQremote' user domain\testuser host 10.1.35.100 addr 10.1.35.100 laddr 10.1.27.48 lport 2222 debug3: match not found debug3: mm_answer_pwnamallow: sending MONITOR_ANS_PWNAM: 1 debug3: mm_request_send entering: type 9 debug2: monitor_read: 8 used once, disabling now debug2: input_userauth_request: setting up authctxt for domain\\testuser [preauth] debug3: mm_start_pam entering [preauth] debug3: mm_request_send entering: type 100 [preauth] debug3: mm_inform_authserv entering [preauth] debug3: mm_request_send entering: type 4 [preauth] debug3: mm_auth2_read_banner entering [preauth] debug3: mm_request_send entering: type 10 [preauth] debug3: mm_request_receive_expect entering: type 11 [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 100 debug1: PAM: initializing for "domain\testuser" debug1: PAM: setting PAM_RHOST to "10.1.35.100" debug1: PAM: setting PAM_TTY to "ssh" debug2: monitor_read: 100 used once, disabling now debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 4 debug3: mm_answer_authserv: service=ssh-connection, style=, role= debug2: monitor_read: 4 used once, disabling now debug3: mm_request_receive entering debug3: monitor_read: checking request 10 debug3: mm_request_send entering: type 11 debug2: monitor_read: 10 used once, disabling now debug3: send packet: type 53 [preauth] debug1: userauth_send_banner: sent [preauth] debug2: input_userauth_request: try method none [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 27.545ms, delaying 23.314ms (requested 6.357ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user domain\\testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 1 failures 0 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 60 [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 6.443ms, delaying 6.272ms (requested 6.357ms) [preauth] Postponed gssapi-with-mic for domain\\testuser from 10.1.35.100 port 20520 ssh2 [preauth] debug3: receive packet: type 61 [preauth] debug3: mm_request_send entering: type 44 [preauth] debug3: mm_request_receive_expect entering: type 45 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 44 debug1: Got no client credentials debug3: mm_request_send entering: type 45 debug3: send packet: type 61 [preauth] debug3: receive packet: type 66 [preauth] debug3: mm_request_send entering: type 48 [preauth] debug3: mm_request_receive_expect entering: type 49 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 48 debug3: mm_request_send entering: type 49 debug2: monitor_read: 48 used once, disabling now debug3: mm_request_send entering: type 46 [preauth] debug3: mm_request_receive_expect entering: type 47 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 46 debug3: mm_answer_gss_userok: sending result 0 debug3: mm_request_send entering: type 47 debug2: monitor_read: 46 used once, disabling now Failed gssapi-with-mic for domain\testuser from 10.1.35.100 port 20520 ssh2 debug3: mm_ssh_gssapi_userok: user not authenticated [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] Connection reset by authenticating user domain\\testuser 10.1.35.100 port 20520 [preauth] debug1: do_cleanup [preauth] debug3: PAM: sshpam_thread_cleanup entering [preauth] debug1: monitor_read_log: child log fd closed debug3: mm_request_receive entering debug1: do_cleanup debug1: PAM: cleanup debug3: PAM: sshpam_thread_cleanup entering debug1: Killing privsep child 82176 debug1: audit_event: unhandled event 12

Dec 28 '21 19:12 pengsiya

SSHD CONFIG

Include /etc/ssh/sshd_config.d/*.conf LogLevel INFO

LoginGraceTime 60 PermitRootLogin no MaxAuthTries 4 MaxSessions 10

HostbasedAuthentication no IgnoreRhosts yes

PermitEmptyPasswords no

ChallengeResponseAuthentication no

GSSAPIAuthentication yes GSSAPIKeyExchange yes

UsePAM yes

X11Forwarding no PrintMotd no PermitUserEnvironment no ClientAliveInterval 300 ClientAliveCountMax 3 MaxStartups 10:30:60

Banner /etc/issue.net

AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

PasswordAuthentication yes

Dec 28 '21 19:12 pengsiya

debug1: Authenticating to clientdns:2222 as 'domain\testuser'

I suspect Linux does not like the domain\ prefix that Windows prepends to your username. As a workaround, specify the username explicitly when SSHing from Windows to Linux.

Once authentication works: to make sec=krb5 work, you probably also need to enable ticket delegation.

Dec 31 '21 15:12 mgkuhn

Tried with just the username and still failed:

ClientLog:

OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2 debug3: Failed to open file:C:/Users/admin/.ssh/config error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2 debug2: resolving "clientdns" port 2222 debug2: ssh_connect_direct debug1: Connecting to clientdns [10.1.27.48] port 2222. debug1: Connection established. debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_rsa type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_rsa-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_dsa type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_dsa-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ecdsa type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ecdsa-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519 error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ed25519 type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ed25519-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_xmss type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_xmss-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug1: Authenticating to clientdns:2222 as 'testuser' debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent debug3: receive packet: type 20 debug1: SSH2_MSG_KEXINIT received debug2: local client KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,[email protected],zlib debug2: compression stoc: none,[email protected],zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug2: peer server KEXINIT proposal debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,[email protected] debug2: compression stoc: none,[email protected] debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: [email protected] MAC: compression: none debug1: kex: client->server cipher: [email protected] MAC: compression: none debug3: send packet: type 30 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug3: receive packet: type 31 debug1: Server host key: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug3: put_host_port: [10.1.27.48]:2222 debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: checking without port identifier debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\admin/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from clientdns debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\admin/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from 10.1.27.48 debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: Host 'clientdns' is known and matches the ECDSA host key. debug1: Found key in C:\Users\admin/.ssh/known_hosts:1 debug1: found matching key w/out port debug3: send packet: type 21 debug2: set_newkeys: mode 1 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: receive packet: type 21 debug1: SSH2_MSG_NEWKEYS received debug2: set_newkeys: mode 0 debug1: rekey in after 134217728 blocks debug3: unable to connect to pipe \\.\pipe\openssh-ssh-agent, error: 2 debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory debug1: Will attempt key: C:\Users\admin/.ssh/id_rsa debug1: Will attempt key: C:\Users\admin/.ssh/id_dsa debug1: Will attempt key: C:\Users\admin/.ssh/id_ecdsa debug1: Will attempt key: C:\Users\admin/.ssh/id_ed25519 debug1: Will attempt key: C:\Users\admin/.ssh/id_xmss debug2: pubkey_prepare: done debug3: send packet: type 5 debug3: receive packet: type 7 debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]> debug3: receive packet: type 6 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 53 debug3: input_userauth_banner Authorized uses only. All activities performed on this system will be monitored. debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: C:\Users\admin/.ssh/id_rsa debug3: no such identity: C:\Users\admin/.ssh/id_rsa: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_dsa debug3: no such identity: C:\Users\admin/.ssh/id_dsa: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_ecdsa debug3: no such identity: C:\Users\admin/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_ed25519 debug3: no such identity: C:\Users\admin/.ssh/id_ed25519: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_xmss debug3: no such identity: C:\Users\admin/.ssh/id_xmss: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password debug3: failed to open file:C:/dev/tty error:3 debug1: read_passphrase: can't open /dev/tty: No such file or directory

ServerLog:

debug2: load_server_config: filename /etc/ssh/sshd_config debug2: load_server_config: done config len = 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config len 626 debug2: /etc/ssh/sshd_config line 13: new include /etc/ssh/sshd_config.d/.conf debug2: /etc/ssh/sshd_config line 13: including /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: filename /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: done config len = 106 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys debug3: checking syntax for 'Match User svcQremote' debug3: /etc/ssh/sshd_config:29 setting LogLevel INFO debug3: /etc/ssh/sshd_config:34 setting LoginGraceTime 60 debug3: /etc/ssh/sshd_config:36 setting PermitRootLogin no debug3: /etc/ssh/sshd_config:39 setting MaxAuthTries 4 debug3: /etc/ssh/sshd_config:40 setting MaxSessions 10 debug3: /etc/ssh/sshd_config:54 setting HostbasedAuthentication no debug3: /etc/ssh/sshd_config:59 setting IgnoreRhosts yes debug3: /etc/ssh/sshd_config:63 setting PermitEmptyPasswords no debug3: /etc/ssh/sshd_config:67 setting ChallengeResponseAuthentication no debug3: /etc/ssh/sshd_config:76 setting GSSAPIAuthentication yes debug3: /etc/ssh/sshd_config:79 setting GSSAPIKeyExchange yes debug3: /etc/ssh/sshd_config:90 setting UsePAM yes debug3: /etc/ssh/sshd_config:96 setting X11Forwarding no debug3: /etc/ssh/sshd_config:100 setting PrintMotd no debug3: /etc/ssh/sshd_config:103 setting PermitUserEnvironment no debug3: /etc/ssh/sshd_config:106 setting ClientAliveInterval 300 debug3: /etc/ssh/sshd_config:107 setting ClientAliveCountMax 3 debug3: /etc/ssh/sshd_config:111 setting MaxStartups 10:30:60 debug3: /etc/ssh/sshd_config:118 setting Banner /etc/issue.net debug3: /etc/ssh/sshd_config:121 setting AcceptEnv LANG LC_ debug3: /etc/ssh/sshd_config:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: /etc/ssh/sshd_config:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: rexec_argv[0]='/usr/sbin/sshd' debug1: rexec_argv[1]='-p' debug1: rexec_argv[2]='2222' debug1: rexec_argv[3]='-ddd' debug3: oom_adjust_setup debug1: Set /proc/self/oom_score_adj from 0 to -1000 debug2: fd 3 setting O_NONBLOCK debug1: Bind to port 2222 on 0.0.0.0. Server listening on 0.0.0.0 port 2222. debug2: fd 4 setting O_NONBLOCK debug3: sock_set_v6only: set socket 4 IPV6_V6ONLY debug1: Bind to port 2222 on ::. Server listening on :: port 2222. debug3: fd 5 is not O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug3: send_rexec_state: entering fd = 8 config len 626 debug3: ssh_msg_send: type 0 debug3: send_rexec_state: done debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 debug3: recv_rexec_state: entering fd = 5 debug3: ssh_msg_recv entering debug3: recv_rexec_state: done debug2: parse_server_config_depth: config rexec len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys debug3: checking syntax for 'Match User svcQremote' debug3: rexec:29 setting LogLevel INFO debug3: rexec:34 setting LoginGraceTime 60 debug3: rexec:36 setting PermitRootLogin no debug3: rexec:39 setting MaxAuthTries 4 debug3: rexec:40 setting MaxSessions 10 debug3: rexec:54 setting HostbasedAuthentication no debug3: rexec:59 setting IgnoreRhosts yes debug3: rexec:63 setting PermitEmptyPasswords no debug3: rexec:67 setting ChallengeResponseAuthentication no debug3: rexec:76 setting GSSAPIAuthentication yes debug3: rexec:79 setting GSSAPIKeyExchange yes debug3: rexec:90 setting UsePAM yes debug3: rexec:96 setting X11Forwarding no debug3: rexec:100 setting PrintMotd no debug3: rexec:103 setting PermitUserEnvironment no debug3: rexec:106 setting ClientAliveInterval 300 debug3: rexec:107 setting ClientAliveCountMax 3 debug3: rexec:111 setting MaxStartups 10:30:60 debug3: rexec:118 setting Banner /etc/issue.net debug3: rexec:121 setting AcceptEnv LANG LC_* debug3: rexec:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: rexec:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: inetd sockets after dupping: 3, 3 Connection from 10.1.35.100 port 20485 on 10.1.27.48 port 2222 rdomain "" debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_8.1 debug1: match: OpenSSH_for_Windows_8.1 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug3: ssh_sandbox_init: preparing seccomp filter sandbox debug2: Network child is on pid 398042 debug3: preauth child monitor started debug3: privsep user:group 112:65534 [preauth] debug1: permanently_set_uid: 112/65534 [preauth] debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth] debug3: ssh_sandbox_child: attaching seccomp filter program [preauth] debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT sent [preauth] debug3: receive packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT received [preauth] debug2: local server KEXINIT proposal [preauth] debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 [preauth] debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,[email protected] [preauth] debug2: compression stoc: none,[email protected] [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug2: peer client KEXINIT proposal [preauth] debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c [preauth] debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa [preauth] debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth] debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,[email protected],zlib [preauth] debug2: compression stoc: none,[email protected],zlib [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug1: kex: algorithm: curve25519-sha256 [preauth] debug1: kex: host key algorithm: ecdsa-sha2-nistp256 [preauth] debug1: kex: client->server cipher: [email protected] MAC: compression: none [preauth] debug1: kex: server->client cipher: [email protected] MAC: compression: none [preauth] debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth] debug3: receive packet: type 30 [preauth] debug3: mm_sshkey_sign entering [preauth] debug3: mm_request_send entering: type 6 [preauth] debug3: mm_sshkey_sign: waiting for MONITOR_ANS_SIGN [preauth] debug3: mm_request_receive_expect entering: type 7 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 6 debug3: mm_answer_sign debug3: mm_answer_sign: KEX signature 0x55aa6c099be0(99) debug3: mm_request_send entering: type 7 debug2: monitor_read: 6 used once, disabling now debug3: send packet: type 31 [preauth] debug3: send packet: type 21 [preauth] debug2: set_newkeys: mode 1 [preauth] debug1: rekey out after 134217728 blocks [preauth] debug1: SSH2_MSG_NEWKEYS sent [preauth] debug1: Sending SSH2_MSG_EXT_INFO [preauth] debug3: send packet: type 7 [preauth] debug1: expecting SSH2_MSG_NEWKEYS [preauth] debug3: receive packet: type 21 [preauth] debug1: SSH2_MSG_NEWKEYS received [preauth] debug2: set_newkeys: mode 0 [preauth] debug1: rekey in after 134217728 blocks [preauth] debug1: KEX done [preauth] debug3: receive packet: type 5 [preauth] debug3: send packet: type 6 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method none [preauth] debug1: attempt 0 failures 0 [preauth] debug3: mm_getpwnamallow entering [preauth] debug3: mm_request_send entering: type 8 [preauth] debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] debug3: mm_request_receive_expect entering: type 9 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 8 debug3: mm_answer_pwnamallow debug2: parse_server_config_depth: config reprocess config len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: checking match for 'User svcQremote' user testuser host 10.1.35.100 addr 10.1.35.100 laddr 10.1.27.48 lport 2222 debug3: match not found debug3: mm_answer_pwnamallow: sending MONITOR_ANS_PWNAM: 1 debug3: mm_request_send entering: type 9 debug2: monitor_read: 8 used once, disabling now debug2: input_userauth_request: setting up authctxt for testuser [preauth] debug3: mm_start_pam entering [preauth] debug3: mm_request_send entering: type 100 [preauth] debug3: mm_inform_authserv entering [preauth] debug3: mm_request_send entering: type 4 [preauth] debug3: mm_auth2_read_banner entering [preauth] debug3: mm_request_send entering: type 10 [preauth] debug3: mm_request_receive_expect entering: type 11 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 100 debug1: PAM: initializing for "testuser" debug1: PAM: setting PAM_RHOST to "10.1.35.100" debug1: PAM: setting PAM_TTY to "ssh" debug2: monitor_read: 100 used once, disabling now debug3: mm_request_receive entering debug3: monitor_read: checking request 4 debug3: mm_answer_authserv: service=ssh-connection, style=, role= debug2: monitor_read: 4 used once, disabling now debug3: mm_request_receive entering debug3: monitor_read: checking request 10 debug3: mm_request_send entering: type 11 debug2: monitor_read: 10 used once, disabling now debug3: send packet: type 53 [preauth] debug1: userauth_send_banner: sent [preauth] debug2: input_userauth_request: try method none [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 31.534ms, delaying 10.395ms (requested 5.241ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 1 failures 0 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 60 [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 4.674ms, delaying 0.567ms (requested 5.241ms) [preauth] Postponed gssapi-with-mic for testuser from 10.1.35.100 port 20485 ssh2 [preauth] debug3: receive packet: type 61 [preauth] debug3: mm_request_send entering: type 44 [preauth] debug3: mm_request_receive_expect entering: type 45 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 44 debug1: Got no client credentials debug3: mm_request_send entering: type 45 debug3: send packet: type 61 [preauth] debug3: receive packet: type 66 [preauth] debug3: mm_request_send entering: type 48 [preauth] debug3: mm_request_receive_expect entering: type 49 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 48 debug3: mm_request_send entering: type 49 debug2: monitor_read: 48 used once, disabling now debug3: mm_request_send entering: type 46 [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 46 debug3: mm_answer_gss_userok: sending result 0 debug3: mm_request_send entering: type 47 debug2: monitor_read: 46 used once, disabling now Failed gssapi-with-mic for testuser from 10.1.35.100 port 20485 ssh2 debug3: mm_request_receive_expect entering: type 47 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_ssh_gssapi_userok: user not authenticated [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] Connection reset by authenticating user testuser 10.1.35.100 port 20485 [preauth] debug1: do_cleanup [preauth] debug3: PAM: sshpam_thread_cleanup entering [preauth] debug1: monitor_read_log: child log fd closed debug3: mm_request_receive entering debug1: do_cleanup debug1: PAM: cleanup debug3: PAM: sshpam_thread_cleanup entering debug1: Killing privsep child 398042 debug1: audit_event: unhandled event 12

Dec 31 '21 18:12 pengsiya

Seems to fail at the preauthentication. Is it a Microsoft SSPI thing? If so, where can I get support from for this?

Dec 31 '21 18:12 pengsiya

@pengsiya

Can you try again with just GSSAPIAuthentication = Yes? Also does the hostname you're connecting to match the host name in Active Directory (or, if not, does an SPN exist on the AD object for any IP or hostname alias)? Do you see the Kerberos token cached when you do a 'klist' on the client system? Wireshark can be good for snooping Kerberos request issues.

As an additional data point, you could try PuTTY with GSSAPI enabled just to eliminate any OpenSSH client GSSAPI nuances.

Jan 03 '22 19:01 NoMoreFood

@NoMoreFood

I tried with just GSSAPIAuthentication = Yes and got the same error.

The hostname should match with the host name in AD. I am able to use password authentication against AD. I tried with FQDN as well and no luck.

klist shows the following (omitted most contents) #2> Client: testuser @ DOMAIN Server: host/svrdns.DOMAIN @ DOMAIN KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96 Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize Start Time: 1/4/2022 23:17:09 (local) End Time: 1/5/2022 9:11:07 (local) Renew Time: 1/11/2022 23:11:07 (local) Session Key Type: AES-256-CTS-HMAC-SHA1-96 Cache Flags: 0 Kdc Called: dc.DOMAIN

With Putty, I got a similar error: Event Log: GSSAPI authentication initialised Event Log: GSSAPI authentication loop finished OK Outgoing packet #0x7, type 66 / 0x42 (SSH2_MSG_USERAUTH_GSSAPI_MIC) 00000000 00 00 00 1c 04 04 04 ff ff ff ff ff 00 00 00 00 ................ 00000010 6a a0 43 a7 d8 0b 42 9f fc 4a 81 9f 96 99 1a 61 j.C...B..J.....a Incoming packet #0x8, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE)

I will give it a try with wireshark.

Thanks!

Jan 05 '22 07:01 pengsiya

Just for reference, when I tried to connect a linux client to the linux server via GSSAPI, it also failed, even though both machines have active login session open and live kerberos tickets. So as I mentioned before this may not be an Win32-OpenSSH specific issue, but a general GSSAPI issue (or SSPI issue).

I really just want to know where I can get support for this issue. Any reply would be appreciated!!

Jan 05 '22 07:01 pengsiya

here is a trimmed version of the ssh logs for Linux -> Linux GSSAPI auth attempt:

Client:

debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 53 debug3: input_userauth_banner Authorized uses only. All activities performed on this system will be monitored. debug3: receive packet: type 51 debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic debug3: start over, passed a different list gssapi-keyex,gssapi-with-mic debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug3: send packet: type 66 debug3: receive packet: type 51 debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 51 debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic debug2: we did not send a packet, disable method debug1: No more authentication methods to try. testuser@svrdns: Permission denied (gssapi-keyex,gssapi-with-mic).

Server:

debug3: send packet: type 53 [preauth] debug1: userauth_send_banner: sent [preauth] debug2: input_userauth_request: try method none [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 31.513ms, delaying 20.344ms (requested 6.482ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="gssapi-keyex,gssapi-with-mic" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 1 failures 0 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 60 [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 5.676ms, delaying 0.807ms (requested 6.482ms) [preauth] Postponed gssapi-with-mic for testuser from 10.1.27.104 port 63924 ssh2 [preauth] debug3: receive packet: type 61 [preauth] debug3: mm_request_send entering: type 44 [preauth] debug3: mm_request_receive_expect entering: type 45 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 44 debug1: Got no client credentials debug3: mm_request_send entering: type 45 debug3: send packet: type 61 [preauth] debug3: receive packet: type 66 [preauth] debug3: mm_request_send entering: type 48 [preauth] debug3: mm_request_receive_expect entering: type 49 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 48 debug3: mm_request_send entering: type 49 debug2: monitor_read: 48 used once, disabling now debug3: mm_request_send entering: type 46 [preauth] debug3: mm_request_receive_expect entering: type 47 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 46 debug3: mm_answer_gss_userok: sending result 0 debug3: mm_request_send entering: type 47 debug2: monitor_read: 46 used once, disabling now Failed gssapi-with-mic for testuser from 10.1.27.104 port 63924 ssh2 debug3: mm_ssh_gssapi_userok: user not authenticated [preauth] debug3: userauth_finish: failure partial=0 next methods="gssapi-keyex,gssapi-with-mic" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 2 failures 1 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 0.064ms, delaying 6.419ms (requested 6.482ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="gssapi-keyex,gssapi-with-mic" [preauth] debug3: send packet: type 51 [preauth] Connection closed by authenticating user testuser 10.1.27.104 port 63924 [preauth]

Jan 05 '22 07:01 pengsiya

The only thing I found in my AD record for the linux machine is that the "Computer name" is full capital letters (no FQDN) but the "DNS name" is all lower cases (with FQDN).

Could this be an issue?

Jan 05 '22 08:01 pengsiya

On the Linux server, what does sudo klist -kte show? It must list a host/hostname.fqdn@DOMAIN service principal name (SPN) for the server, stored in /etc/krb5.keytab. You must have exported from the Active Directory server using ktpass a keytab file for the server and stored it on the Linux server in /etc/krb5.keytab (only readable for root!), for /sbin/sshd to use for its end of the authentication.

On a Linux client, what does KRB5_TRACE=/dev/stdout ssh hostname.fqdn output?

What does klist show afterwards?

That same SPN (possibly without the @DOMAIN suffix) must also be found in an servicePrincipalName attribute in the Active Directory LDAP object representing the Linux server (can be a normal user entry, no need for it to actually be a computer entry), and the hostname.fqdn should also match what a rDNS lookup of the IP address of the SSH server resolves to. Because SSPI will search for the SPN (and Kerberos key) of the SSH server via its hostname in servicePrincipalName.

Jan 05 '22 12:01 mgkuhn

I should add that I have never used, and am not familiar with, sssd. At my department, we just manually create a user object in Active Directory for each Linux host, set the servicePrincipalName of that user object to host/hostname.fqdn, generate a key, export it into a keytab file with ktpass and store it on the Linux server in /etc/krb5.keytab. That's normally all that's needed to cause sshd on a Linux server to use be able to use Active Directory as a Kerberos5 KDC for GSSAPIAuthentication (assuming that the Linux server has already its own getent passwd and getent group user/group databases, and you don't need Active Directory for that as well).

I understand that there are other ways to “join” a Linux server to an Active Directory domain. They all need to somehow generate a Kerberos password for the server, and locally store the corresponding keytab file (= password-derived symmetric key). Some such techniques (e.g. e.g. Samba's net join) store that keytab elsewhere than /etc/krb5.keytab, and thus sshd might not find it (unless told about the new location via /etc/krb5.conf).

So make sure you understand where sssd keeps the SPN keytab that sshd is meant to use.

Jan 05 '22 12:01 mgkuhn

On the Linux server, what does sudo klist -kte show? It must list a host/hostname.fqdn@DOMAIN service principal name (SPN) for the server, stored in /etc/krb5.keytab.

Here is the output of klist -kte:

Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
  12 12/17/2021 14:24:49 SVRDNS$@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 SVRDNS$@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 SVRDNS$@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 host/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 host/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 RestrictedKrbHost/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 RestrictedKrbHost/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 SVRDNS$@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 SVRDNS$@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 SVRDNS$@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 host/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 host/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 RestrictedKrbHost/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 RestrictedKrbHost/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)

You must have exported from the Active Directory server using ktpass a keytab file for the server and stored it on the Linux server in /etc/krb5.keytab (only readable for root!), for /sbin/sshd to use for its end of the authentication.

The keytab was not exported from Active Directory. They were automatically generated by sssd I think when the server is joined to domain (via realm join/adcli)

On a Linux client, what does KRB5_TRACE=/dev/stdout ssh hostname.fqdn output?

[3324776] 1641422550.803248: ccselect module realm chose cache FILE:/tmp/krb5cc_892039766_jhfnGl with client principal testuser@DOMAIN for server principal host/svcdns.domain@DOMAIN [3324776] 1641422550.803249: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422550.803250: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_892039766_jhfnGl) [3324776] 1641422550.803251: Retrying testuser@DOMAIN -> host/svcdns.domain@DOMAIN with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_892039766_jhfnGl) [3324776] 1641422550.803252: Server has referral realm; starting with host/svcdns.domain@DOMAIN [3324776] 1641422550.803253: Retrieving testuser@DOMAIN -> krbtgt/DOMAIN@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422550.803254: Starting with TGT for client realm: testuser@DOMAIN -> krbtgt/DOMAIN@DOMAIN [3324776] 1641422550.803255: Requesting tickets for host/svcdns.domain@DOMAIN, referrals on [3324776] 1641422550.803256: Generated subkey for TGS request: aes256-cts/4973 [3324776] 1641422550.803257: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts [3324776] 1641422550.803259: Encoding request body and padata into FAST request [3324776] 1641422550.803260: Sending request (3021 bytes) to DOMAIN [3324776] 1641422550.803261: Initiating TCP connection to stream 10.1.0.51:88 [3324776] 1641422550.803262: Sending TCP request to stream 10.1.0.51:88 [3324776] 1641422550.803263: Received answer (2952 bytes) from stream 10.1.0.51:88 [3324776] 1641422550.803264: Terminating TCP connection to stream 10.1.0.51:88 [3324776] 1641422550.803265: Response was from master KDC [3324776] 1641422550.803266: Decoding FAST response [3324776] 1641422550.803267: FAST reply key: aes256-cts/577D [3324776] 1641422550.803268: TGS reply is for testuser@DOMAIN -> host/svcdns.domain@DOMAIN with session key aes256-cts/3ADD [3324776] 1641422550.803269: TGS request result: 0/Success [3324776] 1641422550.803270: Received creds for desired service host/svcdns.domain@DOMAIN [3324776] 1641422550.803271: Storing testuser@DOMAIN -> host/svcdns.domain@ in FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422550.803272: Also storing testuser@DOMAIN -> host/svcdns.domain@DOMAIN based on ticket [3324776] 1641422550.803273: Removing testuser@DOMAIN -> host/svcdns.domain@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422550.803275: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 872327293, subkey aes256-cts/D403, session key aes256-cts/3ADD [3324776] 1641422551.8767: ccselect module realm chose cache FILE:/tmp/krb5cc_892039766_jhfnGl with client principal testuser@DOMAIN for server principal host/svcdns.domain@DOMAIN [3324776] 1641422551.8768: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422551.8769: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422551.8771: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 279037212, subkey aes256-cts/8835, session key aes256-cts/3ADD [3324776] 1641422551.8776: Read AP-REP, time 1641422551.8772, subkey aes256-cts/616E, seqnum 130746151 [3324776] 1641422551.8781: ccselect module realm chose cache FILE:/tmp/krb5cc_892039766_jhfnGl with client principal testuser@DOMAIN for server principal host/svcdns.domain@DOMAIN [3324776] 1641422551.8782: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422551.8783: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422551.8785: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422551.8786: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422551.8788: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 597380254, subkey aes256-cts/C26B, session key aes256-cts/3ADD [email protected]: Permission denied (gssapi-keyex,gssapi-with-mic).

see highlighted line:

[3324776] 1641422550.803273: Removing testuser@DOMAIN -> host/svcdns.domain@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl

why is that credential removed?

What does klist show afterwards?

identical as before

That same SPN (possibly without the @DOMAIN suffix) must also be found in an servicePrincipalName attribute in the Active Directory LDAP object representing the Linux server (can be a normal user entry, no need for it to actually be a computer entry), and the hostname.fqdn should also match what a rDNS lookup of the IP address of the SSH server resolves to. Because SSPI will search for the SPN (and Kerberos key) of the SSH server via its hostname in servicePrincipalName.

The linux server is in AD as a computer entry. When I query for the SPN for the server:

setspn -l svcdns Registered ServicePrincipalNames for CN=svcdns,OU=Linux,OU=,OU=Servers,DC=,DC=***: RestrictedKrbHost/svrdns.domain RestrictedKrbHost/SVRDNS host/svrdns.domain host/SVRDNS

Jan 05 '22 22:01 pengsiya

I should add that I have never used, and am not familiar with, sssd. At my department, we just manually create a user object in Active Directory for each Linux host, set the servicePrincipalName of that user object to host/hostname.fqdn, generate a key, export it into a keytab file with ktpass and store it on the Linux server in /etc/krb5.keytab. That's normally all that's needed to cause sshd on a Linux server to use be able to use Active Directory as a Kerberos5 KDC for GSSAPIAuthentication (assuming that the Linux server has already its own getent passwd and getent group user/group databases, and you don't need Active Directory for that as well).

I understand that there are other ways to “join” a Linux server to an Active Directory domain. They all need to somehow generate a Kerberos password for the server, and locally store the corresponding keytab file (= password-derived symmetric key). Some such techniques (e.g. e.g. Samba's net join) store that keytab elsewhere than /etc/krb5.keytab, and thus sshd might not find it (unless told about the new location via /etc/krb5.conf).

So make sure you understand where sssd keeps the SPN keytab that sshd is meant to use.

Thanks @mgkuhn for the followup, much appreciated!

I personally don't have a lot of experience using sssd and in fact our company was just trying to move onto a hybrid platform environment (in the past pure Windows).

One thing I can tell for sure is that on the linux machine (both client and server), I can use the kerberos ticket to do smbclient connection:

(base) testuser@cltdns:~$ smbclient -k //host.fqdn/share
Try "help" to get a list of possible commands.
smb: \>

So the keytab on the server should be somewhat working.

servicePrincipalName isn't something that we control and when I join the machine to AD I didn't have to specify it.

Jan 05 '22 23:01 pengsiya

Also I found using the utility "setspn -l" that our machine account generally has SPN but none of the user account has SPN by default.

Jan 07 '22 18:01 pengsiya

On any Linux machine (that can contact port 88 of the domain controller), can you get as a normal user with

$ kvno host/svrdns.domain
$ klist

a ticket for principal host/svrdns.domain?

If not, does

$ kvno host/svrdns.domain@DOMAIN
$ klist

work?

The kvno command command allows you to test getting a ticket, without actually using it. I think on Windows the klist get option, as in

C:\> klist get host/svrdns.domain
C:\> klist

might do the same.

Jan 09 '22 15:01 mgkuhn

see highlighted line:

[3324776] 1641422550.803273: Removing testuser@DOMAIN -> host/svcdns.domain@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl

why is that credential removed?

I don't understand it, but I can see it as well in a normally working session, so I think that's normal and not an indication of any problem.

I think the line

[3324776] 1641422551.8788: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 597380254, subkey aes256-cts/C26B, session key aes256-cts/3ADD

means everything reported by the KRB5_TRACE output looks fine and a ticket was successfully acquired, so you should see a host/svcdns.domain@DOMAIN ticket in klist.

I'm puzzled by mention of gssapi-keyex in

[email protected]: Permission denied (gssapi-keyex,gssapi-with-mic)

because OpenSSH for Windows 8.1/8.6 don't yet support gssapi-keyex (#15), it only does gssapi-with-mic so far.

Jan 09 '22 15:01 mgkuhn

Does sshd say in

$ journalctl _SYSTEMD_UNIT=ssh.service

why it denied permission? Perhaps the PAM stack doesn't like the user name provided?

Jan 09 '22 15:01 mgkuhn

What does klist show afterwards?

identical as before

Just to clarify: I had meant just klist to check what ticket the user got as a result of attempting ssh -K, not klist -kt to check the keytab file of the server.

Jan 09 '22 15:01 mgkuhn

The only thing I found in my AD record for the linux machine is that the "Computer name" is full capital letters (no FQDN) but the "DNS name" is all lower cases (with FQDN). Could this be an issue?

I don't think so. SSPI doesn't need any of these entries for SSH. The Linux server doesn't really need to be fully “domain joined” in a Windows sense for ssh -K, all it needs is an SPN and a keytab. As I said, we just use normal user entries for our Linux servers and that works as well, e.g.

$ ldapsearch -Y GSSAPI -Q -LLL -H ldap://adsrv.dc.cl.cam.ac.uk -b DC=dc,DC=cl,DC=cam,DC=ac,DC=uk cn=hostDirac
dn: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: hostDirac
description: host/dirac.cl.cam.ac.uk
distinguishedName: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
instanceType: 4
whenCreated: 20160923125142.0Z
whenChanged: 20200723083126.0Z
displayName: hostDirac
uSNCreated: 67775
uSNChanged: 18297641
name: hostDirac
objectGUID:: wChZHsBwPUqBUH8iR6q4kQ== # c0 28 59 1e c0 70 3d 4a 81 50 7f 22 47 aa b8 91
userAccountControl: 544 # PASSWD_NOTREQD(32) | NORMAL_ACCOUNT(512)
codePage: 0
countryCode: 0
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAAVHhoalqY1Is9OO6uFg8AAA== # S-1-5-5-21-1785231444-2345965658-2934847549-3862
sAMAccountName: hostDirac
sAMAccountType: 805306368
userPrincipalName: host/[email protected]
servicePrincipalName: host/dirac.cl.cam.ac.uk
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
dSCorePropagationData: 20170208114138.0Z
dSCorePropagationData: 16010101000001.0Z
msDS-SupportedEncryptionTypes: 24 # AES128-CTS-HMAC-SHA1-96(8) | AES256-CTS-HMAC-SHA1-96(16)

I think only the servicePrincipalName and msDS-SupportedEncryptionTypes attributes really matter here for Kerberos use by SSH: one to find the entry (and associated password/keytab), and the other to control which encryption types are allowed (make sure the AES ones are enabled).

Can you use ldapsearch as above to query your domain controller via GSSAPI authentication? You will need

sudo apt-get install krb5-user ldap-utils libsasl2-modules-gssapi-mit

Jan 09 '22 15:01 mgkuhn

On any Linux machine (that can contact port 88 of the domain controller), can you get as a normal user with
$ kvno host/svrdns.domain
$ klist
a ticket for principal host/svrdns.domain?

If not, does
$ kvno host/svrdns.domain@DOMAIN
$ klist
work?

The kvno command command allows you to test getting a ticket, without actually using it. I think on Windows the klist get option, as in
C:\> klist get host/svrdns.domain
C:\> klist
might do the same.

Yeah kvno works and klist get also works. I am able to see a ticket in both linux and windows machines.

Linux:

$ klist Ticket cache: FILE:/tmp/krb5cc_892039766_dnXXV1 Default principal: testuser@DOMAIN

Valid starting Expires Service principal 01/10/22 07:25:41 01/10/22 17:25:41 krbtgt/DOMAIN@DOMAIN renew until 01/17/22 07:25:41 01/10/22 07:29:31 01/10/22 17:25:41 host/svrdns.domain@DOMAIN renew until 01/17/22 07:25:41

Windows:

#2> Client: testuser @ DOMAIN Server: host/svrdns.domain @ DOMAIN KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96 Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize Start Time: 1/10/2022 7:36:18 (local) End Time: 1/10/2022 17:36:18 (local) Renew Time: 1/16/2022 22:46:07 (local) Session Key Type: AES-256-CTS-HMAC-SHA1-96 Cache Flags: 0 Kdc Called: ****

Jan 10 '22 15:01 pengsiya

Does sshd say in
$ journalctl _SYSTEMD_UNIT=ssh.service
why it denied permission? Perhaps the PAM stack doesn't like the user name provided?

Somehow I don't see any log in the journalctl about the login attempt...

Jan 10 '22 15:01 pengsiya

What does klist show afterwards?

identical as before

Just to clarify: I had meant just klist to check what ticket the user got as a result of attempting ssh -K, not klist -kt to check the keytab file of the server.

I see. But I think the service ticket for the host seems to show up without any problem. Just couldn't authenticate further.

Jan 10 '22 15:01 pengsiya

On the AD side, we didn't turn on the encryption for the service account we are using to connect since it isn't on by default.

What does “turn on encryption” mean exactly? (What exact GUI control or LDAP attribute are you talking about?)

On the server machine we also didn't turn on "Trust this computer for delegation" option. Would you recommend to retry with these two options turned on?

If you want to delegate a ticket, for example because you want from your SSH Linux server to be able to use other Kerberos-authenticated services (e.g., an SMB or NFS file server), then you will need to enable delegation. However, whether delegation is enabled or not for that server should not affect the success of authentication. It should only affect if you have a forwarded ticket on the Linux server after you successfully authenticated.

I still suspect that gssapi-with-mic authentication actually works for you and permission is denied after successful Kerberos authentication, because something else didn't like the outcome of Kerberos authentication (e.g. the user name).

Jan 10 '22 16:01 mgkuhn

Can you run sshd on the Linux server in foreground debug mode, as in

# KRB5_TRACE=/tmp/ssh-krb5trace /sbin/sshd -D -d -p 2222

and then connect to port 2222 with ssh -K -p 2222 ...?

If I do this, I see gssapi-with-mic success reported by sshd with lines such as

Authorized to mgkuhn, krb5 principal mgkuhn@DOMAIN (krb5_kuserok)
[...]
Accepted gssapi-with-mic for mgkuhn from 198.51.100.21 port 35620 ssh2: mgkuhn@DOMAIN

Also have a look at the Kerberos library debug output in /tmp/ssh-krb5trace for clues. (Somehow KRB5_TRACE=/dev/stdout didn't work for me, possibly because some child process used has stdout closed?) I get there:

Decrypted AP-REQ with server principal host/myserver.fqdn@DOMAIN: aes256-cts/949F
AP-REQ ticket: mgkuhn@DOMAIN -> host/myserver.fqdn@DOMAIN, session key aes256-cts/FACA
Negotiated enctype based on authenticator: aes256-cts
Authenticator contains subkey: aes256-cts/05FF
Creating AP-REP, time 1641831034.673366, subkey aes256-cts/3A3E, seqnum 53985744

Jan 10 '22 16:01 mgkuhn

Can you run sshd on the Linux server in foreground debug mode, as in
# KRB5_TRACE=/tmp/ssh-krb5trace /sbin/sshd -D -d -p 2222
and then connect to port 2222 with ssh -K -p 2222 ...?

If I do this, I see gssapi-with-mic success reported by sshd with lines such as
Authorized to mgkuhn, krb5 principal mgkuhn@DOMAIN (krb5_kuserok)
[...]
Accepted gssapi-with-mic for mgkuhn from 198.51.100.21 port 35620 ssh2: mgkuhn@DOMAIN

Yeah all the debug output I produced was using sshd on port 2222.

Also have a look at the Kerberos library debug output in /tmp/ssh-krb5trace for clues. (Somehow KRB5_TRACE=/dev/stdout didn't work for me, possibly because some child process used has stdout closed?) I get there:
Decrypted AP-REQ with server principal host/myserver.fqdn@DOMAIN: aes256-cts/949F
AP-REQ ticket: mgkuhn@DOMAIN -> host/myserver.fqdn@DOMAIN, session key aes256-cts/FACA
Negotiated enctype based on authenticator: aes256-cts
Authenticator contains subkey: aes256-cts/05FF
Creating AP-REP, time 1641831034.673366, subkey aes256-cts/3A3E, seqnum 53985744

I thihnk the stdout worked for me and I believe I have posted the output of the KRB5_TRACE above.

Jan 10 '22 16:01 pengsiya

$ ldapsearch -Y GSSAPI -Q -LLL -H ldap://domain -b DC=***,DC=com cn=svrdns
dn: CN=SRVDNS,OU=Linux,OU=***,OU=Servers,DC=***,DC=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
objectClass: computer
cn: SRVDNS
distinguishedName: CN=SRVDNS,OU=Linux,OU=***,OU=Servers,DC=***,DC=com
instanceType: 4
whenCreated: 20210708173713.0Z
whenChanged: 20220105034836.0Z
uSNCreated: 17092343
memberOf: CN=swd-r-defenderatponly,OU=Required,OU=Software,OU=Groups,DC=***,DC=com
uSNChanged: 21768232
name: SVRDNS
objectGUID:: ****
userAccountControl: 69632
codePage: 0
countryCode: 0
localPolicyFlags: 0
pwdLastSet: 132842534891110482
primaryGroupID: 515
objectSid:: AQUAAAAAAAUVAAAA+3UDDaFR5EKeXbdraxcBAA==
accountExpires: 9223372036854775807
sAMAccountName: SVRDNS$
sAMAccountType: 805306369
operatingSystem: pc-linux-gnu
dNSHostName: svrdns.domain
servicePrincipalName: RestrictedKrbHost/svrdns.domain
servicePrincipalName: RestrictedKrbHost/SVRDNS
servicePrincipalName: host/svrdns.domain
servicePrincipalName: host/SVRDNS
objectCategory: CN=Computer,CN=Schema,CN=Configuration,DC=****,DC=com
isCriticalSystemObject: FALSE
dSCorePropagationData: 16010101000000.0Z
lastLogonTimestamp: 132858277309780824
msDS-SupportedEncryptionTypes: 28

This is the account for the server though. I don't know why but I couldn't search for the "testuser" account I used to connect in LDAP.

Jan 10 '22 16:01 pengsiya

On the AD side, we didn't turn on the encryption for the service account we are using to connect since it isn't on by default.

What does “turn on encryption” mean exactly? (What exact GUI control or LDAP attribute are you talking about?)

The user account does not have encryption turned on:

Jan 10 '22 16:01 pengsiya

The only thing I found in my AD record for the linux machine is that the "Computer name" is full capital letters (no FQDN) but the "DNS name" is all lower cases (with FQDN). Could this be an issue?

I don't think so. SSPI doesn't need any of these entries for SSH. The Linux server doesn't really need to be fully “domain joined” in a Windows sense for ssh -K, all it needs is an SPN and a keytab. As I said, we just use normal user entries for our Linux servers and that works as well, e.g.
$ ldapsearch -Y GSSAPI -Q -LLL -H ldap://adsrv.dc.cl.cam.ac.uk -b DC=dc,DC=cl,DC=cam,DC=ac,DC=uk cn=hostDirac
dn: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: hostDirac
description: host/dirac.cl.cam.ac.uk
distinguishedName: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
instanceType: 4
whenCreated: 20160923125142.0Z
whenChanged: 20200723083126.0Z
displayName: hostDirac
uSNCreated: 67775
uSNChanged: 18297641
name: hostDirac
objectGUID:: wChZHsBwPUqBUH8iR6q4kQ== # c0 28 59 1e c0 70 3d 4a 81 50 7f 22 47 aa b8 91
userAccountControl: 544 # PASSWD_NOTREQD(32) | NORMAL_ACCOUNT(512)
codePage: 0
countryCode: 0
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAAVHhoalqY1Is9OO6uFg8AAA== # S-1-5-5-21-1785231444-2345965658-2934847549-3862
sAMAccountName: hostDirac
sAMAccountType: 805306368
userPrincipalName: host/[email protected]
servicePrincipalName: host/dirac.cl.cam.ac.uk
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
dSCorePropagationData: 20170208114138.0Z
dSCorePropagationData: 16010101000001.0Z
msDS-SupportedEncryptionTypes: 24 # AES128-CTS-HMAC-SHA1-96(8) | AES256-CTS-HMAC-SHA1-96(16)
I think only the servicePrincipalName and msDS-SupportedEncryptionTypes attributes really matter here for Kerberos use by SSH: one to find the entry (and associated password/keytab), and the other to control which encryption types are allowed (make sure the AES ones are enabled).

Can you use ldapsearch as above to query your domain controller via GSSAPI authentication? You will need
sudo apt-get install krb5-user ldap-utils libsasl2-modules-gssapi-mit

So here looks like your 'hostDirac' account is actually a user account as oppose to a computer account in my case? Do I need to create a user account to represent the machine? I think our convention is to use computer account for machine and user account only for real user + service accounts.

Jan 10 '22 16:01 pengsiya

Win32-OpenSSH Win32-OpenSSH copied to clipboard

GSSAPI Authentication from Windows to Linux failed

Win32-OpenSSH
Win32-OpenSSH copied to clipboard