k0sctl not honoring GlobalKnownHostsFile from ssh_config

[Audiobuzz]

Problem Statement

Running k0sctl on Ubuntu 24.04 hosts with signed ssh certificates appears to ignore any entries in the global known_hosts file.

Environment

• Freshly installed Ubuntu 24.04 hosts for both k0sctl, control plane hosts and worker hosts
• All hosts have their SSH host key signed by a common CA and have a single entry in /etc/ssh/ssh_known_hosts
• /etc/ssh/ssh_config has the GlobalKnownHostsFile pointing to /etc/ssh/ssh_known_hosts (default for ubuntu) as per https://manpages.ubuntu.com/manpages/focal/man5/ssh_config.5.html
• ~/.ssh/known_hosts is empty

Expected Behaviour

k0sctl should read both /etc/ssh/ssh_known_hosts and ~/.ssh/known_hosts as per the ssh_config for the host

Current Behaviour

k0sctl complains when trying to connect to hosts with signed certificates with the error:

not connected: client connect: can't connect: ssh: handshake failed: host key mismatch: ssh: no authorities for hostname:

Work Around

Adding the lines from /etc/ssh/ssh_known_hosts to ~/.ssh/known_hosts allows k0sctl to continue but this is not how many systems are configured, expecially when using signed certificates with a common CA

Proposed fix

• Ideally it should read in ssh_config to determine the paths to the known_hosts files including the GlobalKnownHostsFile parameter
• If thats not possible then it should at least try loading /etc/ssh/ssh_known_hosts along with ~/.ssh/known_hosts

[kke]

The handling of ssh configuration and known hosts files is indeed flawed in many ways.