storage/connections/ssh: support validating host key

[benesch]

Feature request

When #15722 is complete, SSH tunnels will no longer validate host keys. This is the right default for cloud environments, as the remote SSH bastion is often an ephemeral cloud resource that is frequently recycled. But security minded users might want the ability to specify the expected SSH host key.

Here's a quick sketch of a proposal. We'd add a new option to SSH TUNNEL like HOST KEY (name to be bikeshed):

CREATE CONNECTION ssh_connection TO SSH TUNNEL (
    HOST '<SSH_BASTION_HOST>',
    USER '<SSH_BASTION_USER>',
    PORT <SSH_BASTION_PORT>,
    HOST KEY 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKH8NDvceJ0LbBDTujj6NSvG6kGDNAHzIIOdr2/kI/Dr'
);

Whenever Materialize establishes the SSH tunnel, it would verify that the remote host presented the specified key.

We'd likely need to support ALTER CONNECTION ... SET (HOST KEY ...), as host keys do change. Dropping and recreating the connection isn't a viable option, since that would a) require recreating the source and any downstream dependencies, and b) regenerate the SSH keys associated with the connection.

I don't think this is worth prioritizing any time soon. No users have asked for it so far. ~And if ALTER CONNECTION is a hard requirement, it'd be a fairly large lift, as we don't currently support ALTER CONNECTION for changing the parameters of a connection (just the special case of rotating SSH keys). We'd need to design quite a bit of infrastructure to plumb changes in connection options down to the running sources (or killing the running sources associated with that connection, idk).~

cc @nmeagan11 @uce