SSH experiences / braindump

[jspaaks]

So this issue is a bit of a mess but I needed a way to record some strange things I noticed while preparing materials for a xenon tutorial https://github.com/xenon-middleware/xenon-tutorial/tree/2b00c8d7625cf2966fdfe7c3e2aaeb6046fd7823. With the below items, we at least have concrete things to follow up on. From this collection, we should probably spawn new, singular issues.

1. ssh config options not used xenon-docker-image's Dockerfile because ssh(d) uses the first option it finds in a file even when there are repeated entries, and we echo options at the bottom. E.g. https://github.com/xenon-middleware/xenon-docker-images/blob/a87f1ef876055e19cedcccf4bae4012db2953258/slurm-ssh/Dockerfile#L10-L13. I'd prefer to copy a complete ssh_config file from the Docker context anyway, that way it's easier to see what the options are. Example setup here https://github.com/xenon-middleware/xenon-tutorial/tree/ed69661cdc7125ea15863fa2360fb8fd23309d10/containers

2. docker images for client side and server side, should be separate. e.g. slurm-ssh should be server side since it is a parent to all slurm images, but then the Dockerfile also has https://github.com/xenon-middleware/xenon-docker-images/blob/a87f1ef876055e19cedcccf4bae4012db2953258/slurm-ssh/Dockerfile#L10-L13

3. ssh client docker image should have different configurations, for example

   • with wrong permissions on directories, xenon should complain
   • with and without a ~/.ssh directory
   • with and without a ~/.ssh/config file
   • with various options in ~/.ssh/config, such as
     • hashKnownHosts
     • strictHostKeyChecking
   • with and without a known_hosts file
   • with a known_hosts file, with various types of entries in there

4. Xenon does not respect certain combinations of configuration parameters, specifically when inpuit argument knownHosts is false (see https://github.com/xenon-middleware/xenon-tutorial/pull/122), so by setting knownHosts to falseI can disobey input parameter stricHostCheck

5. my ssh understands strictHostKeyChecking=ask, xenon doesn't

6. in xenon, the config state is not assembled correctly from system config, user config, command line config

7. in xenon, you can't ask xenon what it thinks the current config state is, like you can with ssh -G xenon@localhost and sshd -T

8. in xenon, there is no way to control fallback behavior. ssh first tries public-private key, then interactive, then username-password (or something) this makes it more difficult to know that something is not working, and more difficult what is going wrong if you realize that something is going wrong

9. rethink options in SSHUtil https://github.com/xenon-middleware/xenon/blob/c24fba2c048c6d3266166d091c39aa1eac39aad8/src/main/java/nl/esciencecenter/xenon/adaptors/shared/ssh/SSHUtil.java#L187-L188

   1. useAgentForwarding is not used and only in the way
   2. stricHostCheck typo plus should be strictHostKeyChecking
   3. stricHostCheck option is unused if I set useKnownHosts false
   4. loadSSHConfig ambiguous which SSHConfig you're talking about: system, user, commandline, or the combination of these

10. xenon-docker-images have multiple key pairs, would be better to have one per image, then you know how you got inside a container, or if it doesn't work, you know why (see https://github.com/xenon-middleware/xenon-docker-images/tree/07db6c0f12621e51bd109a5325fe08b1c0e9123f/slurm-ssh/.ssh)

11. docker images client side ssh has authorized_keys like https://github.com/xenon-middleware/xenon-docker-images/blob/a87f1ef876055e19cedcccf4bae4012db2953258/slurm-ssh/Dockerfile#L27, not sure that's needed if it is only client

12. xenon docker images (slurm:17 image): not sure what the value of PasswordAuthentication is):

    from ``/etc/ssh/sshd_config``
    # The strategy used for options in the default sshd_config shipped with
    # OpenSSH is to specify options with their default value where
    # possible, but leave them commented.  Uncommented options change a
    # default value.
    

    and

    #PasswordAuthentication no
    

    seems to suggest password authentication is disabled by default. Stands to reason, since it is not as secure as keypairs. However, if I do sshd -T, I see passwordauthentication yes. Not sure where that's coming from.