Feature request: new parser sshd -T

[chriscroome]

This is probably one for the far back-burner or bin @kellyjonbrazil :smile: !

The configuration of an OpenSSH server can be printed using sshd -T, for example:

sshd -T | sort
acceptenv LANG
acceptenv LC_*
addressfamily any
allowagentforwarding yes
allowstreamlocalforwarding yes
allowtcpforwarding yes
authenticationmethods any
authorizedkeyscommand none
authorizedkeyscommanduser none
authorizedkeysfile .ssh/authorized_keys .ssh/authorized_keys2
authorizedprincipalscommand none
authorizedprincipalscommanduser none
authorizedprincipalsfile none
banner none
casignaturealgorithms ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256
chrootdirectory none
ciphers [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
clientalivecountmax 3
clientaliveinterval 0
compression yes
disableforwarding no
exposeauthinfo no
fingerprinthash SHA256
forcecommand none
gatewayports no
gssapiauthentication no
gssapicleanupcredentials yes
gssapikexalgorithms gss-group14-sha256-,gss-group16-sha512-,gss-nistp256-sha256-,gss-curve25519-sha256-,gss-group14-sha1-,gss-gex-sha1-
gssapikeyexchange no
gssapistorecredentialsonrekey no
gssapistrictacceptorcheck yes
hostbasedacceptedalgorithms [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256
hostbasedauthentication no
hostbasedusesnamefrompacketonly no
hostkeyagent none
hostkeyalgorithms [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256
hostkey /etc/ssh/ssh_host_ecdsa_key
hostkey /etc/ssh/ssh_host_ed25519_key
hostkey /etc/ssh/ssh_host_rsa_key
ignorerhosts yes
ignoreuserknownhosts no
ipqos lowdelay throughput
kbdinteractiveauthentication no
kerberosauthentication no
kerberosorlocalpasswd yes
kerberosticketcleanup yes
kexalgorithms [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
listenaddress 0.0.0.0:22
listenaddress [::]:22
logingracetime 120
loglevel INFO
macs [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
maxauthtries 6
maxsessions 10
maxstartups 10:30:100
modulifile /etc/ssh/moduli
passwordauthentication yes
permitemptypasswords no
permitlisten any
permitopen any
permitrootlogin without-password
permittty yes
permittunnel no
permituserenvironment no
permituserrc yes
persourcemaxstartups none
persourcenetblocksize 32:128
pidfile /run/sshd.pid
port 22
printlastlog yes
printmotd no
pubkeyacceptedalgorithms [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256
pubkeyauthentication yes
pubkeyauthoptions none
rekeylimit 0 0
revokedkeys none
securitykeyprovider internal
streamlocalbindmask 0177
streamlocalbindunlink no
strictmodes yes
subsystem sftp /usr/lib/openssh/sftp-server
syslogfacility AUTH
tcpkeepalive yes
trustedusercakeys none
usedns no
usepam yes
versionaddendum none
x11displayoffset 10
x11forwarding yes
x11uselocalhost yes
xauthlocation /usr/bin/xauth

For a raw JSON version splitting on the first space on each line would be mostly fine, perhaps the Key/Value file parser could have support for splitting on the first space per line added? However that wouldn't work for HostKey as only the last value would be left.

A dedicated sshd_config parser could do things like split values at commas and / or spaces into lists, however it isn't quite that simple...

For example for Ciphers, HostbasedAcceptedAlgorithms and KexAlgorithms:

if the specified list begins with a ‘+’ character, then the specified signature algorithms will be appended to the default set instead of replacing them. If the specified list begins with a ‘-’ character, then the specified signature algorithms (including wildcards) will be removed from the default set instead of replacing them. If the specified list begins with a ‘^’ character, then the specified signature algorithms will be placed at the head of the default set

Perhaps this could look something this?

KexAlgorithms:
 strategy: append # for +
 algorithms:
   - [email protected]
   - curve25519-sha256,[email protected]
   - ecdh-sha2-nistp256

Another complication are things like AuthenticationMethods:

This option must be followed by one or more lists of comma-separated authentication method names, or by the single string any to indicate the default behaviour of accepting any single authentication method.

None of this initial ideas are great:

AuthenticationMethods: any

AuthenticationMethods:
  - publickey
  - password

AuthenticationMethods:
  1:
    - publickey
    - password
  2:
    - publickey
    - keyboard-interactive

Also since YAML / JSON doesn't support ordered lists and the order is critical that could be an issue and I'm not sure that a variable having multiple potential types is a good idea.

In addition the sshd_config file uses camel case but sshd -T has lower case variable names, I don't know if a mapping for this would be a good thing and / or a necessary option?, this isn't something to worry about since the man page for sshd_config states:

keywords are case-insensitive and arguments are case-sensitive

Another potential gotcha is that some variables take simply yes / no as values, like AllowAgentForwarding, so this would perhaps make sense as a boolean rather than a string, however AllowTcpForwarding allows all, local, no, remote or yes so this could be a boolean or a string depending on the value, however my preference would probably be for it always to be a string since Ansible role validation only support variables being of one type.

To make matters worse there is the Match block, for example in /etc/ssh/sshd_config you could have:

Match group sftp
    AllowGroups sftp
    DenyGroups sudo root
    ChrootDirectory %h
    X11Forwarding no
    AllowTcpForwarding no
    AuthenticationMethods publickey password
    PasswordAuthentication yes
    PubkeyAuthentication yes
    PermitUserRC no
    PermitRootLogin No
    PermitTTY yes
    ForceCommand internal-sftp

However allthough you can get the resulting configuration for a user using sshd -T -C user=foo the results are still in the same flat format and the Match directive is never printed.

Perhaps a JC sshd_config parser isn't a practical suggestion... in any case the full list of configuration options are here.

[kellyjonbrazil]

Hey thanks for the suggestion and analysis! I'm working on /proc files for the next release but I'll take a look at this, too.

[chriscroome]

Thanks!

One other thought, support for ssh -Q would also be great since you could then, for example get a list of available ciphers:

ssh -Q ciphers
3des-cbc
aes128-cbc
aes192-cbc
aes256-cbc
aes128-ctr
aes192-ctr
aes256-ctr
[email protected]
[email protected]
[email protected]

And compare it with the enabled ciphers using ssh -T.

However perhaps a special parser isn't needed when the output is a simple list of items, one per line, is there a already a suitable parser for output like this?

[kellyjonbrazil]

Yeah I prob wouldn’t write a parser for a list of items. K/v pairs at the minimum.

[chriscroome]

Perhaps treating all the results as key / value pairs, seperated at the first space, with one exception for HostKeys, for example:

hostkey1: /etc/ssh/ssh_host_ecdsa_key
hostkey2: /etc/ssh/ssh_host_ed25519_key
hostkey3: /etc/ssh/ssh_host_rsa_key

Would probably be the easiest and perhaps the most sensible thing to do, leave other tools to seperate values into lists and decide if something could be a boolean or a string.

[chriscroome]

I've just discovered the ssh-audit tool and it has JSON output, for example ssh-audit -j github.com | yq -P results in:

banner:
  comments: null
  protocol:
    - 2
    - 0
  raw: SSH-2.0-babeld-81baa361
  software: babeld-81baa361
compression:
  - none
  - [email protected]
  - zlib
enc:
  - [email protected]
  - [email protected]
  - [email protected]
  - aes256-ctr
  - aes192-ctr
  - aes128-ctr
fingerprints:
  - hash: +DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU
    hash_alg: SHA256
    hostkey: ssh-ed25519
  - hash: 65:96:2d:fc:e8:d5:a9:11:64:0c:0f:ea:00:6e:5b:bd
    hash_alg: MD5
    hostkey: ssh-ed25519
  - hash: nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8
    hash_alg: SHA256
    hostkey: ssh-rsa
  - hash: 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48
    hash_alg: MD5
    hostkey: ssh-rsa
kex:
  - algorithm: curve25519-sha256
  - algorithm: [email protected]
  - algorithm: ecdh-sha2-nistp256
  - algorithm: ecdh-sha2-nistp384
  - algorithm: ecdh-sha2-nistp521
  - algorithm: diffie-hellman-group-exchange-sha256
    keysize: 2048
key:
  - algorithm: ssh-ed25519
  - algorithm: ecdsa-sha2-nistp256
  - algorithm: rsa-sha2-512
    keysize: 2048
  - algorithm: rsa-sha2-256
    keysize: 2048
  - algorithm: ssh-rsa
    keysize: 2048
mac:
  - [email protected]
  - [email protected]
  - hmac-sha2-512
  - hmac-sha2-256
target: github.com

It can also be run against localhost

[kellyjonbrazil]

Hi Chris!

Does it look like we can close this parser request out?

[chriscroome]

If the Key/Value file parser had support for splitting on the first whitespace character in addition to : and = plus an alternative duplicate option, for example to append a number rather than disgarding all but the last duplicate, so for example this:

sshd -T | grep -e "^hostkey "
hostkey /etc/ssh/ssh_host_rsa_key
hostkey /etc/ssh/ssh_host_ecdsa_key
hostkey /etc/ssh/ssh_host_ed25519_key

Could perhaps result in:

{
  "hostkey1": "/etc/ssh/ssh_host_rsa_key",
  "hostkey2": "/etc/ssh/ssh_host_ecdsa_key",
  "hostkey3": "/etc/ssh/ssh_host_ed25519_key"
}

Then I'd agree that there wouldn't be much point in a special parser.

[kellyjonbrazil]

I just wasn't sure if the ssh-audit tool met your use case for this issue.

Depending on how common that type of file output is I could make a parser called split or cut or something that splits on the first space and treats every line like an object:

$ echo '
key value1
key value2
key value3
' | jc --split -p
[
  {"key": "value1"},
  {"key": "value2"},
  {"key": "value3"}
]

Could even make it more fancy - like when using the --raw option (or vice versa) it could do this:

$ echo '
key value1
key value2
key value3
' | jc --split -r -p
{
  "key_0": "value1",
  "key_1": "value2",
  "key_2": "value3",
}

Just spitballing here - not sure if this type of space-delimited output is common enough for its own parser.

[chriscroome]

The JSON output from ssh-audit is handy but it only contains public data, it doesn't (and can't by it's nature) print the non-public configuration details that you have access to with sshd -T.

In terms of a new split or cut parser that sounds fine, if you would prefer that to considering extending the key / value parser, I also don't know how common the sshd -T format is...

[kellyjonbrazil]

Hi @chriscroome - I have an initial version of the sshd_conf parser:

https://github.com/kellyjonbrazil/jc/blob/dev/jc/parsers/sshd_conf.py

Here's what it looks like so far:

$ sshd -T | jc --sshd-conf -p
{
  "acceptenv": [
    "LANG",
    "LC_*",
    "test1",
    "test2"
  ],
  "addressfamily": "any",
  "allowagentforwarding": "yes",
  "allowstreamlocalforwarding": "yes",
  "allowtcpforwarding": "yes",
  "authenticationmethods": "any",
  "authorizedkeyscommand": "none",
  "authorizedkeyscommanduser": "none",
  "authorizedkeysfile": [
    ".ssh/authorized_keys",
    ".ssh/authorized_keys2"
  ],
  "authorizedprincipalscommand": "none",
  "authorizedprincipalscommanduser": "none",
  "authorizedprincipalsfile": "none",
  "banner": "none",
  "casignaturealgorithms": [
    "ssh-ed25519",
    "ecdsa-sha2-nistp256",
    "ecdsa-sha2-nistp384",
    "ecdsa-sha2-nistp521",
    "[email protected]",
    "[email protected]",
    "rsa-sha2-512",
    "rsa-sha2-256"
  ],
  "chrootdirectory": "none",
  "ciphers": [
    "[email protected]",
    "aes128-ctr",
    "aes192-ctr",
    "aes256-ctr",
    "[email protected]",
    "[email protected]"
  ],
  "ciphers_strategy": "+",
  "clientalivecountmax": 3,
  "clientaliveinterval": 0,
  "compression": "yes",
  "disableforwarding": "no",
  "exposeauthinfo": "no",
  "fingerprinthash": "SHA256",
  "forcecommand": "none",
  "gatewayports": "no",
  "gssapiauthentication": "no",
  "gssapicleanupcredentials": "yes",
  "gssapikexalgorithms": [
    "gss-group14-sha256-",
    "gss-group16-sha512-",
    "gss-nistp256-sha256-",
    "gss-curve25519-sha256-",
    "gss-group14-sha1-",
    "gss-gex-sha1-"
  ],
  "gssapikeyexchange": "no",
  "gssapistorecredentialsonrekey": "no",
  "gssapistrictacceptorcheck": "yes",
  "hostbasedacceptedalgorithms": [
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "ssh-ed25519",
    "ecdsa-sha2-nistp256",
    "ecdsa-sha2-nistp384",
    "ecdsa-sha2-nistp521",
    "[email protected]",
    "[email protected]",
    "rsa-sha2-512",
    "rsa-sha2-256"
  ],
  "hostbasedauthentication": "no",
  "hostbasedusesnamefrompacketonly": "no",
  "hostkeyagent": "none",
  "hostkeyalgorithms": [
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "ssh-ed25519",
    "ecdsa-sha2-nistp256",
    "ecdsa-sha2-nistp384",
    "ecdsa-sha2-nistp521",
    "[email protected]",
    "[email protected]",
    "rsa-sha2-512",
    "rsa-sha2-256"
  ],
  "hostkey": [
    "/etc/ssh/ssh_host_ecdsa_key",
    "/etc/ssh/ssh_host_ed25519_key",
    "/etc/ssh/ssh_host_rsa_key"
  ],
  "ignorerhosts": "yes",
  "ignoreuserknownhosts": "no",
  "ipqos": [
    "lowdelay",
    "throughput"
  ],
  "kbdinteractiveauthentication": "no",
  "kerberosauthentication": "no",
  "kerberosorlocalpasswd": "yes",
  "kerberosticketcleanup": "yes",
  "kexalgorithms": [
    "[email protected]",
    "curve25519-sha256",
    "[email protected]",
    "ecdh-sha2-nistp256",
    "ecdh-sha2-nistp384",
    "ecdh-sha2-nistp521",
    "diffie-hellman-group-exchange-sha256",
    "diffie-hellman-group16-sha512",
    "diffie-hellman-group18-sha512",
    "diffie-hellman-group14-sha256"
  ],
  "listenaddress": [
    "0.0.0.0:22",
    "[::]:22"
  ],
  "logingracetime": 120,
  "loglevel": "INFO",
  "macs": [
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "hmac-sha2-256",
    "hmac-sha2-512",
    "hmac-sha1"
  ],
  "macs_strategy": "^",
  "maxauthtries": 6,
  "maxsessions": 10,
  "maxstartups": 10,
  "modulifile": "/etc/ssh/moduli",
  "passwordauthentication": "yes",
  "permitemptypasswords": "no",
  "permitlisten": [
    "any"
  ],
  "permitopen": [
    "any"
  ],
  "permitrootlogin": "without-password",
  "permittty": "yes",
  "permittunnel": "no",
  "permituserenvironment": "no",
  "permituserrc": "yes",
  "persourcemaxstartups": "none",
  "persourcenetblocksize": "32:128",
  "pidfile": "/run/sshd.pid",
  "port": [
    22
  ],
  "printlastlog": "yes",
  "printmotd": "no",
  "pubkeyacceptedalgorithms": [
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "ssh-ed25519",
    "ecdsa-sha2-nistp256",
    "ecdsa-sha2-nistp384",
    "ecdsa-sha2-nistp521",
    "[email protected]",
    "[email protected]",
    "rsa-sha2-512",
    "rsa-sha2-256"
  ],
  "pubkeyauthentication": "yes",
  "pubkeyauthoptions": "none",
  "rekeylimit": 0,
  "revokedkeys": "none",
  "securitykeyprovider": "internal",
  "streamlocalbindmask": "0177",
  "streamlocalbindunlink": "no",
  "strictmodes": "yes",
  "subsystem": "sftp",
  "syslogfacility": "AUTH",
  "tcpkeepalive": "yes",
  "trustedusercakeys": "none",
  "usedns": "no",
  "usepam": "yes",
  "versionaddendum": "none",
  "x11displayoffset": 10,
  "x11forwarding": "yes",
  "x11uselocalhost": "yes",
  "xauthlocation": "/usr/bin/xauth",
  "maxstartups_rate": 30,
  "maxstartups_full": 100,
  "rekeylimit_time": 0,
  "subsystem_command": "/usr/lib/openssh/sftp-server"
}

Let me know what you think!

[kellyjonbrazil]

I don't think this version will support sshd_config files that include Match blocks. I'd like to see some (sanitized) real-world file examples that have Match blocks. I could either ignore them or maybe do something like this:

{
  "foo": "bar",
  "matches": [
    {
      "pattern": "abcd",
      "key1": "val1"
      "key2": "val2",
      ...
    },
    ...
  ]
}

[chriscroome]

This is an example of a Match block from /etc/ssh/sshd_config:

Match group chroot
    AllowGroups chroot
    DenyGroups root sudo
    ChrootDirectory /chroots/%u
    X11Forwarding no
    AllowTcpForwarding no
    AuthenticationMethods publickey password
    PasswordAuthentication yes
    PubkeyAuthentication yes
    PermitUserRC no
    PermitTTY yes
    PermitRootLogin No

However I can't remember where I found some examples of the syntax to use when running sshd -T -C, have you come across any?

[kellyjonbrazil]

No, no luck yet. It should be easy enough to ignore lines that start with Match or whitespace for now, but not sure if that is a good heuristic.

[chriscroome]

There shouldn't be an issue with Match, with the example block above, if the foobar user is a member of the chroot group then you can get the SSH config for them like this:

sshd -T -C user=foobar

And the results looks the same as simply for sshd -T except the variables that are set in the match block are displayed rather then the defaults.

[kellyjonbrazil]

Yeah, I think that makes sense for ssh -T [-C...]. I was hoping to be able to also parse the sshd config file as well, but of course I won't be able to do it to the extent that sshd does.

From what I've read, you are supposed to put Match blocks at the end of the file and you can terminate Match blocks with Match all. So maybe I either ignore all lines after Match or anything between a Match xxx and Match all.

[kellyjonbrazil]

Ok, I added the logic to ignore the Match blocks. Let's see how that works out.

[chriscroome]

Sounds good, FWIW my plan is to generate Match blocks using Ansible and for each block to be in a separate file that is Include'd, see this issue. I also have an issue to make use of the parser you are writing.

[kellyjonbrazil]

Can there be multiple Include lines? If so, I'll probably make that one a list as well.

[kellyjonbrazil]

I think I figured out the Include directive. Looks like you could have multiple paths (guessing space delimited) on a single line and also multiple directives on other lines. This is not well documented, but I think I have accounted for all the scenarios. This should be good to go in the next release.

https://github.com/kellyjonbrazil/jc/blob/dev/jc/parsers/sshd_conf.py