Public key auth fails against certain servers when using rsa-sha2-* based keys

[pmrowla]

When connecting to SSH servers using a rsa-sha2-256 or rsa-sha2-512 key, asyncssh expects that the server returns the proper algorithm in MSG_USERAUTH_PK_OK, and fails here when the algorithm does not match: https://github.com/ronf/asyncssh/blob/10aa6f4082977a74bbd34496280c5d29968771a5/asyncssh/auth.py#L348 (as far as I can tell this is behaving properly according to the SSH RFC's)

However, it appears that there are existing servers that don't behave properly, and always return ssh-rsa as the algorithm in MSG_USERAUTH_PK_OK, even when the server properly supports the RSA SHA2 keys (the server returns the proper rsa-sha2-256 or rsa-sha2-512 key data in the packet, but always returns the algorithm name as ssh-rsa).

In particular, this happened when trying to connect to github as of a few weeks ago, but in testing is apparently no longer reproducible today, probably due to the change documented here?

Note: GitHub is improving security by dropping older, insecure key types.

RSA keys (ssh-rsa) with a valid_after before November 2, 2021 may continue to use any signature algorithm. RSA keys generated after that date must use a SHA-2 signature algorithm. Some older clients may need to be upgraded in order to use SHA-2 signatures.

I should also note that we have received a report with the same issue from a DVC user (where we use asyncssh for Git remote operations and also via sshfs, but unfortunately the user does not have access the server and (so far) hasn't been able to provide us with any info about what particular sshd implementation/version is causing the problem.

Given that this is no longer reproducible with Github, this is maybe not something that needs to be addressed in asyncssh right now, but we thought this is an issue you should at least be aware of. And I'll update this ticket if/when we get more info about any other specific servers that behave in this way.

---

For reference, this was reproducible by generating an RSA key with ssh-keygen -t rsa -b 4096, adding it in github, and then using the following script (and it would fail on connect with the exception ProtocolError("Key mismatch")):

async def run_client():
    async with asyncssh.connect(
        "github.com",
        username="git",
        client_keys=["id_rsa_github"],
    ) as conn:
        print("connected")
        result = await conn.run("git receive-pack nonexistent_repo", check=False)
        print(result.stderr, end="")

try:
    asyncio.get_event_loop().run_until_complete(run_client())
except (OSError, asyncssh.Error) as exc:
    sys.exit('SSH connection failed: ' + str(exc))

We were able to work around the (no longer reproducible) issue with github's server by monkeypatching _process_public_key so that it would accept ssh-rsa from the server when the local key signature was any of ssh-rsa or rsa-sha2-*:

from asyncssh.auth import MSG_USERAUTH_PK_OK, _ClientPublicKeyAuth

def _process_public_key_ok_gh(self, _pkttype, _pktid, packet):
    from asyncssh.misc import ProtocolError

    algorithm = packet.get_string()
    key_data = packet.get_string()
    packet.check_end()

    if (
        (
            algorithm == b"ssh-rsa"
            and self._keypair.algorithm
            not in (
                b"ssh-rsa",
                b"rsa-sha2-256",
                b"rsa-sha2-512",
            )
        )
        or (algorithm != b"ssh-rsa" and algorithm != self._keypair.algorithm)
        or key_data != self._keypair.public_data
    ):
        raise ProtocolError("Key mismatch")

    self.create_task(self._send_signed_request())
    return True

_ClientPublicKeyAuth._packet_handlers[
    MSG_USERAUTH_PK_OK
] = _process_public_key_ok_gh

[ronf]

I really appreciate the heads-up, Peter.

If the only issue is with the value of 'algorithm' in the PK_OK packet not matching the keypair's algorithm but the actual public key data still matches, it might be possible to fix this by changing the test from:

        if (algorithm != self._keypair.algorithm or
                key_data != self._keypair.public_data):

to:

        if (algorithm not in self._keypair.sig_algorithms or
                key_data != self._keypair.public_data):

It might also be possible to just remove the first half of this test, since self._keypair.public_data actually contains both the algorithm and the key data and from what you've said here it looks like the key_data returned is still an exact match for what was sent out. However, I think it'd probably be best if I waited to relax this check until there's a reproducible test case. So, please let me know if you hear about any other servers with this problem.

Thanks very much!

[ronf]

Closing this for now, but feel free to re-open if you see an issue like this again. There have also been a few improvements in this area, particularly when trying to use RSA SHA-2 signatures with OpenSSH certificates. In particular, see #570, which replaces a previous fix in #517 with something that works for both older and newer OpenSSH servers.