(WIP) Add ed25519 support

[kevans91]

This ended up with a little more churn after the rsa refactoring we already landed as I got a clearer view of the pkgsign API. I'll break this PR up into some more consumable bits, but I wanted to get a draft up to make sure there's no glaring issues with the high-level overview. FreeBSD 11 is also likely still broken for the time being, but I will fix that. There are also some comments that I'll need to go back and revise now that my understanding has evolved.

pkgsign API

The new pkgsign API in libpkg is modelled after the RSA API that it's replacing. The main difference is that a consumer only needs to know which signer it wants up-front, it does not need to supply a password callback or key path. These are supplied after the fact to make it less awkward for those consumers that just want to do verification; before they did ad-hoc direct calls to rsa_verify() and rsa_verify_cert(), but now they're forced to create a pkgsign context and dispatch verification requests through the API.

The API does not currently support dynamic registration of pkgsign implementations, but that's something I'd like to eventually support so that a plugin could add different or experimental support for new signers.

RSA signer

The existing RSA signer has been abstracted further and lives in pkgsign_ossl.c. The OpenSSL signer internally has 'profiles' that are selected based on the name that was used to create the pkgsign context (see: ossl_new) which describe how it needs to work and must match the name provided to pkg-repo. The internals of verification has been refactored out of verify_cert and verify to make it easier to ensure we got it right.

PKG_HASH_TYPE_SHA256_HEX is supported as a hash method solely for legacy rsa compatibility, but it should not be used for any other going forward. Signing commands are currently expected to sign a sha256 of the sha256 hex checksum; the latter has been hardcoded in ossl_verify_cert_cb because this is not configurable. See TODO below.

There are two ways to verify/sign which are toggled between with PFLAG_RAW in the profile: raw op or one-shot digest.

Raw op

This matches what the previous implementation was doing, directly using EVP_PKEY_{sign,verify}. The underlying API assumes that the data has been pre-hashed, so the digest method must always be specified.

One-shot digest

One-shot uses the EVP_Digest{Verify,Sign}* methods for the underlying operation. ed25519 can only work specifically with EVP_Digest{Sign,Verify}() rather than the 'streaming' digest mode.

TODO

What's still needed?

• [ ] ossl_verify_cert_cb should avoid using sprof->cert_hash if we're doing a one-shot digest operatiion
• [ ] fingerprints: "function" configuration to verify_cert ("function" doesn't seem to be used beyond setting HASH_SHA256)
• [ ] fingerprints: add a "signature_type" field (it looks like old pkg will just ignore unknown fields in the fingerprint file)
• [ ] documentation
• [ ] probably more

[vstakhov]

Long time ago, I have designed a library for that purposes: https://github.com/vstakhov/asignify

[kevans91]

Long time ago, I have designed a library for that purposes: https://github.com/vstakhov/asignify

Ah, I'll take a look at that, thanks! It would be nice to have a method that we could make ed25519 available on FreeBSD 11 (even though it goes EoL before the end of the year).

My main concerns with this were portability and interoperability, because I want it to be easy to generate a key to work with it without requiring additional tooling that isn't likely to be present. As I write this, I think it would be cool to add a pkg-pkey utility that could standardize an interface for generating whatever keys pkg wants to work with using libasignify or openssl. It'd be a low priority, but then one doesn't need to reference external stuff to generate a key if they're not familiar -- pkg pkey generate [ed25519|rsa].