GPG support

[SantiagoTorres]

Description of issue or feature request:

The in-toto python implementation supports gpg keys and signatures. We could add this feature using the existing golang pgp support. (the module exists, but finding it is a pain...) Current behavior: any metadata with rfc4880 packets will fail to load

Expected behavior: They should be parsed and handled appropriately

[shibumi]

Aren't we supporting PKCS8 keys already? Doesn't this mean that we support loading GPG keys from disk, too? Or do you mean docking to the gpg-agent for all operations? I am kinda afraid this would be a huge effort.

I read through https://tools.ietf.org/html/rfc4880 and it mentions PKCS1-5 as key format.. I think we can load such keys already, because we are using the pkcs/pem golang libraries. Maybe we should generate a few test keys with GPG?

[nmiyake]

Chiming in on this issue because I am interested in this as well.

Summary

I believe the intent of this issue is to point out that the reference implementation supports using GPG signatures for signing and verification (and for storing in the in-toto JSON), and the golang implementation is not interoperable with that. The current support for RSA keys can be used in cases where you can export the GPG private key, but it does not support use cases that involve hardware tokens that generate and store GPG keys and perform GPG signature operations but don't allow exporting the private keys (for example, YubiKeys) -- this is presumably why the support for GPG signing and verification exists in the reference implementation.

The Python reference implementation has implemented this feature, but the manner in which GPG signatures are represented and interpreted in the generated in-toto JSON is not part of the specification. Documenting these format/expectations more explicitly should probably be done as part of this work so that other future implementations that want to add this support (like Java) can refer to the spec rather than having to read through and match the behavior of the reference implementation.

The actual GPG signing/invocations is pretty straightforward (the reference implementation calls gpg directly in a subprocess and the equivalent can be done in the Go implementation), but there is a lot of supporting code in github.com/secure-systems-lab/securesystemslib that deals with parsing packets and putting things in the right format that would need to be ported and verified.

Full details

The Python reference implementation basically implements a parallel code path to handle GPG signing. The actual GPG operations are done by executing the local gpg executable. Although the CLI API is technically not guaranteed to be stable, this is a pretty reasonable approach (git uses this approach for GPG signing for commits/tags).

However, beyond just calling out to gpg, the Python implementation does a lot of custom work in doing things like parsing the signature packet and storing headers as part of the key JSON -- for example, a GPG signature generated by the reference implementation is of the form:

{
 "signatures": [
  {
   "keyid": "aad6ec15d80aca160e1a0e7041fc235573127eb3",
   "other_headers": "04000108001d162104aad6ec15d80aca160e1a0e7041fc235573127eb30502615bdc17",
   "signature": "5be45bc0d63301d487d3eac7605d0440aba0017fcaef987066d1451ed02e696b53ffa6404bb97698becc69ec4cbe2efc58e4148151aa48c2d0d6b3f5544672cf4d596c4ac0bdff6350e4a8c4a034fae3286e123e3e934c2c14bace75126d8b9e2fea08055a21674875dac6284e1c8eba142cab8f0b14b2547f50135745dc5b051b2b7a59c1e5578b68917ca2d3c061235c0a3c93c649df4140e0acda9b779b2fefd0e767e376afb67c7bde86fd904e75d99efef664ed1561471645b3bb642adf1cf2707819e9246cb71410445e00cc130d237e8904710266411d3bc166ffb3c90407f54d52cc3f5e514bfc8823053a681136a230983c3c2bf38371b456cb1d1eae322119e3a697e0cfed27504eb55bcf8bae8cb33c88ef155477aa97b523834a8efe1efc0fb12de6a32b84c4eb4230682c5554d34e6fd54de6928c675a98df633a1ed17abb77958e2da0b5230c57f7e4c817b42aa86d26a6583fdbb2c96e0d6c46e7bafe02b1d600a146476b7434bd511741414e40309d6d99715f75b1644bb04470311520b76114a4cf75abceaf805f3b2d8ce2b0cca56dbba231116b83b0aabf920b3602e06c334de13ceccb8b5ef9c0f3fd1b85a2c07126d513d9c45ccc54d5036d4a31a4f4299f8081811e83fe6d50aeb85c09959e43f46ce8d9053da9dba330f57faacbadeaa5bc1533fc03bcd8d3f1b24f66ad403691d6118a43df7a74"
  }
...

Note the presence of the other_headers field that is not documented in the spec (filed https://github.com/in-toto/docs/issues/55 to track that) -- how the output of the gpg executable's "sign" operation is transformed to the "other_headers" and "signature" parts and how verification is performed are things that need to be documented (or at least matched) in order for different implementations to be compatible with each other.

Implementing this would effectively consist of:

1. Add CLI flags for specifying GPG keys
2. If GPG keys are specified, execute gpg to perform signing operations for signing/retrieving public keys
3. Determine the correct content of the signature object for GPG signatures (right now this is not documented in the spec -- it is just implemented in Python in the reference implementation) and write/port logic to generate for signing operations
4. For verification, write/port logic that gets GPG public key and transforms input signature data to a form that can be verified

The baseline for feature completion would be that the Golang and Python implementation can sign and verify GPG signatures produced by each other. However, it would also be helpful to formalize the expectations as part of the specification (or at least to document them) so that it's not necessary for all implementations to check against each other.