support PGP/MIME format?

Open winnie-chaintope opened this issue 3 years ago • 15 comments

final MessageBuilder builder = MessageBuilder.prepareMultipartAlternativeMessage();
final MimeMessage mimeMessage = builder.buildMimeMessage();

does this package have options to build mimeMessage in PGP/MIME format? probably had to do with the builder setup, but i didn't find relevant function

Oct 01 '20 00:10 winnie-chaintope

I'd love to support PGP in this project but currently do not know how to accomplish this.

Oct 02 '20 12:10 robert-virkus

Currently, I work on a Flutter PGP mail client based on enough_mail. For my project, I decided it is better to separate PGP from IMAP/SMTP. I am using https://pub.dev/packages/openpgp to realize the encryption.

If PGP support was included into enough_mail, it would simply be a secondary implementation of flutter_openpgp which does not make that much sense.

Oct 27 '20 10:10 TheOneWithTheBraid

Has anything happened here? Looks like TheOneWithTheBraid did not publish anything on GitHub so far. I noticed the extension mechanism in Maily. Can it be used for that purpose?

Jul 13 '21 17:07 tallinn1960

Never mind. I looked into what extensions are able to provide and that doesn't cut it as it seems. I am willing to contribute to the task at hand, the incorporation of PGP into Maily. But I would like to have a little help here regarding enough_mail. One needs a kind of pipeline from the original message-text to the final e-mail with recursion to implement PGP as every PGP-encrypted part of a MIME-Message will yield one or more MIME message parts (which may include encrypted parts as well - such situations may appear if someone answers to an encrypted mail). My development environment is macOS and I have access to both GPG mail for macOS and Gpg4win for Outlook (which handles PGP encryption differently, there are two "standards" to support). And, being a security consultant with the speciality of PKI applications (PGP is one of those), I guess, I am capable of doing that stuff.

Jul 13 '21 21:07 tallinn1960

Hey that would be super-awesome and I will gladly help you with the integration. Maybe I can try to build the generic decryption / encryption pipeline you are describing and you can integrate the actual PGP logic there? Looking very much forward to this!!!

Jul 14 '21 07:07 robert-virkus

Had a look this evening on the PGP/MIME spec, that seems to be rather easy. The whole MIME-Message is optionally signed, then encrypted and produces a overall MIME mail of type multipart/encrypted, which yields either the cleartext MIME-message or a MIME-element of multipart/signed with two parts, the cleartext MIME-message and the signature data. But as with all standards that's theory. Only macOS mail with GPGmail extension produces a full standard compliant format. Evolution on Linux produces a multipart/mixed message and that there is PGP encryption in place is only recognizable by looking at the parts, there is one (the second of three) that clearly indicates an encrypted message. Only cleartext signed messages are full standard compliant here (i.e. a multipart/signed message). I will look into Thunderbird and Gpg4win/Outlook this weekend.

But the real mess starts with PGP-INLINE, which is an alternative encryption and signature format, where the Mime-Parts of a message are independently PGP-signed and/or encrypted. Those parts are recognizable by things like ---BEGIN PGP MESSAGE---/---END PGP MESSAGE--- delimiters. IIRC Gpg4win is generating PGP-INLINE messages with Outlook (and again IIRC that is as Outlook does not provide access to the whole MIME-message-source for the plugin, but to a data structure presenting a list of parts). But I will check that this weekend.

But if the pipeline detects and delivers the parts which need cryptographic treatment to a decryption and/or signature verification routine than that would be a straightforward application of the openpgp package for flutter, which would return decrypted Mime-Messages (full source with headers and all, which then would be fed into the pipeline to generate a presentation again) and/or signature verification info. Producing encrypted and/or signed messages is even simpler, as we should stick with the PGP/Mime standard like macOS Mail/GPGMail does. All clients I've used so far understand that format. Once the full cleartext mime-message is produced, it is fed into a signature generation routine (optionally) which yields a multipart/signed message and then optionally into an encryption routine, which would yield a multipart/encrypted message.

Care needs to be taken to present all MIME-Messages in canonical format to the signature generation and verification routine.

Other than the message processing (reading and generating encrypted and/or signed messages) Maily would need a keystore with some key management functionality, an option to generate keys, an option to import keys (both public keys and public/private key pairs) from either the files area or from a key server, an option to delete keys and an option to export keys. And an option to read keys from mail into the keystore. They are delivered in a standard format, easy to recognize. And, of course, functionality for the trust management of keys (presenting key-hashes for verification and giving the user the option to trust and/or even sign a key). We should investigate the option to generate and/or store private keys in the secure space provided by smartphones, the "Secure Enclave" in iOS and whatever this thing is called on Android.

Sounds like fun.

Jul 14 '21 18:07 tallinn1960

Thanks for these insights!

Decrypt and Encrypt

Check for Decryption

If I understood you correctly, for checking if a message is encrypted, we have to check different levels:

Check if the Content-Type is multipart/encrypted or application/pgp-encrypted, if not:
Check if the message contains any application/pgp-encrypted part, if not:
Check if any text/plainortext/htmlmessage part contains both ---BEGIN PGP MESSAGE---and---END PGP MESSAGE---` delimiters.

Decrypt a Message

Identify any part needing decryption (see above) and if PGP can be determined as its decryption mechanism, move it over to the PgpDecryptor (or something like this). This library needs to know the private key, of course.

Encrypt a Message

Generate a symmetric key, encrypt the full message with it, then encrypt the symmetric key with the public key of all recipients and generate the final message - I hope that's pretty much handled by the existing OpenPGP package.

Signing

Check for Signature

That's easier, right? Just check if this is a multipart/signed message with two parts, the first being the original MIME message in "canonical format" (whatever that is) and the second being a application/pgp-signature part.

Check a Signature

Depending on the hashing algorithm re-generate the signature based on the public key of the sender and compare with the signature in the application/pgp-signature part

Sign a Message

Generate the final message
Convert to it's canonical form
Calculate the signature using the private key and generate a new message based on the canonical form message and the signature as well as the public meta data

Did I get this somewhat right? What did I miss? I will check https://datatracker.ietf.org/doc/html/rfc3156 and try to understand this whole thing better.

Thanks, Robert

Jul 14 '21 18:07 robert-virkus

Forgot to mention: Maily already uses the flutter_secure_storage plugin, so it can safely store key data.

Jul 14 '21 18:07 robert-virkus

So back from weekend and a day of work with keycloak (an openid/oauth service provider and IDM) and sw we are writing for it.

Thunderbird is full standard compliant with PGP/MIME and as I feared Outlook produces PGP INLINE.

Your understanding is quite accurate. Here is an attempt of a more detailed description of what needs to be done:

An encrypted full PGP/MIME compliant message will always have multipart/encrypted as the main MIME message part. Evolution on Linux screws this up by using multipart/mixed instead. However, what follows then is identical: two parts (if you count the part for non-MIME-compliant mail readers, three) with the first one being of type application/pgp-encrypted (it is a safe indicator for a PGP/MIME-encrypted message, but contains only a version number and can be discarded for now, as the version number will be 1), the second one containing the message, possibly base64-encoded, which yields an -----BEGIN PGP MESSAGE-----/-----END PGP MESSAGE----- blob. This must be fed to flutter openpgp for decryption together with the private key(s) of the receiver. The decryption routine will return a mime-message, possibly of content-type: multipart/signed.

Lets ditch PGP-INLINE for the moment (I need to look deeper into that) and look at encryption. No, you don't need to generate any cryptographic material outside of flutters openpgp. The hybrid encryption is as a whole generated within that code. However, I still need to look at how to provide the public keys of all receivers to the encryption routine. The openpgp api may need some tweaking as it looks like one can only provide one public key for encryption which is bullocks. In general a mail is encrypted for all receivers and the sender in one pass, all get the same encrypted copy. The input message would be the mime-message with all relevant headers (content-type, encoding and such). The encryption route would yield the -----BEGIN PGP MESSAGE-----/-----END PGP MESSAGE----- blob, which than could be put into a PGP/MIME-Message.

Signing: the text input for signature will be the full mime-message with all mime-headers. It must be transformed into "canonical" MIME-format, whatever that is beyond line-endings needing to be CR/LF. That byte array will be signed by openpgp, which generates a detached signature as a -----BEGIN PGP SIGNATURE-----/-----END PGP SIGNATURE----- blob, which will make the second part of the multipart signed message to be constructed, the input for the signature being the first part.

Signature verification: both parts of the multipart/signed MIME-message are extracted, the first part converted into canonical format if it isn't already, both parts are the given to openpgp for verification. The verification result is just true/false, indicating a valid/invalid signature. Trust considerations beyond the mathematical signature verification are beyond the scope of openpgp. A pgp keystore has trust settings for public keys obtained from elsewhere managed by the owner of the keystore.

The keystore that I mentioned earlier must deliver public and private keys queried by e-mail-address. For encryption the needed public keys are those of the recipients (including the sender), for signature verification it is the e-mail address of the sender.

The "blobs" that pgp processes are all ascii-armored and need no further encoding in general. However, some mail-clients encode them base64 nevertheless.

I verified as well that all PGP-supporting mail clients will generate responses from the cryptographically processed mime messages, not from the original, possibly encrypted and/or signed content. So, no further complications there. The cryptographic operations are a one, optionally two-pass thing in and out. Apart from PGP-INLINE that is. Stay tuned.

I could produce some encrypted and/or signed samples generated by the various clients. You might find them helpful. Where can I upload those? And I would need a public key of yours for PGP so you will be able to decrypt things in those mails.

Jul 19 '21 18:07 tallinn1960

Sorry for the late answer, I was busy with some iCalendar work.

Having samples by various clients would be super-helpful. I guess the easiest way is just to add them here to this issue.

Here's my public key:

Aug 03 '21 09:08 robert-virkus

Now I am to apologize, but it was a busy summer.

Using your public key I’ve send you a variety of PGP mails generated by GPG Mail/MacOS - they should be pretty standard and examples of what to expect of other clients (aside Outlook). I am currently in the process to setup E-Mail Clients (Evolution and Thunderbird on Linux, Outlook with gpg4win/gpgo plugin on Windows) to generate some more. Interestingly I found that Thunderbird supports encryption of mail subjects. No one else does as far as I know. But it is a valuable function.

I looked at the openpgp package on pub.dev and found that it supports encryption for mutiple recpients. The public key material for all recipients is provided as concatened text of the ascii-armored public keys, each embodied within a BEGIN PUBLIC KEY/END PUBLIC KEY block. So the application of the package to the mails should be straightforward.

While setting up a test account for Maily I found some flaws in the „other email“ setup. I created a pull request with a fix for this.

Am 03.08.2021 um 11:21 schrieb Robert Virkus @.***>:

Sorry for the late answer, I was busy with some iCalendar work.

Having samples by various clients would be super-helpful. I guess the easiest way is just to add them here to this issue.

Here's my public key:

-----BEGIN PGP PUBLIC KEY BLOCK-----

-----END PGP PUBLIC KEY BLOCK----- — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Enough-Software/enough_mail/issues/89#issuecomment-891684157, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHH2PXZNXGD5EZMBXAJMEGTT26YH5ANCNFSM4R7YE3XA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

Nov 08 '21 18:11 tallinn1960

So eventually I found the time to start working on pgp integration.

As this requires the open_pgp flutter plugin (there seems to be no pure dart implementation of openpgp) it does not fit into enough_mail unfortunately but needs to go into enough_mail_app. Find a branch "pgp" on my GitHub-fork of enough_mail_app with some testing/sample code for pgp decryption and signature verify.

Sep 03 '22 12:09 tallinn1960

Wow, this is so very, very cool!

It's kinda sad that there is no pure dart open-pgp library available I guess that OpenPGP requires Flutter since it relies on the FFI interface.

So what would be the next step from your point of view?

Sep 04 '22 08:09 robert-virkus

I will work on composition of PGP/Mime messages next.

What I would like you to do is to propose a place where the test/sample code should be put to work within enough_mail_app. Due the nature of E2E encryption, decryption of a received message is a temporary thing: i.e. once a MimeMessage is received, if it happens to be encrypted, a temporary MimeMessage needs to be constructed from it by encryption(and possible signature verification) for display purposes. The result should be (possibly safely by zeroing the memory, however, I would not know how to do that in Dart) discarded once the user closes the message.

The other thing I worked on today is importing key files into Maily. All OS provides mechanisms to „send“ a file of a certain type to an app. In the PGP case that would be *.asc files containing private und public keys. I managed to configure a Flutter app to launch or activate when the device recieves such a file (like by Airdrop). However, I do not know how to hand that over to the Flutter/dart code. But my idea of a minimum viable product is Maily taking public and private keys send to it by OS mechanisms (like receiving by Airdrop or Beam or reading from a cloud service by the Files app). Key file distribution of public and private keys to all devices of a user is essential for all practical purposes, as there should be only one key file per mail-account used on all devices with access to the mail account.

Am 04.09.2022 um 10:26 schrieb Robert Virkus @.***>:

Wow, this is so very, very cool!

It's kinda sad that there is no pure dart open-pgp library available I guess that OpenPGP https://pub.dev/packages/openpgp requires Flutter since it relies on the FFI interface.

So what would be the next step from your point of view?

— Reply to this email directly, view it on GitHub https://github.com/Enough-Software/enough_mail/issues/89#issuecomment-1236287725, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHH2PX2SIX3NEDQE5K2OP2DV4RMLHANCNFSM4R7YE3XA. You are receiving this because you commented.

Sep 04 '22 16:09 tallinn1960

So, after I eventually managed to get an uri send by the OS into flutter code and learned about enabling Flutter Deep Linking that way (which is off by default) I looked into the openpgp plugin what information it gives from the key received. And it seems to be that all the information needed can't be obtained through the implemented openpgp api. The user-id of a key containing the e-mail address of the user is missing.

Now I am contemplating about creating another openpgp plugin, which would use Kotlin Native Multiplatform (which can create macOS/iOS frameworks and Android code) and the Bouncycastle Java OpenPGP implementation. Stay tuned. It may be that the maintainer of openpgp beats me to it and updates the openpgp api. I created an issue there.

But so far this bad. Whereas a received mail can be decrypted and verified without having knowledge about the keys, to confirm that the email address of the sender matches the user-id in the signature verification key is essential for the security. And upon sending an encrypted e-mail one has to choose the keys matching the e-mail addresses involved.

The only solution possible right now would be for the Maily user adding the "missing" email information (as it is not missing but unaccessible) upon receiving a key. That is cumbersome and weakens security, however, the key ids and hashes still provide some means of key verification. And one can expect the user not to fool himself by importing private keys with wrong e-mail-addresses as this would break all his signatures. He just needs to be careful if he has more than one private key (for more than one e-mail address, I have a lot) not to mix things up.

I will create a dialog for key import that presents the verification data (key id short, long, hash) and allows the user to add an e-mail address to the key. And a password for private key decryption, if needed. The key would then be added to a persistent table with e-mail address, key data, and optional the password for decryption. This information eventually will be encrypted by hardware, I know how to do that on iOS, and colleagues know how to do that on Android. This will be another, but small, plugin.

That dialog won't be linked to any other screen of Maily for the moment, as it will be called directly via onGenerateRoute. I will push it on top of any screen displayed at that moment and pop it, once the key is stored. It may be later reused to check and edit key metadata. We just need to find a place for a button directing the user to the key management screens.

Sep 07 '22 20:09 tallinn1960

enough_mail enough_mail copied to clipboard

support PGP/MIME format?

Decrypt and Encrypt

Signing

enough_mail
enough_mail copied to clipboard