IETF lists do From-munging, including within multipart/signed MIME part contents, in violation of RFC 1847 §2.1

[dkg]

Message-ID [email protected] appears at the mail archive at: https://mailarchive.ietf.org/arch/msg/openpgp-dt/6mqbfSI5OVmVtF9R6i2zSSjYx4M/

This is the original message in message/rfc822 form.

Note that From a simpler in the original is rendered as >From a simpler in the archive. This appears to be From-munging.

In this case, it is also breaking §2.1 of RFC 1847 which reads:

Intermediate message transfer agents must not alter the content of a multipart/signed in any way, including, but not limited to, changing the content transfer encoding of the body part or any of its encapsulated body parts.

The copy of the message i received as an e-mail message (which has the Archived-At header field) also has been From-munged. This may be an issue that happens before the mailarchive gets to touch the message -- i can't see deeply enough into the IETF mailing list infrastructure to tell -- but i'm quite sure that the message is not From-munged when it is injected into the IETF's SMTP server. If you think the problem is happening before the message reaches the mailarchive, feel free to redirect this bug report somewhere else.

[dkg]

To be clear, the reason that this is a problem is that the cryptographic signature on the message is broken.

[dkg]

I just tested another mail path to the archive, using the [email protected] mailing list from protonmail, and confirmed that from-munging is happening there too:

https://mailarchive.ietf.org/arch/msg/ietf-smtp/C1r1ssVW21z3lpSJNV2swPy15z4/

This second message was unsigned, so the from-munging it's not a violation of RFC 1847, but it would still be nice to avoid the from-munging, which shouldn't be necessary in a modern mail setup.

[jrlevine]

From munging is an ancient hack dating from the 1970s because the Unix mbox format uses "From " as a separator line. That's a lot older than any cryptographic software so surely this is an issue people have dealt with before. I expect something in mailman is helpfully adding the < when saving a message into an archive file but I don't know the internals well enough to know where. If the archive is in mbox format with multiple messages per file, the munging is literally a feature, not a bug.

[dkg]

Yep, i am aware of the history of From munging and its relationship to mbox.

If you have to from-munge because you're storing in mbox format, there are reversible ways to do so (e.g. insert an additional ">" whenever you encounter the regex /^>*From / upon storing, and always remove one > from lines matching /^>+From / upon retrieval). These techniques should leave the message intact. Whether the IETF's mailing list infrastructure needs to store messages in mbox format or not, it is not doing reversible From-munging, so it is literally a bug, and not a feature.

[rjsparks]

The backing store is individual files (maildir format IIRC)

See https://mailarchive.ietf.org/arch/msg/secdir/BDhtxVilF8J0y2eQg4P5a-xJx0E/ for an example of the damage inside the message.

Note that this particular >From is also present when looking at the message through imap.

Also note the lack of the escaping in the source material at https://datatracker.ietf.org/doc/review-ietf-6lo-use-cases-12-secdir-lc-sparks-2022-04-05/.

The damage before it lands in storage is of note:

.../data/archive/secdir> grep -i from 'BDhtxVilF8J0y2eQg4P5a-xJx0E='
From nobody Tue Apr  5 14:05:00 2022
Received: from ietfa.amsl.com (localhost [IPv6:::1])
From: Robert Sparks via Datatracker <[email protected]>
>From the abstract: "The document targets an audience who would like to
Appendix A is neither introduced nor referenced from the body of the document.

Something in the mailchain is doing an unexpected thing that it shouldn't.

[dkg]

thanks for digging into this, @rjsparks (and i'm sorry to see that one of your messages got bit by this too)!

If this issue needs to be reassigned to some other component of IETF infrastructure (before it hits the mail archive) that would be understandable too. If that happens, please let me know where it's being reassigned so i can follow along with the fixing. I'm happy to help test any proposed fix if there's a way for me to do that.

[NUARI19]

Message-ID [email protected] appears at the mail archive at: https://mailarchive.ietf.org/arch/msg/openpgp-dt/6mqbfSI5OVmVtF9R6i2zSSjYx4M/

This is the original message in message/rfc822 form.

Note that From a simpler in the original is rendered as >From a simpler in the archive. This appears to be From-munging.

In this case, it is also breaking §2.1 of RFC 1847 which reads:

Intermediate message transfer agents must not alter the content of a multipart/signed in any way, including, but not limited to, changing the content transfer encoding of the body part or any of its encapsulated body parts.

The copy of the message i received as an e-mail message (which has the Archived-At header field) also has been From-munged. This may be an issue that happens before the mailarchive gets to touch the message -- i can't see deeply enough into the IETF mailing list infrastructure to tell -- but i'm quite sure that the message is not From-munged when it is injected into the IETF's SMTP server. If you think the problem is happening before the message reaches the mailarchive, feel free to redirect this bug report somewhere else.

[email protected]

[rpcross]

It looks like this is happening prior to message archival.

A test message did not get munged: https://mailarchive.ietf.org/arch/msg/testlist/w1Ei5WSEOOJHM_qV3NpYLSmPqmo/

Also, we have raw bytes that are submitted for archival for all recent messages. On inspection, recent examples of munged Froms show the corruption in the submitted data: https://mailarchive.ietf.org/arch/msg/dnsop/UnGoAQ0tYtjttfUrZwWnhWcfg5k/# https://mailarchive.ietf.org/arch/msg/sidrops/Je506Q-PotBu1oc1fCmXlYjXkB8/ https://mailarchive.ietf.org/arch/msg/admin-discuss/FBu8RN_hwPMg5-_bGUjJTXvuv4s/

[rpcross]

On March 6 call, discussed building a container to test the email chain for From munging, might be related to DMARC rewrites.

[rpcross]

This is caused by Postconfirm's (Python2) use of message.as_string() mangling the From