OpenDMARC
OpenDMARC copied to clipboard
trusteddomainproject
DMARC aggregate report doesn't follow RFC guidelines
I've noticed the aggregate reports from opendmarc-reports v1.3.2 do not follow RFC7489 Appendix C guidelines and schema.
Attachment: Attachment media type invalid (should be "application/gzip" not "application/zip") Attachment filename doesn't follow ABNF (should be "xml.gz", not "zip")
Missing elements: Parent: version PolicyPublishedType: fo DKIMAuthResultType: selector SPFAuthResultType: scope SPFAuthResultType: domain ReportMetadataType: email (missing when not set by client, this should be mandatory). IdentifierType: envelope_from (empty is allowed for messages with a null reverse-path)
It looks like the report generator is based on the pre IETF drafts (2013) as described here.
You are correct, the attachment type and filename/extension need to be adjusted to the current RFC guidelines. I have added this to the items to triage and correct for the next release.
Feel free to cannibalize my change for the report format (from my opendmarc-unafraid fork), they were quite straight forward and are in active use both on my private server and on my employer's servers, so they work. They're not really beautiful though...
I haven't added a solution for the missing items though.
So, right now the reports opendmarc produces (which have been stable (as in, unchanging) for a while) are something people know how to work with. Nobody's come to me and said "we're rejecting your report". Everyone building a business around this as an ecosystem knows how to swallow the reports we produce.
If we make your changes, we could break some expectations, only to break them again later when we gain full RFC compliance.
In order to add the missing items, it requires changes to the c code, the db schema, and the perl.
I'm going to milestone this for a future 1.5.0 release. It's not going to make 1.4.1 or whatever bugfix point release we put out.
This makes absolutely no sense to me. The people using OpenDMARC are not on the receiving end of the generated reports. There are more DMARC-report-generators out there, most of them RFC compliant, and those reports are processed just fine by the 'ecosystem'. Adding missing elements to a report will not lead to rejection. Worst case, the elements aren't processed or used. I'm a developer at a DMARC-report-processing service where we process RFC compliant reports and had to make exception rules for reports based on the RFC draft that OpenDMARC produces.
The missing elements help domain administrators solve their SPF / DKIM / DMARC problems. Missing elements like DKIMAuthResultType: selector, SPFAuthResultType: scope and SPFAuthResultType: domain are trivial in resolving these issues.
In my opinion, RFC compliance should be a #1 priority for OpenDMARC. The RFC is its right to exist.
What you say about exceptions is some of my point. You've had to write exceptions for what opendmarc currently produces. I'm sure others have as well. If we applied FlinkFlonk's improvements (which would bring us closer to compliance by fixing the file type, and the like), you'd need to tweak your exceptions. As would others. If we went full-on over to putting out RFC compliant, it would just work. That's where I'd like to be.
My numbers one and two priorities are getting the CVE's we've got solved, and getting stable code into people's hands.
1.4.0 was pushed out before I joined this project. I've been on it for a week, and I've had commit access, for...< 72 hours.
Patches are welcome, but I'm still getting familiar with the codebase.
The exceptions are not based on the specific generator, but on the reports/elements themselves. So making reports more RFC compliant does not mean we have to change anything. We process reports that violate RFC guidelines by ignoring the fact that the mandatory elements are missing. If the elements are there, we process them as usual. As for the attachment issue, we check for both .zip and .xml.gz attachments and process accordingly. It would be very inefficient to check the generator first, and then process by a set of RFC exception rules per generator.
I agree that CVE's are more important but RFC compliance should not be delayed for compatibility reasons.
To be honest, my changes aren't that big, they are a step nearer to compliance with the RFC but not that important. They are, however, "low-hanging fruit", ie. changes that can be implemented with not much risk, while bringing opendmarc (which is supposed to be the "official" open-source implementation of the RFC) from the draft-status to a more "correct" implementation.
As for reports not being read, that is a) up to the domain owner, and b) dependant on so much more than if the report is compressed with zip or gzip. I'd say now, with the RFC having come out of draft in 2015, there are more implementations using xml.gz instead of zip, just because those implementations were made after the RFC came out of draft.
What I can see, however, is that all the "big ones" - google, amazon, even mail.ru - in my dmarc-report-feed are sending gzip-compressed xml-files. And as for the reports my server is sending out, the ones with non-functioning rua-addresses are a much bigger problem, ie. none of the reports came back with claims of unreadability.
I agree with @flinkflonk. The compression type was changed in the pre-IETF draft published January 4, 2013 (over 8 years ago).
Have a look at my blog about DMARC reports here. The top 5 DMARC report suppliers (in order: google.com, Yahoo! Inc., emailsrvr.com, zoho.com, AMAZON-SES) all produce reports with issues, which is disappointing. OpenDMARC could and should set a good example here.