Improve null MX recommendation in verdict based on SPF value

[mxsasha]

Building #905, the verdict "you have no mail servers, you should probably have null MX" (info) should only occur when SPF is -all (and no others), i.e. only if SPF confirms no mail is sent from this domain. For any other SPF, the result should be greyed out and verdict should not mention null MX

[mxsasha]

I don't remember the discussion, but while this makes sense to me, it does mean some changes in our DNS lookups:

The status is set in three places, as this affects three test types: https://github.com/internetstandards/Internet.nl/blob/ebfebc5217ecb5df3f7a11c22427442f25415db1/checks/tasks/dnssec.py#L186-L187 https://github.com/internetstandards/Internet.nl/blob/ebfebc5217ecb5df3f7a11c22427442f25415db1/checks/tasks/ipv6.py#L237-L238 https://github.com/internetstandards/Internet.nl/blob/ebfebc5217ecb5df3f7a11c22427442f25415db1/checks/tasks/tls.py#L1100-L1101

The decision is made here, which I think happens once, before the different celery tasks are spawned. https://github.com/internetstandards/Internet.nl/blob/ebfebc5217ecb5df3f7a11c22427442f25415db1/checks/tasks/shared.py#L135

At the decision time, we don't have the SPF records. That's done much later, in a specific celery task. We don't have communication between our checks, so the only way is to make the SPF lookup, parsing and evaluation part of do_mail_get_servers. It's a bit sloppy, because some of that work is repeated later, but it's the only way to support this.

[mxsasha]

This affects only domains with zero MX records. The old behaviour is:

• If there is A or AAAA, status is no_null_mx
• Otherwise, status is no_mx

The new behaviour is:

• If there is A or AAAA, and SPF's only term is -all, status is no_null_mx
• Otherwise, status is no_mx

Where no_mx is a neutral opinion, and no_null_mx is a "you should probably be setting null mx" opinion. The content for these labels will probably need to be updated, note that they exist in multiple tests.

[mxsasha]

Change is merged, content update pending, see comment above

[baknu]

Have been working on content for this one. However, it is quite a puzzle.

Made adjustments for now to /test/mailtls/no-null-mx/description_en.md and to test/mailtls/no-mx/description_en.md. Still have to translate these in Dutch and maybe also need to adjust the "summary"-files in these directories. Furthermore, the respective files in /detail/mail/tls/starttls-exists/verdict, /detail/mail/ipv6/mx-AAAA/verdict and /detail/mail/dnssec/mx-exists/verdict probably need adjustments. Btw note that the naming of some of the respective text labels in the these directories is not consistent atm ("no-mx"is called "other-2", "other" and "no-mailservers" there).

Anyway, as I worked on the content, I began to doubt the change a bit.

If there is no MX in combination with A/AAAA and SPF's only term is -all, I believe the verdict indeed should be to recommend configuring Null MX (so status no_null_mx).

However, if there is no MX in combination with A/AAAA and no SPF with -all only (the latter means either no SPF record at all or an SPF record with other terms than -all only), I believe the "no-mx" verdict text should be something like:

The SPF record on your domain indicates that your domain may be used for sending mail. However, your domain does not have a receiving mail server (MX), which could hinder deliverability of your sent mails because not having an MX may be seen by receivers as an indicator for spam. Therefore if you really want to send mail from this domain, configure an MX. However, if you do not want to send mail from this domain, configure an SPF record with the -all term only and a "Null MX" record.

Background:

• "M3AAWG Sending Domains Best Common Practices":

Domains used to send emails should have a proper MX record pointing to a functional email server.

• See also the article "How PTR Records, A Records, and MX Records can Affect Email Server Deliverability" mentioned in my earlier remark https://github.com/internetstandards/Internet.nl/issues/905#issuecomment-1484081359

@mxsasha @bwbroersma Wondering how you see this. Thanks.

[mxsasha]

@mxsasha @bwbroersma Wondering how you see this. Thanks.

I agree, this is a good clarification.

[baknu]

Done. Changes in:

• /test/mailtls/no-null-mx/
• /test/mailtls/no-mx/
• /detail/mail/tls/starttls-exists/verdict
• /detail/mail/ipv6/mx-AAAA/verdict
• /detail/mail/dnssec/mx-exists/verdict