E-Mails fetched by fetchmail trigger enough rspamd rules to be marked as spam

[lhw]

I currently fetch some emails from GMail which get filtered through rspamd and due to the nature of how the headers look almost all of them get marked as spam or straight up rejected. Here is a list of applied rules directly related to fetchmail for some common message:

VIOLATED_DIRECT_SPF (3.5)
HFILTER_HELO_5 (3)
GOOGLE_FORWARDING_MID_MISSING (2.5)
FORGED_RECIPIENTS (2)
R_SPF_FAIL (1)
BAD_REP_POLICIES (0.1)
RCVD_NO_TLS_LAST (0.1)

which adds up to 12.5. With the default mark as spam being 15 anything else will set it off.

[rwdj]

Interesting. I think I have the same issue.
How can I just disable all spam rules or rspamd? Editing the rules would be nice, which I assume is what this ticket is asking? It doesn’t really say what it wants. Anyways, I want to completely disable it. Accept all spam. No fetched message left behind.

[ofthesun9]

I see two issues here:

1. We have a setting available in the admin panel to enable/disable the antispam (per user). This is what @rwdj would be looking for... But emails have still to pass the initial scan.. see faq
2. You receive emails that have such a high spam score that they are rejected.: You might go the admin panel, from the admin panel then visit the antispam page: Under History, you will see the score of the rejected messages, Under Configuration, you can increase the threshold value for spam rejection. (Default is 15)

[rwdj]

Holy crap. Over 58% of my emails were rejected! Thank you. Setting ANTISPAM=none in mailu.env disabled my.domain/admin/antispam, although the service was still running. This should be enough for me.

[rwdj]

Oh, I did disable the antispam for the user, but it still rejected emails. Disabling antispam for the user was the first thing I did, so it's the initial scan that did it, as you said.

[lhw]

I did not actually find a solution for this. In the end I changed most of my accounts to send mails directly to mailu. Anything received via fetchmail is basically automatically spam. I'd say fetchmail is unsuable the way it is configured right now.

[rwdj]

Going to my.domain/admin/antispam and raising the values by two orders of magnitude worked for me. At least, for disabling it. But the lack of control over the rules seems to be a problem. Although, I think it uses an ML algorithm that can be trained, so it might not be straightforward to change that. Someone would have to give more clarity. And the defaults are definitely not good.

[aksdb]

I guess that explains why mailcow doesn't pipe their fetched mails through the MTA but directly loads them via IMAP. However I actually want my mails to be scanned, even when I fetch them from other mailboxes :-/

Forged Recipient can probably be dealt with by registering the mail adresses that you fetch from, so rspamd has a chance to know which adress is fine and which is not. The other ones though ... could be hard. Although I guess we are not the only ones using rspamd in such a constellation.

[aksdb]

Just an idea:

rspamd works with different config sections that contain certain matchers and apply certain rules, right? So it should work with the following "changes":

• The "Fetched Account" page is extended so you can define for each account, which email adresses it is expected to receive (this could, if empty, default to the "username", which often equals the one and only email adress anyway).
  • Preferably, wildcards should be allowed (since, for example google allows stuff like yourmail+<something>@gmail.com); but this can be considered "additionaly feature" :grin:
• For every "Fetched Account", a separate rspamd config section is created that matches the fetched mail adresses.
  • The mail should therefore no longer be considered forged; since rspamd now should consider this address as kind of an alias.
  • The other problematic rules (like SPF check) can be changed in weight (to 0) within these sections. That way they would still work properly for "normal" emails (received via the MTA; not fetched).

Unfortunately fetchmail doesn't support adding custom headers to fetched mails, otherwise it would be a bit easier matching fetched mails.

[Nebukadneza]

Hi There,

The Mailu-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using Mailu, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a reaction on this post (use the :smiley: icon to the top-right).

• 👍️ if you need this to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
• 🎉 if you find it a nice bonus, but no deal-breaker
• 🚀 if you want to work on it yourself! We want to keep this voting open for 2 weeks from now, so please help out!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Legogris]

@stale keep

[parisni]

I don't understand yet how the fetchmail process injects mails into dovecot and how rspamd filter them but apparently dovecot provides a way for external service to do so (the /dovecot/deliver binary). This apparently not the way the current implementation does. see the conversation here for details any explanation on the current implementation welcome

[aksdb]

@parisni When you do this all on a local machine, this is possible. But in mailu those a separate containers. Therefore the container that runs fetchmail can't call anything related to dovecot - at least not via filesystem.

[parisni]

@aksdb good catch. So how does fetchmail deal ? it is able to write mails directly in the dovecot folders ?

[Diman0]

It should be possible to add a multimap rule to rspamd to automatically accept emails from fetchmail. https://mailu.io/1.9/antispam.html#how-can-i-block-emails-from-a-domain Here is documented how to create a file with domains to blacklist using a multimap rule.

Perhaps the IP address could be used to accept the email? This would involve assigning a static ip to the fetchmail service:

#override.d/multimap.conf
#Tip: Each setting must be closed with a semi-colon ';'.
local_wl_ip {
  type = "ip";
  filter = "email:domain";
  map = "/etc/rspamd/override.d/whitelist_ip.inc";
  score = 0;
  description = "IP address of client is on IP whitelist";
  group = "local_bl";
  action = "accept";
}

In the overrides folder for rspamd (see the previous link), you create the file whitelist_ip.inc with the IP address of fetchmail. Now all emails from fetchmail are automatically accepted by rspamd.

All possible types are listed here https://rspamd.com/doc/modules/multimap.html#map-types. I think matching on IP is the best. I suspect other types can be forged.

[Diman0]

rspamd has been updated that fetchmail connects via the local network. This disables multiple tests in rspamd. This should prevent messages from being marked as spam.

For a permanent fix it should be configurable via the admin interface under user settings whether

• Fetchmail pulled email must go through the spam filter; or
• Fetchmail pulled email must skip the spam filter. Some users actually want all pulled email to be checked for spam. This should be a per user configuration on the settings page.

So we would need to change:

• Changes to the frontend (new knob on the Settings page)
• Changes to the database (everything configured in admin is stored in the database, that includes the new setting)
  • This also includes a database migration script to add a new column that stores the new setting.
• Modified internal endpoint (used by fetchmail for retrieving config from admin)
• Changes to fetchmail for retrieving config (polling time & what delivery port to use)

[nextgens]

@parisni @aksdb The solution is to use LMTP ; Check #2529 out and please report back whether it works for you