Configure Dovecot to automatically delete old spam messages in Jammy

[myfirstnameispaul]

In reviewing adding a feature to Dovecot to automatically delete old messages in the spam folder, I discovered this is very easy to do using the autoexpunge mailbox setting in 15-mailboxes.conf:

  mailbox Spam {
    special_use = \Junk
    auto = subscribe
    autoexpunge = 30d
  }

I took the 30 days time as an example since it is what Google uses by default.

However, maybe it's not actually that easy:

namespace/mailbox/autoexpunge

Default: 0

Values: Time

New in version v2.2.20.

Expunge all mails in this mailbox whose saved-timestamp is older than this value.

For IMAP and POP3 this happens after the client is already disconnected.

For LMTP this happens when the user’s mail delivery is finished. Note that in case there are multiple recipients, autoexpunging is done only for some of the recipients to prevent delays with the mail delivery: The last recipient user is autoexpunged first. Next, the first recipient user is autoexpunged (because the first user’s mail was kept open in case it could be directly copied to the other users). None of the middle recipient users are autoexpunged.

mailbox_list_index = yes is highly recommended when using this setting, as it avoids actually opening the mailbox to see if anything needs to be expunged.

mail_always_cache_fields = date.save is also recommended when using this setting with sdbox or Maildir, as it avoids using stat() to find out the mail’s saved-timestamp. With mdbox and obox formats this isn’t necessary, since the saved-timestamp is always available.

Dovecot documentation for the recommended mailbox_list_index and mail_always_cache_fields when using autoexpunge:

---

mailbox_list_index

• Default: yes
  
• Values: Boolean

Dovecot indexes live at the root of user’s mailbox storage, and allows quick lookup of mailbox status instead of needing to open all mailbox indexes separately.

Enabling this optimizes the server reply to IMAP STATUS commands, which are commonly issued by clients. This also needs to be enabled if you wish to enable the IMAP NOTIFY extension.

But this has been configured differently from the default yes stated in the documentation:

doveconf -a | grep mailbox_list
mailbox_list_index = no

As best I can determine, this is the dovecot-core default for 2.2.x. Based on their changelog, it does appear it was a relatively new feature and somewhat unstable, but it looks like it was patched several times and was set to default yes as of 2.3.x (the version that is in Jammy should be printed with doveconf -a | grep mailbox_list). I can find no other reason not to use yes in Bionic.

For your reading enjoyment, there is further documentation on the Mail index file format.

---

mail_always_cache_fields

• Default: <empty>
• Values: String

The fields specified here are always added to cache when saving mails, even if the client never accesses these fields.

See Mail Cache Settings for details and for the list of fields.

See also

mail_cache_fields

There isn't much documentation on mail_always_cache_fields, but as best I can tell the configuration is placed in 10-mail.conf.

---

There seems to be a performance improvement by setting mailbox_list_index_very_dirty_syncs = yes:

mailbox_list_index_very_dirty_syncs

Default: no

Values: Boolean

If enabled, assume that the mailbox list index is fully updated so that stat() will not be run for mailbox files/directories.

Note that the default in the Dovecot GitHub repo mirror and in Jammy appears to be yes, despite the documentation.

After reviewing all of this, I'd like to recommend setting the following configuration changes in mail-dovecot.sh in Jammy:

# Set the location where we'll store user mailboxes. '%d' is the domain name and '%n' is the
# username part of the user's email address. We'll ensure that no bad domains or email addresses
# are created within the management daemon.
# Sets mailbox_list_index to predictably support autoexpunge.
# Sets  mailbox_list_index_very_dirty_syncs for improved performance and the recommended
# mail_always_cached_fields to avoid finding the mail's saved timestamp.
tools/editconf.py /etc/dovecot/conf.d/10-mail.conf \
	mail_location=maildir:$STORAGE_ROOT/mail/mailboxes/%d/%n \
	mail_privileged_group=mail \
	first_valid_uid=0
        mailbox_list_index = yes
        mailbox_list_index_very_dirty_syncs = yes
        mail_always_cache_fields = date.save

# Add autoexpunge to automatically remove messages in Spam and Trash folders 
# older than 30 days
tools/editconf.py /etc/dovecot/conf.d/10-mail.conf \15-mailboxes.conf
	mailbox Spam {
                special_use = \Junk
                auto = subscribe
                autoexpunge = 30d
        }
        mailbox Trash {
                special_use = \Trash
                auto = subscribe
                autoexpunge = 30d
       }

Note that the configuration settings other than autoexpunge and mail_always_cache_fields are the defaults already in the file, I'm just not totally clear on how the syntax of editconf.py is supposed to work.

The additional Trash configuration could be considered to close #617, and the 30d is the same as Google policy on messages in Trash.

I can't test this right now as I don't have Jammy available at the moment, but as I understand it this will make the server very slow immediately after upgrade since dovecot will have to build the index files, but this is a new default setting so it will be a part of the upgrade.

I also discovered the below setting, which I think is specifically for maildir and outside of the mailbox_list_index:

maildir_very_dirty_syncs

Default: no

Values: Boolean

If enabled (yes), Dovecot is assumed to be the only MUA that accesses Maildir directly, so the cur/ directory is scanned only when its mtime changes unexpectedly or when the mail cannot otherwise be found.

If enabled and another process (or a Dovecot process which doesn’t update index files) does changes to cur/ while the mailbox is simultaneously being modified by Dovecot, Dovecot may not notice those external changes. It is still safe to deliver new mails to new/ using non-Dovecot software (except with mailbox_list_index = yes, changes aren’t noticed outside INBOX).

However, I did not research further than to confirm it is disabled, and I'm not sure if enabling would improve performance beyond mailbox_list_index, and it is not otherwise related to autoexpunge.

[downtownallday]

FWIW, I'm not in favor of this change. Rather, I would prefer that the documentation be updated and/or an "optional" script added to the source tree that shows how to do this as a cron job using doveadm expunge and doveadm purge.