chatmail
Send MDNs to self
We currently synchronize "seen" status of messages by setting \Seen flag on IMAP and then looking for new \Seen flags using CONDSTORE IMAP extension. This approach has multiple disadvantages:
- It requires that the server supports
CONDSTOREextension. For example Maddy does not support CONDSTORE yet: https://github.com/foxcpp/maddy/issues/727 - It leaks the seen status to the server without any encryption.
- It requires more than just store-and-forward queues and prevents replacing IMAP with simpler protocols like POP3 or UUCP or some HTTP-based API for queue polling.
A simpler approach would be to send MDNs to self when bcc_self (aka multidevice) config is enabled, regardless of whether the message requested and MDN. If MDN was requested and we have MDNs enabled, then also send to the message sender, but MDN to self should be sent regardless of whether read receipts are actually enabled.
sync_seen_flags and CONDSTORE check is better completely removed, maybe after one release. store_seen_flags_on_imap can be kept for unencrypted non-chat messages if it's not much code.
One potential problem with sending MDNs is that it may trigger ratelimits on some providers and count as another recipient. We cannot upload MDNs over IMAP like sync messages because we usually need to send MDN to the original sender as well.
for the record, the 60 emails per minute will likely be problematic with this (and probably is already with normal MDN and sync messages?) also for bots like the PixelSocial bot that needs to distribute messages to users 60 emails per minute is too low, we should really increase this to something like 1000 per minute just to have some boundary without being super conservative
Chatmail relays should monitor the number of ratelimits triggered via logs, if it's too many then we should increase the limits.
Increasing rate limits on relays is a somewhat independent issue, and bots could get a different limit anyway.
Note that classic email providers like Gmail are not a primary concern anymore, and are anyway inferior to relays in several ways.
note2 that adding self address(es) as recipient to MDNs might reduce traffic as no separate imap-seen-flagging operation needs to happen. That Gmail potentially counts this as another recipient is their problem.
iirc MDN as well as sync-messages support combining multiple messages (sent fast ot offline) into one. that may mitigate the concern of too many messages as well
and bots could get a different limit anyway.
there is no way for chatmail operator to know which accounts are bots, would require manual whitelisting (but yeah the bots part it is a bit out of topic here hopefully this wouldn't affect bots because they are single-device most of the time)
It is not clear if we want to remove seen flags because it will break some flows with using delta chat as a reader for classic emails, so the issue is now only about sending MDNs to self.
Another reason to send MDNs to self is that it will trigger push notification so your other devices will know that you have read the message and dismiss the notification: https://support.delta.chat/t/notification-dismissal-sync-like-all-other-messenger-apps/3974
Chatmail servers currently do not send a push notification when a message from self is received, this will also need to be changed.
What about only doing this on chatmail relays? afaict, all the advantages only happen on chatmail relays, and all the disadvantages only on classic mail servers.
keeping double logic / different paths and ways of doing things always complicate things, I don't think this is a problem, non-chatmail users will never send MDNs because they peers don't use Delta Chat or if they are using it with other Delta Chat users they want to disable MDNs anyways because it drains their rate limit
I agree that it's better to have the same logic for both cases. As for rate limits, we can send an MDN to self only for the last seen message in the chat (which will mark all previous messages as noticed) and do this e.g. once per minute, most users don't switch devices so frequently.
