maddy icon indicating copy to clipboard operation
maddy copied to clipboard

Maildir storage backend

Open emersion opened this issue 7 years ago • 17 comments

Probably maildir

emersion avatar Mar 05 '18 15:03 emersion

https://github.com/flashmob/go-guerrilla/tree/master/backends implemented multiple backends using a Backend interface just as you did with your https://github.com/emersion/go-smtp#server

xeoncross avatar Feb 03 '19 02:02 xeoncross

What's up with https://github.com/emersion/go-imap-disk? Can we use it? (of course, we need to finish it first)

foxcpp avatar Mar 23 '19 16:03 foxcpp

We can use it, but it doesn't use maildir, it uses a bolt database. This is fine too, but a maildir backend would probably be higher priority for compatibility (makes migrating easier).

emersion avatar Mar 25 '19 18:03 emersion

https://github.com/luksen/maildir Except that we also need some way to store UIDs and some other meta-data (custom flags, internal date).

foxcpp avatar Mar 25 '19 23:03 foxcpp

Dovecot stores UIDs in a uidlist file. I've been working on a project similar to maddy, on and off (maddy is better!), and implemented a parser for it here: https://gitlab.com/lupine/crockery/tree/master/internal/imap/uidlist

lupine avatar May 11 '19 13:05 lupine

We currently want to implement support for Maildir storage by extending https://github.com/emersion/go-imap-disk which uses BoltDB for all meta-data. Personally, I don't like the idea of supporting other servers internal formats in maddy. Also we can't use your code anyway due to incompatible licenses (crockery's AGPL and maddy's MIT).

foxcpp avatar May 11 '19 15:05 foxcpp

Are the UIDs the only metadata we need to save?

emersion avatar May 12 '19 19:05 emersion

Mailbox-related: UIDVALIDITY, UIDNEXT Message-related: flags, RFC822 size [1], internal date [2], UID

[1] - Probably we would have to store message using native line separators to reduce possible interoperability problems [research needed], but we should still report RFC822 size with CRLF (Is that right?). [2] - Probably FS "modification time"?

foxcpp avatar May 12 '19 20:05 foxcpp

Probably we would have to store message using native line separators to reduce possible interoperability problems

No, MIME says you MUST use CRLF. Using non-CRLF line endings with MIME messages leads to various issues including DKIM signature breakage.

emersion avatar May 12 '19 20:05 emersion

Related discussion: https://dovecot.org/list/dovecot/2013-May/090499.html

The original (djb) definition of Maildir assumed that messages would be written to the maildir with LF line endings, and both MTA and POP server would translate back to CRLF as needed. Dovecot (as a POP server) can deal with messages in either format, and should always return them to clients with CRLF.

It mentions POP, but I think it applies to IMAP too.

foxcpp avatar May 13 '19 06:05 foxcpp

Gah that is annoying.

Also note that:

mail_save_crlf setting controls if mails are saved as CRLF or LF to storage.

emersion avatar May 13 '19 08:05 emersion

Started some work here. https://github.com/foxcpp/go-imap-maildir

Not high priority though, go-imap-sql is working reasonably well.

foxcpp avatar Feb 27 '20 23:02 foxcpp

go-imap-maildy is ready for use. Integration is blocked by #259 since it was written for v2.

foxcpp avatar Aug 03 '20 14:08 foxcpp

Blocked by https://github.com/foxcpp/go-imap-maildir/issues/1 since we need to use go-imap-mess "external update" adapter for maddyctl synchronization.

foxcpp avatar Jan 06 '22 22:01 foxcpp