Big messages cause an APPEND timeout

[Profpatsch]

• system/distribution (with version): nixos master
• version: offlineimap v7.0.6, imaplib2 v2.55 (bundled), Python v2.7.12
• server: sslin.df.eu

The error from https://github.com/OfflineIMAP/offlineimap/issues/262 still happens.

Steps to reproduce the error

• have a big message and low bandwidth
• try to move the message between folders

In my case I had a message that was 40MB and I wanted to move it from my inbox folder to the archive folder. After some time I always get:

 Establishing connection to SNIP:993 (MailRemote)
 ERROR: Saving msg (<SNIP>) in folder 'INBOX.Archive', repository 'MailRemote' failed (abort). Server responded: command: APPEND => no response after 120.0 secs
Message content was: Received: from [SNIP_IP] (helo=SNIP)
	by SNIP with esmtp (Exim 4.68)
	(envelope-from <SNIP>)...ZWYKNjEwMzA3MQolJUVPRgo=

Looks to me like the transfer takes >120 sec (quite an arbitrary time) and offlineimap doesn’t tell the server it’s still transferring, so the server aborts.

[nicolas33]

I guess you're right. I think offlineimap doesn't split the file into smaller chunks.

I don't know why the server has this requirement, though.

[chris001]

Besides increasing the timeout, it'd be good to look at rewriting this algorithm, possibly add functions to imaplib2, to do an IMAP MOVE command and IMAP COPY command, where appropriate, to save on bandwidth, increase speed and improve reliability. https://tools.ietf.org/html/rfc6851 Documentation from 2013 for IMAP MOVE with semantics on how MOVE is an atomic operation and can be simulated with COPY, set the flag DELETE, and EXPUNGE. https://tools.ietf.org/html/rfc4315 Docs for UIDPLUS with details on UID COPY.

[nicolas33]

@chris001 You're right. Patches are welcome. ,-)

[chris001]

@nicolas33 Thanks :) Instead of patching, how about exploring the possibility of using IMAPClient https://github.com/mjs/imapclient a newer improved code base which is, in the opinion of many, better suited for dealing with email messages at a higher level, than imaplib , it uses imaplib internally, which implements copy() and move(). Certainly worth looking at. Offlineimap shouldn't have to implement error-prone parsing routines to handle the raw text responses from the IMAP server, this job is something for a middle layer, such as IMAPClient, to do solidly. Your thoughts?

[nicolas33]

@chris001 Open a new bug report or start a new mailing list thread to not hijack this report, please.

[chris001]

@nicolas33 New issue opened! #415 @Profpatsch Sorry to hijack your issue!