imaplib.py SEARCH => got more than 10000 bytes

[damianmoore]

Hi, I just tried to run sync-engine via this Docker image (haven't had a chance to clone and do my own installation yet). I created the account with bin/inbox-auth filling in details for my personal IMAP server running Dovecot and that worked.

I went on to start setting up the N1 client through config.json but it kept falling through to the Nylas account creation form, no-matter what I entered. I went back to the server and saw that there were errors syncing the account.

Traceback (most recent call last):
      File \"/opt/sync-engine/inbox/util/concurrency.py\", line 46, in wrapped
    return func(*args, **kwargs)
      File \"/opt/sync-engine/inbox/mailsync/backends/imap/generic.py\", line 313, in initial_sync
    self.initial_sync_impl(crispin_client)
      File \"/opt/sync-engine/inbox/mailsync/backends/imap/generic.py\", line 341, in initial_sync_impl
    remote_uids = crispin_client.all_uids()
      File \"/opt/sync-engine/inbox/crispin.py\", line 532, in all_uids
    fetch_result = self.conn.search(['ALL'])
      File \"/usr/local/lib/python2.7/dist-packages/imapclient/imapclient.py\", line 725, in search
    return self._search(criteria, charset)
      File \"/usr/local/lib/python2.7/dist-packages/imapclient/imapclient.py\", line 749, in _search
    data = self._raw_command_untagged(b'SEARCH', args)
      File \"/usr/local/lib/python2.7/dist-packages/imapclient/imapclient.py\", line 1083, in _raw_command_untagged
    typ, data = self._raw_command(command, args)
      File \"/usr/local/lib/python2.7/dist-packages/imapclient/imapclient.py\", line 1146, in _raw_command
    return self._imap._command_complete(to_unicode(command), tag)
      File \"/usr/lib/python2.7/imaplib.py\", line 913, in _command_complete
    raise self.error('command: %s => %s' % (name, val))
    error: command: SEARCH => got more than 10000 bytes

My Dovecot setup is pretty standard, although I do have many thousands of emails in the account which I've collected over the years.

I found this question which is related. There is also a Python issue and I can see the very latest code in their Mercurial repo has increased this but I think this was merged only about a year ago.

I suspect that if I ran a super modern version of Python the problem would be solved. There are no warnings though about version of Python required. Maybe sync-server could just override the value of _MAXLINE in imaplib.py if it is too restrictive.

[fooishbar]

Same here, so I can't pull any mail via the sync engine. :(

[grinich]

Closing due to stale and can't reproduce. Try the latest Nylas Mail release!

[fooishbar]

@grinich I was able to reproduce it with the 0d607a3c17 megacommit. This will presumably happen until you either move away from imaplib, or apply the workaround the comment suggests. It goes without saying that you will need quite a large mailbox (enough to return >10kb results when searching) in order to reproduce this. And specifically, for my case, it's Dovecot on the other end.

[elartey]

@damianmoore add the following line right below the import imaplib in /usr/local/lib/python2.7/dist-packages/imapclient/imapclient.py

imaplib._MAXLINE = 40000

This resolved the issue for me. Reference: http://stackoverflow.com/questions/25457441/reading-emails-with-imaplib-got-more-than-10000-bytes-error

[elartey]

@damianmoore make sure you make a backup of the file before making the changes. cheers

[damianmoore]

@elartey thanks for the tip but my issue was more that sync-engine should override this itself by default. It's a confusing error that might not even be visible to people trying to run this. Telling people to modify a standard library is not good practice in the long run.