"ValueError: too many values to unpack" when syncing from Exchange 2007

[FelixSchwarz]

General informations

• system/distribution (with version): CentOS 7 (x68_64)
• offlineimap version (offlineimap -V): latest git master (c8831608, v7.0.14)
• Python version: python-2.7.5-48.el7.x86_64
• CLI options: -c .offlineimaprc --dry-run --info -d imap

…
Server welcome string: * OK The Microsoft Exchange IMAP4 service is ready.
Server capabilities: ('IMAP4', 'IMAP4REV1', 'AUTH=NTLM', 'AUTH=GSSAPI', 'AUTH=PLAIN', 'IDLE', 'NAMESPACE', 'LITERAL+')

ERROR: could not correctly parse server response; got: (\HasNoChildren) "/" INBOX/Von	Betreff	Erhalten	Gr&APYA3w-e	Kategorien	
Traceback (most recent call last):
  File "…/offlineimap.py", line 26, in <module>
    oi.run()
  File "…/offlineimap/init.py", line 80, in run
    self.__serverdiagnostics(options)
  File "…/offlineimap/init.py", line 511, in __serverdiagnostics
    account.serverdiagnostics()
  File "…/offlineimap/accounts.py", line 192, in serverdiagnostics
    self.ui.serverdiagnostics(remote_repo, 'Remote')
  File "…/offlineimap/ui/UIBase.py", line 473, in serverdiagnostics
    folders = repository.getfolders()
  File "…/offlineimap/repository/IMAP.py", line 475, in getfolders
    flags, delim, name = imaputil.imapsplit(s)
ValueError: too many values to unpack

The problem is that .imapsplit() returns multiple names (splitted at \t) so obviously just three variables don't cut it.

As a workaround I just skip the (one) line which causes this:

            try:
                parts = imaputil.imapsplit(s)
                if len(parts) > 3:
                    self.ui.error("unexpected list result %r (name=%r ?)" % (s, parts[2:]))
                    continue
                flags, delim, name = parts
            except ValueError:

[nicolas33]

Care to send a patch?

[FelixSchwarz]

But what is the right behavior here? I mean sure, just skipping the folder is better than crashing.

However I don't really know much about IMAP. Is Exchange telling me here that there is a folder "Von Betreff Erhalten Größe Kategorie" below INBOX? Or should that be 5 folders? Also I think this should be added to the unit tests. Any pointers which place to modify?

[chris001]

"Von Betreff Erhalten Größe Kategorie" = "From Subject Obtained Size Category" This appears to be the localized translation of the important columns "From Subject Received Size Category" which Exchange 2007 has added to the end of the name of the inbox "INBOX" and separated with a delimiter "/".

[nicolas33]

Some information was missing in the initial report. Hence, my first reply.

But what is the right behavior here? I mean sure, just skipping the folder is better than crashing.

Yes.

However I don't really know much about IMAP. Is Exchange telling me here that there is a folder "Von Betreff Erhalten Größe Kategorie" below INBOX?

Honestly, I have NO idea what the Exchange server is responding. What you get is the response for the LIST (or SUB) command. I can't tell since you did not provided the conf as requested.

Both commands should return folder names in the format described by the RFC.

Providing us the full response of the server would help to understand what's going on.

[nicolas33]

What's the version of the Exchange server? Is this a known issue about Microsoft's product?

[chris001]

This crash appears to be related to improperly handling SMTPUTF8 internationalized messages. https://tools.ietf.org/html/rfc6857

Post-Delivery Message Downgrading for Internationalized Email Messages

Abstract

The Email Address Internationalization (SMTPUTF8) extension to SMTP allows Unicode characters encoded in UTF-8 and outside the ASCII repertoire in mail header fields. Upgraded POP and IMAP servers support internationalized messages. If a POP or IMAP client does not support Email Address Internationalization, a POP or IMAP server cannot deliver internationalized messages to the client and cannot remove the message. To avoid that situation, this document describes a mechanism for converting internationalized messages into the traditional message format. As part of the conversion process, message elements that require internationalized treatment are recoded or removed, and receivers are able to recognize that they received messages containing such elements, even if they cannot process the internationalized elements.

[nicolas33]

First, I don't get why imaplib2 is returning those values instead of the folder names.

[FelixSchwarz]

Thank you very much for your input. I'll need some days until I have free slot to investigate the issue more thoroughly (especially as I stumbled over another issue when calling the script a second time).