Error handling of invalid date headers produces more errors

[me-and]

General informations

• system/distribution (with version): Debian Bullseye
• offlineimap version (offlineimap -V): offlineimap v7.3.0, imaplib2 v3.05, Python v3.9.2, OpenSSL 1.1.1n 15 Mar 2022
• Python version: 3.9.2
• server name or domain: Gmail
• CLI options: -u basic -o -a <emailaddress> -f '[Gmail]/All Mail'

Configuration file offlineimaprc

[general]
accounts = <emailaddress>
metadata = ~/.cache/offlineimap/.offlineimap

[Account <emailaddress>]
localrepository = Local
remoterepository = Remote

[Repository Local]
type = GmailMaildir
localfolders = ~/.cache/offlineimap/<emailaddress>
nametrans = lambda f: f.replace('&', '&-')
utime_from_header = yes

[Repository Remote]
type = Gmail
maxconnections = 4
remoteuser = <emailaddress>
remotepassfile = ~/.<emailaddress>-offlineimap-password
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
folderfilter = lambda f: f in ('[Gmail]/All Mail', '[Gmail]/Starred', 'Test', '[Gmail]/Bin')
nametrans = lambda f: f.replace('&-', '&')
readonly = False

pythonfile (if any)

None

Logs, error

ERROR: Copying message 68353 [acc: <emailaddress>]
  cannot unpack non-iterable NoneType object
Thread 'Copy message from Remote:[Gmail]/All Mail' terminated with exception:
Traceback (most recent call last):
  File "/usr/share/offlineimap3/offlineimap/folder/Maildir.py", line 384, in savemessage
    date = self.get_message_date(msg, 'Date')
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 740, in get_message_date
    datetuple = parsedate_tz(msg.get(header))
  File "/usr/lib/python3.9/email/message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "/usr/lib/python3.9/email/policy.py", line 163, in header_fetch_parse
    return self.header_factory(name, value)
  File "/usr/lib/python3.9/email/headerregistry.py", line 601, in __call__
    return self[name](name, value)
  File "/usr/lib/python3.9/email/headerregistry.py", line 196, in __new__
    cls.parse(value, kwds)
  File "/usr/lib/python3.9/email/headerregistry.py", line 305, in parse
    value = utils.parsedate_to_datetime(value)
  File "/usr/lib/python3.9/email/utils.py", line 198, in parsedate_to_datetime
    *dtuple, tz = _parsedate_tz(data)
TypeError: cannot unpack non-iterable NoneType object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/share/offlineimap3/offlineimap/threadutil.py", line 146, in run
    Thread.run(self)
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/share/offlineimap3/offlineimap/folder/Gmail.py", line 293, in copymessageto
    super(GmailFolder, self).copymessageto(uid, dstfolder, statusfolder, register)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 807, in copymessageto
    new_uid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/share/offlineimap3/offlineimap/folder/GmailMaildir.py", line 119, in savemessage
    return super(GmailMaildirFolder, self).savemessage(uid, msg,
  File "/usr/share/offlineimap3/offlineimap/folder/Maildir.py", line 392, in savemessage
    datestr = self.get_message_date(msg)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 740, in get_message_date
    datetuple = parsedate_tz(msg.get(header))
  File "/usr/lib/python3.9/email/message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "/usr/lib/python3.9/email/policy.py", line 163, in header_fetch_parse
    return self.header_factory(name, value)
  File "/usr/lib/python3.9/email/headerregistry.py", line 601, in __call__
    return self[name](name, value)
  File "/usr/lib/python3.9/email/headerregistry.py", line 196, in __new__
    cls.parse(value, kwds)
  File "/usr/lib/python3.9/email/headerregistry.py", line 305, in parse
    value = utils.parsedate_to_datetime(value)
  File "/usr/lib/python3.9/email/utils.py", line 198, in parsedate_to_datetime
    *dtuple, tz = _parsedate_tz(data)
TypeError: cannot unpack non-iterable NoneType object

Steps to reproduce the error

Attempt to parse an email with a clearly bogus Date header. In my case, the problematic header based on inspecting the email.message.EmailMessage object with pdb is as below:

(Pdb) p {key: value for key, value in msg._headers if key == 'Date'}
{'Date': '7/17/2009\r\n                                                                                                                                                                                                                                              '}

From digging in the code, when using utime_from_header, if the Date header is unparsable, MaildirFolder.get_message_date can raise an exception. The error handling code for this, copied below, attempts to call through to the same function, though, which inevitably raises exactly the same exception. This means (a) the diagnostics that the ui.warn call is supposed to output don't get printed, and (b) the entire sync fails.

https://github.com/OfflineIMAP/offlineimap3/blob/253f97a3e9476beae837d4d64114fd9698e94498/offlineimap/folder/Maildir.py#L411-L424

[me-and]

From a bit more digging, it looks like part of the issue is that email.message.EmailMessage.__get__ will attempt to parse date headers as DateTime objects; it looks like it's done that since Python v3.3.0 / v2.7.5, per https://github.com/python/cpython/commit/0b6f6c82b51b7071d88f48abb3192bf3dc2a2d24.

That clearly fails when the format is as bogus as this. I think it's probably right that Python raises an exception in this circumstance (although I think "TypeError: cannot unpack non-iterable NoneType object" is not a particularly helpful error…) so I think offlineimap should cope with this without trying to parse the header a second time and thus generating the same exception in what's supposed to be the exception handling code.