When pushing old mails to Gmail, Gmail shows sync date rather than original date

[tremby]

General informations

• system/distribution (with version): Ubuntu 18.04.4 LTS
• offlineimap version (offlineimap -V): offlineimap v7.1.5, imaplib2 v2.57 (system), Python v2.7.17, OpenSSL 1.1.1 11 Sep 2018
• Python version: 2.7.17
• server name or domain: -
• CLI options: -

Configuration file offlineimaprc

[general]
accounts = test

[Account test]
localrepository = Local
remoterepository = Gmail

[Repository Local]
type = Maildir
localfolders = ~/mail-test
sync_deletes = no
readonly = true

[Repository Gmail]
type = Gmail
remoteuser = **@gmail.com
remotepass = **
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
maxconnections = 3

pythonfile (if any)

none

Logs, error

n/a I think

Steps to reproduce the error

With old mails in my Maildir, I'm running a one-way sync to Gmail.

Once complete, the mail in Gmail is all showing the date and time of the migration, rather than the date and time in the message headers.

There's a thread about this here: https://support.google.com/mail/forum/AAAAK7un8RU0PU1gQuXNi8/?hl=en&gpf=%23!topic%2Fgmail%2F0PU1gQuXNi8 -- but I'm not sure if anything in that is useful.

This is similar or the same as #331 but maybe something changed in Gmail because I see the migration date/time in conversation view as well as in the list view.

That other issue was closed since it was deemed not to be OfflineIMAP's fault, but if I transfer the mail from the original IMAP server to Gmail with another client like Thunderbird the dates appear correctly. So Thunderbird and OfflineIMAP must be doing something differently.

When I click "show original" on a message transferred from OfflineIMAP I see in Gmail's table at the top in the "created on" row something like "Delivered after 367481788 seconds". When I do the same on a message transferred via Thunderbird I see on each one "Delivered after -3600 seconds". Maybe this is a clue...?

I wonder if it's something to do with the interim Maildir storage in my OfflineIMAP process (I'm going from original IMAP server one-way to local Maildir via OfflineIMAP, then changing the configuration to go from now-read-only local Maildir one-way to Gmail). In Thunderbird I'm dragging from one remote to another.

What I really want is to set up two remotes in OfflineIMAP and have it sync between them without it being a two-stage operation, but I don't think OfflineIMAP supports that.

[nicolas33]

Interesting report. Having any other IMAP client doing it as expected is a good point.

[teddybeermaniac]

Hi, I deal with very similar scenario, and when I enabled utime_from_header and filename_use_mail_timestamp on both local repositories everything worked as expected. But I think it requires you to re-download messages from the source IMAP server before syncing them to Gmail. I'm not sure if you have to enable them both everywhere, but don't have time to investigate which combination is enough, because I have to sync a few mailboxes before I loose access the server tomorrow :smile:, but I can experiment with them when I have some time.

[nicolas33]

We should document this.

[tremby]

@teddybeermaniac, I tried this and it did the trick.

I experimented until I found the minimal set of options. All that was needed was utime_from_header = yes and this one is only needed on the local mailbox. I didn't try switching it off for just one of the download/upload stages; I left it on for both.

I left filename_use_mail_timestamp unspecified in all sections of the config, and left utime_from_header unspecified on both remotes.

Thanks!

[nicolas33]

Thank you both for the feedbacks!

[tremby]

Is this not still a bug? Why should I have to use that particular feature to get timestamps to appear correctly when sending the mails up to another server?

I don't know if this really exists, but maybe I'm using a filesystem which doesn't support timestamps on files. (Looks like there are some like the original ext.)

The docs on utime_from_header only seem to suggest it'll affect what's in your local Maildir ("useful if you are doing some processing/finding on your Maildir"); it doesn't suggest it'll read that timestamp again when copying mail from here to a remote, or do anything else with it. Whether or not I'm reading that correctly and what's really happening is the expected behaviour, without utime_from_header switched on aren't timestamps meant to be taken from the headers in the email anyway?

In short, from reading the docs I would have expected exactly the same behaviour with or without utime_from_header being set, except the filesystem timestamps of the emails locally in the Maildir.

[nicolas33]

The current behaviour is to use the filesystem mtime.

One of the reason is that taking the time by other means might fail (not all emails have a 'Date' header and when found it's not always correct). By taking the file mtime we ensure a consistent behaviour for all the emails.

IOW, if you want the INTERNALDATE to be set from the email 'Date' header as found when downloaded the requirement is to enable utime_from_header and have a compatible filesystem before any download.

I agree that the utime_from_header option is not enough documented.

[nicolas33]

Also, it should not be hard to introduce a script to fix the mtime from the 'Date' header in local maildirs (all the hard work is already coded in the internal libraries). We don't have such a script yet but I'd welcome PR in this area.