pywin32 icon indicating copy to clipboard operation
pywin32 copied to clipboard

Datetimes retrieved seem to not have their timezones set correctly

Open ashaw596 opened this issue 3 years ago • 13 comments

They default to the UTC timezone and are thus the wrong time since the dates seem to be in localtime.

ashaw596 avatar Sep 16 '21 01:09 ashaw596

This can be seen when retrieving ModificationTime from mailitems from outlook

ashaw596 avatar Sep 16 '21 01:09 ashaw596

This is because pywin32 doesn't really know what timezone to use due to the lack of timezone support in the standard library. The time should be correctly converted from the local time to UTC, so you should be able to use pytz or pytimezone to convert it back to a local time. If you can demonstrate that not being true then I'll reopen this, but otherwise it's working as intended.

mhammond avatar Sep 16 '21 03:09 mhammond

Sorry the bug is that it's not being correctly converted. It's giving the time as UTC, but it's actually only correct if interpreted in your local time zone

On Wed, Sep 15, 2021, 8:27 PM Mark Hammond @.***> wrote:

This is because pywin32 doesn't really know what timezone to use due to the lack of timezone support in the standard library. The time should be correctly converted from the local time to UTC, so you should be able to use pytz or pytimezone to convert it back to a local time. If you can demonstrate that not being true then I'll reopen this, but otherwise it's working as intended.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mhammond/pywin32/issues/1760#issuecomment-920546656, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPROBUQUCAN6KN2PV6UFZTUCFPZBANCNFSM5EDUNYOA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

ashaw596 avatar Sep 16 '21 03:09 ashaw596

Please consider reopening the issue. @mhammond

ashaw596 avatar Sep 16 '21 03:09 ashaw596

Can you please try and demonstrate the bug via https://github.com/mhammond/pywin32/blob/master/com/win32com/test/testDates.py - it explicitly checks utc and local times etc. It's worth noting that COM dates do not carry timezone into, so local times is really just a convention, so the issue might be outlook. I don't have outlook, so I'm unable to test that.

mhammond avatar Sep 16 '21 04:09 mhammond

Hmm here is the code I'm using rn to demonstrate. Let me take a look at that. It might be just outlook pywin32 API related.

import datetime

from win32com.client import Dispatch

outlook = Dispatch("Outlook.Application")
outlook_mapi = outlook.GetNamespace("MAPI")

mail_item = outlook.CreateItem(0)
mail_item.Close(0)

mail_item.Subject = "test"
mail_item.Save()

print("Modification time(now since we just saved it)")
print(mail_item.LastModificationTime)
print("Datetime now", datetime.datetime.now())
print("UTC Datetime now", datetime.datetime.now().astimezone(datetime.timezone.utc))

output is Modification time(should be now since we just saved it) 2021-09-15 21:19:35.898000+00:00 Datetime now 2021-09-15 21:19:35.917913 UTC Datetime now 2021-09-16 04:19:35.917913+00:00

warning that code will create a message in your outlook draft folder (and requires outlook to be installed)

ashaw596 avatar Sep 16 '21 04:09 ashaw596

It is giving the modification time in the local timezone, but claiming it's UTC which is incorrect.

ashaw596 avatar Sep 16 '21 04:09 ashaw596

Yeah whoops missed that part of you message. Could just be outlook

ashaw596 avatar Sep 16 '21 04:09 ashaw596

Hmm so this test does fail.

    def testNoTimeZone(self):
        self.check(
            datetime(
                year=2000,
                month=12,
                day=25,
                microsecond=500000,
            )
        )

I'm guessing that it doesn't support no time zone currently?

ashaw596 avatar Sep 16 '21 04:09 ashaw596

I assume that fails just because you can't compare dates with timezones and those without? eg: datetime(year=2000, month=12, day=25, tzinfo=TimeZoneInfo.local()) == datetime(year=2000, month=12, day=25) returns False

mhammond avatar Sep 16 '21 04:09 mhammond

I think is not an issue. Here is a vbs script behaving similar:

function utc(var)
  set dateTime = CreateObject("WbemScripting.SWbemDateTime")    
  dateTime.SetVarDate(var)
  r = dateTime.GetVarDate (false)
  set dateTime = Nothing
  utc = r
end function 	

sub print_time(mail_item, stage):
  wscript.echo "** " & stage
  wscript.echo "Modification time : " & mail_item.LastModificationTime
  wscript.echo "Creation time     : " & mail_item.CreationTime
  wscript.echo "Datetime now      : " & FormatDateTime(Now)
  wscript.echo "UTC Datetime now  : " & FormatDateTime(utc(Now))
end sub  

set outlook = CreateObject("Outlook.Application")
set outlook_mapi = outlook.GetNamespace("MAPI")

set mail_item = outlook.CreateItem(0)
print_time mail_item, "Created"
mail_item.Close(0)
print_time mail_item, "Closed [1]"

mail_item.Subject = "test AB"
print_time mail_item, "Subject changed"
mail_item.Save()
print_time mail_item, "Saved"
mail_item.UnRead = True
print_time mail_item, "UnRead"
mail_item.Save()
print_time mail_item, "Saved [2]"
mail_item.Close(0)
print_time mail_item, "Closed [2]" 

set outlook_mapi = nothing

outlook.Quit
set outlook = Nothing

Result:

$ cscript //nologo o.vbs                
** Created                              
Modification time : 1/1/4501            
Creation time     : 1/1/4501            
Datetime now      : 9/16/2021 5:34:37 PM
UTC Datetime now  : 9/16/2021 2:34:37 PM
** Closed [1]                           
Modification time : 9/16/2021 5:34:38 PM
Creation time     : 9/16/2021 5:34:38 PM
Datetime now      : 9/16/2021 5:34:37 PM
UTC Datetime now  : 9/16/2021 2:34:37 PM
** Subject changed                      
Modification time : 9/16/2021 5:34:38 PM
Creation time     : 9/16/2021 5:34:38 PM
Datetime now      : 9/16/2021 5:34:37 PM
UTC Datetime now  : 9/16/2021 2:34:37 PM
** Saved                                
Modification time : 9/16/2021 5:34:38 PM
Creation time     : 9/16/2021 5:34:38 PM
Datetime now      : 9/16/2021 5:34:38 PM
UTC Datetime now  : 9/16/2021 2:34:38 PM
** UnRead                               
Modification time : 9/16/2021 5:34:38 PM
Creation time     : 9/16/2021 5:34:38 PM
Datetime now      : 9/16/2021 5:34:38 PM
UTC Datetime now  : 9/16/2021 2:34:38 PM
** Saved [2]                            
Modification time : 9/16/2021 5:34:38 PM
Creation time     : 9/16/2021 5:34:38 PM
Datetime now      : 9/16/2021 5:34:38 PM
UTC Datetime now  : 9/16/2021 2:34:38 PM
** Closed [2]                           
Modification time : 9/16/2021 5:34:38 PM
Creation time     : 9/16/2021 5:34:38 PM
Datetime now      : 9/16/2021 5:34:38 PM
UTC Datetime now  : 9/16/2021 2:34:38 PM

As one can see, LastModificationTime is in local time and not UTC (I'm on UTC+3). I think this is because this is not store time (which is PR_LAST_MODIFICATION_TIME in UTC) but Outlook MSG time

Looking with MFCMAPI on MAPI properties I see: Tag: 0x30080040 Type: PT_SYSTIME Property Name: PR_LAST_MODIFICATION_TIME Other Names: PidTagLastModificationTime, ptagLastModificationTime DASL: http://schemas.microsoft.com/mapi/proptag/0x30080040 LowDateTime: 0xF9D9F2D0 HighDateTime: 0x01D7AB07 Date: 02:34:37.565 PM 9/16/2021

So the MAPI property is indeed UTC, but adding the column Modified in the Drafts view (where the messages are saved), the date displayed is Thu 9/16/2021 5:35 PM - which maps to LastModificationTime Outlook property.

MSDN says LastModificationTime maps to PidTagLastModificationTime (described here), indeed described as 2.2.3.17 PidTagLastModificationTime Data type: PtypTime ([MS-OXCDATA] section 2.11.1) The PidTagLastModificationTime property ([MS-OXCMSG] section 2.2.2.2) contains the date and time that an Address Book object was last modified in Coordinated Universal Time (UTC).

Although I can't find something explicitely stated, I think Outlook uses Application.TimeZones to return the date/time values in "user" time, while the MAPI properties are UTC/store time.

ndesktop avatar Sep 16 '21 14:09 ndesktop

Oh I think I get it. So the return type from outlook is fundamentally without a time zone. It's a little confusing since if I understand correctly the datetime object we're returning is based on datettime, they have a version that is offset independent(no timezone), but it seems like we're always return one with a utc timezone

ashaw596 avatar Sep 16 '21 16:09 ashaw596

Yes, this is where MSDN documentation is not very accurate. MAPI/storage props are UTC, but OOM looks like is oriented towards user settings (timezone, display format etc.). Anyway, I think the issue should be closed since pywin32 is not altering the result.

ndesktop avatar Sep 17 '21 07:09 ndesktop