Zeep reduces the millisecond accuracy from 7 down to 6 in a date field

[marco-itnetsys]

Version: zeep 2.5.0

WDSL: https://www.migrationwiz.com/API/2.0/WebService.asmx?wsdl

Script:

import pretend  # pip install pretend

from zeep import Client
from zeep.transports import Transport

client = Client('https://www.migrationwiz.com/API/2.0/WebService.asmx?wsdl')
response = pretend.stub(
    status_code=200,
    headers={'Cache-Control': 'private, max-age=0', 'Content-Type': 'text/xml; charset=utf-8', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding', 'Server': 'Microsoft-IIS/10.0', 'Request-Context': 'appId=cid-v1:53a2eb5f-e059-4d9a-a2f5-0572c1f34d39', 'X-Powered-By': 'ASP.NET', 'X-BitTitan-Origin': 'IISPROD-22-vmss_3', 'Date': 'Wed, 02 May 2018 14:51:17 GMT', 'Content-Length': '640'},
    content="""
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body>
    <ExecuteResponse xmlns="http://bittitan.com/migrationwiz/1">
      <ExecuteResult xsi:type="AuthenticateResponse">
        <Diagnostics>X-BitTitan-Origin:IISPROD-2000003
X-BitTitan-ExecutionTime:93
X-HttpRequestTraceId:00000000-0000-0000-0000-000000000000</Diagnostics>
        <Ticket>
          <UserId>redacted</UserId>
          <ExpirationDate>2018-05-02T22:51:18.5210882Z</ExpirationDate>
          <IsPrivileged>false</IsPrivileged>
          <Purpose>0</Purpose>
          <ImpersonatingUserId>00000000-0000-0000-0000-000000000000</ImpersonatingUserId>
          <Hash>redacted</Hash>
        </Ticket>
      </ExecuteResult>
    </ExecuteResponse>
  </soap:Body>
</soap:Envelope>
    """)

# AuthenticateRequest = ZEEP.get_type('ns0:AuthenticateRequest')
# authenticate_request = AuthenticateRequest(EmailAddress="redacted",
#                                            Password="redacted",
#                                            ImpersonateUserId=xsd.SkipValue)
# result = ZEEP.service.Execute(authenticate_request)
operation = client.service._binding._operations['Execute']
result = client.service._binding.process_reply(client, operation, response)
print(result)

Hi, I'm having the issue that Zeep reduces the millisecond accuracy from 7 down to 6 in a date field. The server gives me an authentication ticket with a value called ExpirationDate and when I sent it back unaltered it misses the 7th millisecond number (in this case the 2):

Server
    <ExpirationDate>2018-05-02T22:51:18.5210882Z</ExpirationDate>
<ns0:ExpirationDate>2018-05-02T22:51:18.521088Z</ns0:ExpirationDate>
Zeep

[mvantellingen]

Thanks, are you able to submit a PR which adds a failing unittest for this?

[marco-itnetsys]

Sure!

As the default datetime only supports a millisecond value up to 999999 (the root cause of the issue), do you want me to assert against numpy.datetime64, similar to the test_parse_invalid_values test in test_xsd_parse? Or would you prefer a more blackbox approach to the test, where I feed it xml and assert that the outgoing xml is correct? If so, do you already have a similar test I can base it on?

Thank you!

[mvantellingen]

So re-reading your issue it seems this is a limitation of the datetime type in python right? You could subclass zeep.xsd.builtins.DateTime to use numpy.datetime64 and register it in zeep.xsd.builtins.default_types and see if that solves the issue?

[marco-itnetsys]

I didn't realize the datetime limitation until I looked into writing the unittest either.

Thanks for pointing me in the right direction. I changed the code to leave datetime values untouched as string, so I can use this API, but I'm not sure how to best implement something that would work for everyone.

If I understand it correctly the issue with datetime64 is that it doesn't track the precision (it adds two zeros) and it doesn't have built in functionality to output dates as specifically formatted strings.

Maybe storing the original value and using that to generate the response if the value hasn't changed?

Are you interested in extending zeep to support this or is it too much of an edge case? (The BitTitan Migrationwiz API is public, not something company internal)

[podfran]

I am struggling with a related issue: The datetime in the response I would get would be '9999-12-31T23:59:59.9999999', which is DateTime.MaxValue in C#. This would cause:

ERROR:zeep.xsd.types.simple:Error during xml -> python translation
Traceback (most recent call last):
  File "C:\Users\fpodborski\PycharmProjects\raido\Dev\Src\PythonSeleniumTests\venv\lib\site-packages\zeep\xsd\types\simple.py", line 79, in parse_xmlelement
    return self.pythonvalue(xmlelement.text)
  File "C:\Users\fpodborski\PycharmProjects\raido\Dev\Src\PythonSeleniumTests\venv\lib\site-packages\zeep\xsd\types\builtins.py", line 44, in _wrapper
    return func(self, re.sub(r"[\n\r\t ]", " ", value).strip())
  File "C:\Users\fpodborski\PycharmProjects\raido\Dev\Src\PythonSeleniumTests\venv\lib\site-packages\zeep\xsd\types\builtins.py", line 180, in pythonvalue
    return isodate.parse_datetime(value)
  File "C:\Users\fpodborski\PycharmProjects\raido\Dev\Src\PythonSeleniumTests\venv\lib\site-packages\isodate\isodatetime.py", line 56, in parse_datetime
    tmptime = parse_time(timestring)
  File "C:\Users\fpodborski\PycharmProjects\raido\Dev\Src\PythonSeleniumTests\venv\lib\site-packages\isodate\isotime.py", line 131, in parse_time
    return time(int(groups['hour']), int(groups['minute']),
ValueError: second must be in 0..59

It is related to how isodate parses dates and that Python only has accuracy down to 1 microsecond in datetime and not one 100 nanosecond like in C#, as mentioned above. I have reported this to isodate, but a temp fix here is also possible:

--- venv/Lib/site-packages/zeep/xsd//types/builtins.py	2021-10-01 11:56:33.426973000 +0200
+++ builtins.py	2021-10-01 12:12:51.543643900 +0200
@@ -173,6 +173,10 @@
             value += "T00:00:00"
         elif (len(value) == 19 or len(value) == 26) and value[10] == " ":
             value = "T".join(value.split(" "))
+        fraction_second = re.search(r'\d{7}', value)
+        if fraction_second:
+            if int(fraction_second[0]) > 9999990:
+                value = re.sub(r'\d{7}', '999999', value)
         return isodate.parse_datetime(value)

Here I am primarily trying to avoid the edge case of rounding 0.9999995 and higher to 1.0. Note: This would be a temporary hack until a better solution would be developed to accommodate for one 100 nanosecond accuracy.