Edm.DateTime ATOM format is not parsed correctly

[OmniTroid]

When execution reaches https://github.com/SAP/python-pyodata/blob/master/pyodata/v2/service.py#L765

It assumes the format is JSON, however the ATOM formatting is still perfectly valid, but the library seemingly has no mechanism to detect if this is the case or not.

So if this library is used against a server that ATOM-formats its Edm.DateTime, the following error happens:

pyodata.exceptions.PyODataModelError: Malformed value 2022-01-04 for primitive Edm.DateTime type. Expected format is /Date(<ticks>[±<offset>])/

[phanak-sap]

Hi @OmniTroid could you please provide test that would reproduce this behavior? I am not sure what to do to see what you see.

Could you also please provide information on which version of Pyodata this behavior occurs? If it is on latest 1.8.0, could you please try your scenario on previous version 1.7.1, to determine if it is from latest changes around Edm.DateTime?

If you want to provide metadata file or ideally the log file, feel free to attach here to issue or create PR in this repository - https://github.com/phanak-sap/pyodata-issue-files

[squeakyboots]

I have the same error running v1.9.1 of pyodata against an SAP ByDesign tenant's OData service: https://my301795.sapbydesign.com/sap/byd/odata/cust/v1/ablemate/BusinessPartnerCollection?$filter=InternalID eq 'NBS'

The JSON returned:

{
	"d": {
		"results": [
			{
				"__metadata": {
					"uri": "https://my301795.sapbydesign.com/sap/byd/odata/cust/v1/ablemate/BusinessPartnerCollection('00163E0265D71ED28ECFF48DDCD64CF2')",
					"type": "cust.BusinessPartner"
				},
				"ObjectID": "00163E0265D71ED28ECFF48DDCD64CF2",
				"InternalID": "NBS",
				"CSM_KUT": "",
				"ProjectManager_KUT": "",
				"Consultant_KUT": "",
				"ServiceLevel_KUT": "",
				"Platform_KUT": "",
				"LifeCycleStatusCode": "2",
				"RegionCode": "UT",
				"CountryCode": "US",
				"RoleCode": "BBP000",
				"LastChangeDateTime": "/Date(1639167537708)/",
				"SupportThroughDate_KUT": null,
				"BusinessPartnerFormattedName": "Navigator Business Solutions",
				"URI": "nbs-us.com"
			}
		]
	}
}

I believe it's the "LastChangeDateTime" value causing the error. I recognize it's not ATOM formatted, but it seems to be a similar problem handling date. Let me know if another bug would be appropriate.

Edit This change fixed it for me, but I don't know if it's a good one to pull in to the main and doesn't really address dates as a whole.

Edit 2 Rolling back to v1.7.1 fixed it for me.

Edit 3 It doesn't look like there's a datetime offset test at all in the test cases. I think it should be something like:

@pytest.fixture
def complex_type_property_declarations():
    return {
        'TestString': (Types.parse_type_name('Edm.String'), "'FooBar'", "'FooBar'", 'FooBar'),
        'TestBoolean': (Types.parse_type_name('Edm.Boolean'), False, 'false', False),
        'TestInt64': (Types.parse_type_name('Edm.Int64'), '123L', '123L', 123),
        'TestDateTime': (Types.parse_type_name('Edm.DateTime'), "/Date(2147483647000)/", "datetime'2038-01-19T3:14:7'",
                         datetime.datetime(2038, 1, 19, hour=3, minute=14, second=7, tzinfo=datetime.timezone.utc)),
        'TestDateTimeOffset': (Types.parse_type_name('Edm.DateTimeOffset'), "/Date(2147483647708)/",
                               "datetimeoffset'2038-01-19T03:14:708+00:00'",
                               datetime.datetime(2038, 1, 19, hour=3, minute=14, second=7, microsecond=708000,
                                                 tzinfo=datetime.timezone.utc))
    }

[phanak-sap]

Hi squeakyboots - thanks for thorough bug report and even own investigation + tryout fix.

I believe you have different problem than Omnitroid (guessing, there is still no code to reproduce, maybe it is the same, but it mentions return from server in ATOM formatting, not JSON as you provided). I would prefer you to create a new bug - I am not doing it, since you are referencing it from your project. Just copypaste there your part and I will copypaste there this answer and then delete both in here.

First of all - your edc scanner is great. This is exactly the tool I wanted to build when I would have some spare time :-) I need to check into more detail how you solved the odata v4 support.. isn't that something that should be here as well?

Second - please do a PR with your fix and the new tests (and reference the new bug there). If it fixed your problem, it is worth to be upstream.

Third - there are actually a datetime offset tests - but maybe not with the case that reproduces your bug. Refer to this commit (commit message, code changes and new tests), which re-worked the DateTime + DateTimeOffset and maybe you can reproduce the bug with addition of new line to correct parametrized test. https://github.com/SAP/python-pyodata/commit/ec56995551779cb99ae0a5734a7ecc3449e5191d#diff-7997cda67851c8c96f44d1246b73701adfa2afb643042bf34d11f872c2c5b974

[phanak-sap]

Based on provided info, this seems like at the moment duplicate of #231.

@squeakyboots - I totally forgot about your solution for same problem, it does look better than my ugly adding of "+0000" to the input :) So reused yours to the master branch.

[phanak-sap]

note for original @OmniTroid - ATOM is at the moment not supported at all in the pyodata, only the literal and JSONs... but should be, that is for sure.. but that is overall new feature.