Timezone aware fields in JSON response lose their timezone on conversion to datetime

[pjmcdermott]

list_events() returns timezone aware data for the openDate field, e.g. as indicated by the 'Z' for the UTC time zone in the response below:

>>> events=trading.betting.list_events()
>>> events[0].json()
'{"event":{"id":"29043131","name":"Aguada v Bigua","countryCode":"UY","timezone":"GMT","openDate":"2018-12-12T00:15:00.000Z"},"marketCount":3}'

However, following conversion to a datetime.datetime object the timezone information is lost.

>>> events[0].event.open_date
datetime.datetime(2018, 12, 12, 0, 15)

Conversion from JSON to datetime.datetime should not lose the timezone info.

[liampauling]

Open to PR’s

[rsampaio-0]

Hi!

Is this still open? I'd like to contribute. It seems that the problem is in line 42 of betfairlightweight/compat.py. The parse_datetime_as_naive format ignores timezones.

[mberk]

As this is essentially an interface change that will break existing code, for example:

events = trading.betting.list_events()
# Pretend betfairlightweight now returns aware datetimes
open_date = events[0].event.open_date.replace(tzinfo=datetime.timezone.utc)

# Raises TypeError: can't subtract offset-naive and offset-aware datetimes
time_until_event_starts = open_date - datetime.datetime.utcnow()

the default behaviour needs to remain the same with the returning of aware datetimes controlled by a parameter similar to lightweight.

BaseResource.strip_datetime feels like a better place to apply the timezone than parse_datetime:

    @staticmethod
    def strip_datetime(value, offset_aware=False):
        """
        Converts value to datetime if string or int.
        """
        if isinstance(value, basestring):
            try:
                dt = parse_datetime(value)
                if offset_aware:
                    dt = dt.replace(tzinfo=datetime.timezone.utc)
                return dt
            except ValueError:
                return
        elif isinstance(value, integer_types):
            try:
                dt = datetime.datetime.utcfromtimestamp(value / 1e3)
                if offset_aware:
                    dt = dt.replace(tzinfo=datetime.timezone.utc)
                return dt
            except (ValueError, OverflowError, OSError):
                return

The problem is that the places BaseResource.strip_datetime is called from (of which there are many) are typically the __init__s of objects which are exclusively concerned with assigning fields values and have no easy access to the state that would control this behaviour.