BUG: to_datetime() with errors='coerce' raises AttributeError on MacOS

[nhoover]

trafficstars

Pandas version checks

• [X] I have checked that this issue has not already been reported.

• [X] I have confirmed this bug exists on the latest version of pandas.

• [X] I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

>>> import pandas as pd
>>> pd.to_datetime(['2018-10-26 12:00:00', '10-3-2014 13:00'], format='mixed', errors='coerce')
DatetimeIndex(['2018-10-26 12:00:00', '2014-10-03 13:00:00'], dtype='datetime64[ns]', freq=None)
>>> pd.to_datetime(['2018-10-26 12:00:00', '10-3-2014 13:AA'], format='mixed', errors='coerce')
DatetimeIndex(['2018-10-26 12:00:00', 'NaT'], dtype='datetime64[ns]', freq=None)
>>> pd.to_datetime(['2018-10-26 12:00:00', '10-3-2014 13:00:00 PDT'], format='mixed', errors='coerce')
<stdin>:1: FutureWarning: Parsing '{res.tzname}' as tzlocal (dependent on system timezone) is deprecated and will raise in a future version. Pass the 'tz' keyword or call tz_localize after construction instead
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/nathanhoover/dev/nstrava/venv/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 1082, in to_datetime
    result = convert_listlike(argc, format)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nathanhoover/dev/nstrava/venv/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 455, in _convert_listlike_datetimes
    result, tz_parsed = objects_to_datetime64ns(
                        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nathanhoover/dev/nstrava/venv/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 2177, in objects_to_datetime64ns
    result, tz_parsed = tslib.array_to_datetime(
                        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas/_libs/tslib.pyx", line 402, in pandas._libs.tslib.array_to_datetime
  File "pandas/_libs/tslib.pyx", line 527, in pandas._libs.tslib.array_to_datetime
AttributeError: 'NoneType' object has no attribute 'total_seconds'

Issue Description

When converting a date string with to_datetime(errors='coerce') an AttributeError is raised in one case where the input string contains a timezone.

Expected Behavior

In the above examples, the first case '10-3-2014 13:00' is properly converted as '2014-10-03 13:00:00' The second case '10-3-2014 13:AA' properly converts to 'NaT' with no exception The third case '10-3-2014 13:00:00 PDT' should either work or convert to NaT - either way no AttributeError should be raised.

Installed Versions

INSTALLED VERSIONS

commit : 0f437949513225922d851e9581723d82120684a6 python : 3.11.4.final.0 python-bits : 64 OS : Darwin OS-release : 22.6.0 Version : Darwin Kernel Version 22.6.0: Wed Jul 5 22:21:56 PDT 2023; root:xnu-8796.141.3~6/RELEASE_X86_64 machine : x86_64 processor : i386 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8

pandas : 2.0.3 numpy : 1.25.0 pytz : 2022.7.1 dateutil : 2.8.2 setuptools : 65.5.1 pip : 23.2.1 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 3.1.2 IPython : None pandas_datareader: None bs4 : None bottleneck : None brotli : None fastparquet : None fsspec : None gcsfs : None matplotlib : None numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : None pyreadstat : None pyxlsb : None s3fs : None scipy : None snappy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None zstandard : None tzdata : 2023.3 qtpy : None pyqt5 : None

[Cyddharth-Gupta]

take

[Cyddharth-Gupta]

Hey @nhoover, just checked the issue, the third case is also converting to NaT without throwing any exception. I am using the latest version of pandas and it isn't occurring there. Also, I tried it in an online compiler and it seemed to return the desired result, i.e., NaT.

[asishm-wk]

Doesn't reproduce for me either (on WSL2).

With 2.0.3 I get DatetimeIndex(['2018-10-26 12:00:00', '2014-10-03 13:00:00'], dtype='datetime64[ns]', freq=None)

On main I get a deprecation warning with the same result as above. FutureWarning: Parsed string "10-3-2014 13:00:00 PDT" included an un-recognized timezone "PDT". Dropping unrecognized timezones is deprecated; in a future version this will raise. Instead pass the string without the timezone, then use .tz_localize to convert to a recognized timezone.

[jenno-verdonck]

With 2.0.3 on windows I get different results depending on the input. I never seem to get the AttributeError .

Input: pd.to_datetime(['2018-10-26 12:00:00', '10-3-2014 13:00:00 PDT'], format='mixed', errors='coerce') Output: DatetimeIndex(['2018-10-26 12:00:00', '2014-10-03 13:00:00'], dtype='datetime64[ns]', freq=None)

Input: pd.to_datetime(['10-3-2014 13:00:00 PDT'], errors='coerce') Output: UnknownTimezoneWarning: tzname PDT identified but not understood. Pass 'tzinfos' argument in order to correctly return a timezone-aware datetime. In a future version, this will raise an exception. warnings.warn("tzname {tzname} identified but not understood. " DatetimeIndex(['2014-10-03 13:00:00'], dtype='datetime64[ns]', freq=None)

The same exception as reported above was also reported here #54481.

[nhoover]

Well it's so hard to reproduce and interesting that the same issue was reported again a few hours after I did. So it's happening for at least 2 people. Running on Windows 10 and Windows 11 with same 3.11.4 python/2.0.3 pandas I get the same as @jenno-verdonck

I submitted the issue running on MacOS 13.5 so adjusted the issue title.

[nhoover]

A friend tried this on an M1 Mac and also couldn't reproduce. So maybe it's only on Intel Mac. Mine is a 2019 Mac Book Pro: 2.4 GHz 8-Core Intel Core i9.

[lithomas1]

I can't reproduce but, If this happens utcoffset should be returning None at this line https://github.com/pandas-dev/pandas/blob/bdb509f95a8c0ff16530cedb01c2efc822c0d314/pandas/_libs/tslib.pyx#L527

cc @MarcoGorelli

[mqk]

This issue is still present in pandas 2.2.3. I'm on an M1 Mac.

In a brand new venv:

(tst) ➜  Analysis python --version
Python 3.11.10

(tst) ➜  Analysis pip install pandas
[...]
Installing collected packages: pytz, tzdata, six, numpy, python-dateutil, pandas
Successfully installed numpy-2.2.1 pandas-2.2.3 python-dateutil-2.9.0.post0 pytz-2024.2 six-1.17.0 tzdata-2024.2

(tst) ➜  Analysis pip freeze
numpy==2.2.1
pandas==2.2.3
python-dateutil==2.9.0.post0
pytz==2024.2
six==1.17.0
tzdata==2024.2

(tst) ➜  Analysis python -c 'import pandas as pd; pd.to_datetime("2023-07-18 21:06:21.214000 PDT")'
<string>:1: FutureWarning: Parsing 'PDT' as tzlocal (dependent on system timezone) is deprecated and will raise in a future version. Pass the 'tz' keyword or call tz_localize after construction instead
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/mikekuhlen/.virtualenvs/tst/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 1101, in to_datetime
    result = convert_listlike(np.array([arg]), format)[0]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mikekuhlen/.virtualenvs/tst/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 435, in _convert_listlike_datetimes
    result, tz_parsed = objects_to_datetime64(
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mikekuhlen/.virtualenvs/tst/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 2398, in objects_to_datetime64
    result, tz_parsed = tslib.array_to_datetime(
                        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "tslib.pyx", line 414, in pandas._libs.tslib.array_to_datetime
  File "tslib.pyx", line 578, in pandas._libs.tslib.array_to_datetime
AttributeError: 'NoneType' object has no attribute 'total_seconds'

Happy to help triage this further.

[jbrockmendel]

What timezone was the machine you were using localized in? In particular, were you located in US/Pacific timezone?

The deprecation in the OP has been enforced, so it wouldn't surprise me if it no longer reproduces for anyone.

[nhoover]

I no longer have access to a Mac. On my current windows system with python 3.13.8/pandas 2.3.3, I can't reproduce the problem either. It is in US/Pacific, same as my previous Mac.

[mqk]

I still reproduces for me with python 3.11.10 and updated packages:

(tst) ➜  Analysis pip freeze
numpy==2.3.3
pandas==2.3.3
python-dateutil==2.9.0.post0
pytz==2025.2
six==1.17.0
tzdata==2025.2

(tst) ➜  Analysis python -c 'import pandas as pd; pd.to_datetime("2023-07-18 21:06:21.214000 PDT")'
<string>:1: FutureWarning: Parsing 'PDT' as tzlocal (dependent on system timezone) is deprecated and will raise in a future version. Pass the 'tz' keyword or call tz_localize after construction instead
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/mikekuhlen/.virtualenvs/tst/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 1106, in to_datetime
    result = convert_listlike(np.array([arg]), format)[0]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mikekuhlen/.virtualenvs/tst/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 437, in _convert_listlike_datetimes
    result, tz_parsed = objects_to_datetime64(
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mikekuhlen/.virtualenvs/tst/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 2415, in objects_to_datetime64
    result, tz_parsed = tslib.array_to_datetime(
                        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas/_libs/tslib.pyx", line 412, in pandas._libs.tslib.array_to_datetime
  File "pandas/_libs/tslib.pyx", line 578, in pandas._libs.tslib.array_to_datetime
AttributeError: 'NoneType' object has no attribute 'total_seconds'

What timezone was the machine you were using localized in? In particular, were you located in US/Pacific timezone?

Yes, I'm in US/Pacific timezone.

[jbrockmendel]

<string>:1: FutureWarning: Parsing 'PDT' as tzlocal (dependent on system timezone)

That deprecation has been enforced, so that message should not be showing up on main.

[mroeschke]

Since this probably isn't showing up on the main branch, going to close