BUG: enlarging a DataFrame by adding an extra column with a tz-aware datetime results in object dtype
Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[X] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
import datetime as dt
df = pd.DataFrame([{'id': 1},{'id': 2},{'id': 3}])
_time = dt.datetime.utcfromtimestamp(1695887042)
_time = _time.replace(tzinfo=dt.timezone.utc)
df.loc[df.id>=2, 'time'] = _time
print(df)
print(df.dtypes)
Issue Description
the above outputs
id time
0 1 NaT
1 2 2023-09-28 07:44:02+00:00
2 3 2023-09-28 07:44:02+00:00
id int64
time object
dtype: object
Expected Behavior
it would be nice to get to
id time
0 1 NaT
1 2 2023-09-28 07:44:02+00:00
2 3 2023-09-28 07:44:02+00:00
id int64
time datetime64[ns, UTC]
dtype: object
right away
Installed Versions
[1/1] Generating write_version_file with a custom command
- /home/marcogorelli/.virtualenvs/pandas/bin/ninja
INSTALLED VERSIONS
commit : 99efe62afd183e8600daa7020dbbcf8cf59d76cd python : 3.10.12.final.0 python-bits : 64 OS : Linux OS-release : 5.10.102.1-microsoft-standard-WSL2 Version : #1 SMP Wed Mar 2 00:30:59 UTC 2022 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_GB.UTF-8 LOCALE : en_GB.UTF-8
pandas : 2.1.0rc0+99.g99efe62afd.dirty numpy : 1.24.4 pytz : 2023.3.post1 dateutil : 2.8.2 setuptools : 59.6.0 pip : 23.2.1 Cython : 0.29.33 pytest : 7.4.2 hypothesis : 6.86.2 sphinx : 6.2.1 blosc : 1.11.1 feather : None xlsxwriter : 3.1.4 lxml.etree : 4.9.3 html5lib : 1.1 pymysql : 1.4.6 psycopg2 : 2.9.7 jinja2 : 3.1.2 IPython : 8.15.0 pandas_datareader : None bs4 : 4.12.2 bottleneck : 1.3.7 dataframe-api-compat: None fastparquet : 2023.8.0 fsspec : 2023.9.1 gcsfs : 2023.9.1 matplotlib : 3.7.3 numba : 0.57.1 numexpr : 2.8.6 odfpy : None openpyxl : 3.1.2 pandas_gbq : None pyarrow : 13.0.0 pyreadstat : 1.2.3 python-calamine : None pyxlsb : 1.0.10 s3fs : 2023.9.1 scipy : 1.11.2 sqlalchemy : 2.0.21 tables : 3.8.0 tabulate : 0.9.0 xarray : 2023.8.0 xlrd : 2.0.1 zstandard : 0.21.0 tzdata : 2023.3 qtpy : None pyqt5 : None None
the title suggests the issue is about the Series constructor, but the example seems to be about setitem-with-expansion. Is the example minimal?
thanks, have updated
im seeing the expected behavior on main. could use a test.
Take
Take