pandas icon indicating copy to clipboard operation
pandas copied to clipboard

BUG/API: to_json with non-nano TimedeltaIndex

Open jbrockmendel opened this issue 1 month ago • 3 comments

Pandas version checks

  • [x] I have checked that this issue has not already been reported.

  • [x] I have confirmed this bug exists on the latest version of pandas.

  • [x] I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

# based on test_timedelta_as_label

df = pd.DataFrame([[1]], columns=[pd.Timedelta("1D").as_unit("us")])
result = df.to_json(date_format="iso")

expected = '{{"P1DT0H0M0S":{{"0":1}}}}'
assert result == expected  # nope! '{"P0DT0H1M26.400S":{"0":1}}'


result = df.to_json(date_format="epoch")
expected = '{{"86400000":{{"0":1}}}}'
assert result == expected  # nope! '{"86400":{"0":1}}'

Issue Description

#63196 affects test_timedelta_as_label so that we get "us" unit for df.columns (in the example i added the .as_unit("us"), which affects the to_json output. For the epoch format that might be plausible but the iso format case is pretty clearly wrong.

cc @willayd

Expected Behavior

N/A

Installed Versions

Replace this line with the output of pd.show_versions()

jbrockmendel avatar Nov 30 '25 22:11 jbrockmendel

Looks like the code always assume nanoseconds when stringifying the index, at least according to the below comment:

https://github.com/pandas-dev/pandas/blob/1895c382296bca720acc358b996f4800dbdf6c9c/pandas/_libs/src/vendored/ujson/python/objToJSON.c#L1319

That's unfortunate that it doesn't raise....but it any case needs implementation

WillAyd avatar Dec 01 '25 15:12 WillAyd

can this (or something similar around us/ns) be related to this behavior we've seen on PyArrow when using pandas devel/nightlies for the last 4-5 days? https://github.com/apache/arrow/issues/48314

raulcd avatar Dec 03 '25 09:12 raulcd

joris's comments on the arrow issue are correct.

jbrockmendel avatar Dec 03 '25 15:12 jbrockmendel