BUG/API: to_json with non-nano TimedeltaIndex
Pandas version checks
-
[x] I have checked that this issue has not already been reported.
-
[x] I have confirmed this bug exists on the latest version of pandas.
-
[x] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
# based on test_timedelta_as_label
df = pd.DataFrame([[1]], columns=[pd.Timedelta("1D").as_unit("us")])
result = df.to_json(date_format="iso")
expected = '{{"P1DT0H0M0S":{{"0":1}}}}'
assert result == expected # nope! '{"P0DT0H1M26.400S":{"0":1}}'
result = df.to_json(date_format="epoch")
expected = '{{"86400000":{{"0":1}}}}'
assert result == expected # nope! '{"86400":{"0":1}}'
Issue Description
#63196 affects test_timedelta_as_label so that we get "us" unit for df.columns (in the example i added the .as_unit("us"), which affects the to_json output. For the epoch format that might be plausible but the iso format case is pretty clearly wrong.
cc @willayd
Expected Behavior
N/A
Installed Versions
Replace this line with the output of pd.show_versions()
Looks like the code always assume nanoseconds when stringifying the index, at least according to the below comment:
https://github.com/pandas-dev/pandas/blob/1895c382296bca720acc358b996f4800dbdf6c9c/pandas/_libs/src/vendored/ujson/python/objToJSON.c#L1319
That's unfortunate that it doesn't raise....but it any case needs implementation
can this (or something similar around us/ns) be related to this behavior we've seen on PyArrow when using pandas devel/nightlies for the last 4-5 days? https://github.com/apache/arrow/issues/48314
joris's comments on the arrow issue are correct.