resdata icon indicating copy to clipboard operation
resdata copied to clipboard

Circumvent Pandas bugs for years > 2262

Open berland opened this issue 4 years ago • 4 comments

Issue #802

Approach

Pandas is very eager to convert numpy datetime64[*] timestamps to datetime64[ns], and will error hard if a datetime64[ms] timestamp is beyond year 2262. Avoid this by giving Pandas datetime.datetime objects instead

berland avatar Feb 11 '21 09:02 berland

Can one of the admins verify this patch?

ertomatic avatar Feb 11 '21 09:02 ertomatic

Related to https://github.com/pandas-dev/pandas/issues/39727

berland avatar Feb 11 '21 09:02 berland

Can you write a unit test that tests that this does the right thing and also that existing user code doesn't break with this change? Thanks.

pinkwah avatar Feb 11 '21 09:02 pinkwah

Can you write a unit test that tests that this does the right thing and also that existing user code doesn't break with this change? Thanks.

Not sure I dare to touch the test-code in here. Identical codelines are tested in ecl2df:

https://github.com/equinor/ecl2df/pull/267

berland avatar Feb 16 '21 08:02 berland

The pandas issue was closed (as fixed) 2 weeks ago, https://github.com/pandas-dev/pandas/issues/39727, is this still relevant for us?

lars-petter-hauge avatar Jan 05 '23 15:01 lars-petter-hauge

I guess it is relevant until we have Pandas 2.0 (not released) in our stable environment, which is hard to predict when is feasible.

berland avatar Jan 06 '23 09:01 berland

@berland cam you take a look at this PR? Tests are not running and it seems to need a rebase main.

hnformentin avatar Jan 18 '23 08:01 hnformentin