Non-Nanosecond precision support by pandas now triggering warning for SEVIRI reader

[guidocioni]

Describe the bug Just FYI, the latest version of pandas started supporting non-nanosecond precision values for datetime, so something in seviri_base.py is tringgering a warning. Not a huge issue, but may be worth investigating and correcting it for the future release of satpy.

Here the warning

satpy/readers/seviri_base.py:385: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.

Environment Info:

• satpy 0.41.1
• pandas 2.0.0
• numpy 1.24.2
• xarray 2023.4.1

[djhoese]

Thank you for bringing this up. This is a known issue (and may have another issue here on github) and we've talked to xarray about it but are still not quite sure if we need to do anything. Xarray is basically telling us that in the future it won't automatically convert time values to nanosecond-level precision, but we also don't need that level of precision (at least in my understanding). From conversations with xarray devs it isn't clear to me what xarray will do when pandas has completely supports non-nanosecond times. In our case, that's completely fine so I'm not sure why we "deserve" to see this warning.

You're right that seviri_base.py has a lot of non-nanosecond time usage and is probably where this could be (easily?) fixed. If anyone wants to try to put a pull request together to try to avoid this warning will still preserving the existing behavior (don't corrupt time values by interpreting milliseconds as nanoseconds) then I think we'd be happy to accept it.

[guidocioni]

I could try to draft a PR, but I don't know the codebase, so for me it would be just a matter of going through the lines that are causing the warning, correct those and hope that these are the only ones affected :D I don't have many different files to test besides the one that I'm using in my pipeline

[lobsiger]

Just to confirm the above. If I make a multiscene image of two MSG satellites I get 96 warnings:

/home/eumetcast/miniconda3/envs/pytroll/lib/python3.11/site-packages/satpy/readers/seviri_base.py:386: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time. dataset.coords['acq_time'] = ('y', acq_time)

The resulting images are not affected. But this might frighten some users of my Satpy scripts.

[guidocioni]

@djhoese in the meantime I went into the code and tried to understand if there was an easy way to to a substitution of all the lines where the conversion to datetime was happening but it was harder than I thought. Do you know if there's an easy way to silence ONLY this specific warning?

[djhoese]

with warnings.catch_warnings():
    warnings.simplefilter("ignore", category=UserWarning, message="Converting non-nanosecond.*")
    ... all your code ...

Is how I would do it for one particular piece of code. I think you can do that simplefilter at the top of a script if you really wanted to without the catch_warnings and it would also work.

[guidocioni]

with warnings.catch_warnings():
    warnings.simplefilter("ignore", category=UserWarning, message="Converting non-nanosecond.*")
    ... all your code ...

Is how I would do it for one particular piece of code. I think you can do that simplefilter at the top of a script if you really wanted to without the catch_warnings and it would also work.

Sorry, feel like a stupid asking that but last time that I tried it was not working. The following at the beginning of the script worked like a charm

import warnings
warnings.filterwarnings("ignore", category=UserWarning, message="Converting non-nanosecond.*")

[djhoese]

It was definitely off the top of my head so there's a good chance I got it wrong.