pvoutput icon indicating copy to clipboard operation
pvoutput copied to clipboard

Convert downloaded data from localtime to UTC

Open JackKelly opened this issue 6 years ago • 6 comments
trafficstars

JackKelly avatar Aug 10 '19 18:08 JackKelly

so make sure all data is in UTC?

peterdudfield avatar Jan 04 '22 16:01 peterdudfield

Yeah, converting to UTC sounds good... If I remember correctly, the data from PVOutput.org doesn't have an explicit timezone but is in London timezone. I think somewhere in the PVOutput code it'll be necessary to do something like df = df.tz_localize("Europe/London").tz_convert("UTC") (and then removing these two lines from nowcasting_dataset/data_sources/pv/pv_data_source.py)

JackKelly avatar Jan 06 '22 17:01 JackKelly

some of it done in - https://github.com/openclimatefix/pvoutput/pull/102

peterdudfield avatar Mar 16 '22 12:03 peterdudfield

Just in case we come back to this later and need to remind ourselves (!): BB at PVOutput.org says that PVOutput.org returns data in the PV system's local time.

When we get round to downloading PVOutput.org data from other countries (which Sheffield Solar are already doing, I think) then we'll need to be able to give the pvoutput code multiple timezones. e.g. we need to tell pvoutput "these PV systems are in Europe/London timezone. These other PV systems are in Rome/Italy timezone. Please convert them to UTC".

JackKelly avatar Mar 17 '22 07:03 JackKelly

A neat way to solve issue https://github.com/openclimatefix/nowcasting_dataset/issues/620 would be to modify pvoutput so the new code explicitly sets the timezone to UTC (i.e. the times would have an explicit UTC offset, like this: 2022-03-17 08:00+00:00). That way, nowcasting_dataset (and any other code that consumes data downloaded by pvoutput) can be certain which timezone the data is in :slightly_smiling_face: (and, in general, perhaps we should always default to saving timestamps with explicit UTC offsets, to avoid any ambiguity. Although I seem to remember that NetCDF doesn't like timezone-aware timestamps, although I could be wrong!)

JackKelly avatar Mar 17 '22 07:03 JackKelly

I agree, that would be neat.

Needs a bit of thinking as somewhere we have to set the timezone, and that could proably be loaded from metadata

peterdudfield avatar Mar 17 '22 08:03 peterdudfield