graph_weather icon indicating copy to clipboard operation
graph_weather copied to clipboard

Add support for training on Google's ARCO ERA5

Open jacobbieker opened this issue 2 years ago • 4 comments

Detailed Description

The Google ERA5 archive is available: https://github.com/google-research/ARCO-ERA5 and goes back quite far. Its currently being expanded, but already covers years of data at the native model resolution. This is probably quicker and faster to pull data from than HuggingFace, and is already on Google Public Datasets, so should be easy to integrate.

Context

Its a lot of data in an accessible and publicly available place that we can try to leverage for training the graph models easily.

Possible Implementation

A HuggingFace dataset script like for the EUMETSAT data? Or can just add Xarray opening it in this repo directly as well.

jacobbieker avatar Nov 10 '22 08:11 jacobbieker

The HuggingFace datasets script will be hosted here https://huggingface.co/datasets/openclimatefix/arco-era5

jacobbieker avatar Nov 27 '22 17:11 jacobbieker

Hi, is this issue available to work on?

ishachadalavada avatar Apr 03 '24 18:04 ishachadalavada

Yes, I believe so, I can assign you it?

jacobbieker avatar Apr 04 '24 07:04 jacobbieker

Yes, that sounds good!

ishachadalavada avatar Apr 05 '24 19:04 ishachadalavada