era5_in_gee
era5_in_gee copied to clipboard
Functions and Python scripts to ingest ERA5 data into Google Earth Engine
ERA5 reanalysis data in Google Earth Engine
This repository contains a set of functions and example scripts to ingest Copernicus ERA5 reanalysis data into Google Earth Engine based on manifest uploads.
The functions were developed during the process of making a subset of the ERA5 reanalysis data available in Google Earth Engine.
Workflow overview
The workflow consists of six major steps:
- Downloading hourly data as daily files or monthly aggregates in
NetCDF
format from the Climate Data Store withcdsapi
- Aggregating hourly files to daily means or sums (total precipitation) with
xarray
- Converting
NetCDF
data files toGeoTiff
withgdal
- Uploading hourly, daily and monthly
GeoTiff
files to Google Cloud Platform (GCP) withgoogle-cloud-storage Python API
- Creating image manifests (JSON-based files) describing the metadata and band names of the resulting Earth Engine asset
- Ingesting data files uploaded to GCP as assets into Earth Engine with
earthengine-api
and manifest uploads
Repository content
- ERA5 in GEE functions
- Python script
- Jupyter notebook
- Example workflows
- Hourly assets
- Daily assets
- Daily single asset
- Monthly assets
- Example manifest files
- Hourly multiple assets
- Daily multiple assets
- Daily single asset
- Monthly multiple assets
- Example scripts for
-
Retrieve ERA5 reanalysis from the CDS with
cdsapi
- Delete blobs from GCP buckets
-
Retrieve ERA5 reanalysis from the CDS with
Python packages required
Naming convention and folder structure
The scripts base on the following naming convention
and folder_structure
:
Naming Convention
Data is downloaded from the Climate Data Store with the following syntax:
-
era5_
+name_of_variable_
+year_
+month_
+day
+.nc
- Example for a file of 2m dewpoint temperature from 1 January 1980:
era5_2m_dewpoint_temperature_1980_01_01.nc
- Example for a file of 2m dewpoint temperature from 1 January 1980:
Folder Structure:
(example for 2m air temperature)
-
era5_t2m
-
nc
-
hourly
-
year
(e.g. 1979)
-
-
daily
-
year
(e.g. 1979)
-
-
monthly
-
year
(e.g. 1979)
-
-
-
tiff
-
hourly
-
year
(e.g. 1979)
-
-
daily
-
year
(e.g. 1979)
-
-
monthly
-
year
(e.g. 1979)
-
-
-
-
manifests
-
era5_hourly
-
year
(e.g. 1979)
-
-
era5_daily
-
year
(e.g. 1979)
-
-
era5_monthly
-
year
(e.g. 1979)
-
-
manifest_template.json
-
References
- ERA5 reanalysis data in the Climate Data Store
- Google Earth Engine
- ERA5 DAILY | GEE Public Data Catalog
- ERA5 MONTHLY | GEE Public Data Catalog
- ERA5 in GEE | Slides from the EarthEngineVirtualMeetup in Dec 2019
- ERA5 reanalysis data available in Earth Engine - ECMWF Newsletter Number 162 Winter 2020
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.