physicsnemo icon indicating copy to clipboard operation
physicsnemo copied to clipboard

Downloading and pipeline for constants and forcings for GraphCast

Open divijkhaitan opened this issue 9 months ago • 0 comments

I tried to download data from era5 using the 34var.yaml in examples/weather/dataset_download, but got the message: ValueError: chunks keys ('time',) not found in data dimensions ('valid_time', 'latitude', 'longitude'). This was with an extremely small subset of the required surface + 37 level data. With the whole dataset I kept getting denied permission from sending too many requests. Issue #631 recommends the usage of the unified_recipe for weather datasets, however I'm having trouble on handling the downloading and pipelining for the forcings and the constants for the model as outlined in the paper. I was able to translate the pre-trained weights that deepmind provided into the modulus implementation of the model with the goal of to reproducing the results and performing some additional training on those weights. The forcings need to be fed into the model at both inputs steps and the output step, while the constants are fixed for every grid point and to be fed into the model. Is there any way in the existing infrastructure to do this or should I be writing additional code to handle this? Thanks!

divijkhaitan avatar Mar 05 '25 15:03 divijkhaitan