Model performance plots

[vitusbenson]

Hi @jacobbieker ,

Happy new year!:) I was wondering if you ever measured the performance of your models with this code. Like is it similar to Keisler etc? I saw there are some pretrained weights on Huggingface, but I am a bit puzzled how to use them (otherwise would just create the plots myself). Is there some tutorial or similar? Thought I'd ask you directly before trying to reverse-engineer what you did.

Thanks in advance, Vitus

[jacobbieker]

Hi! So there are pretrained weights, but they are not very good. Part of the issue has been that the models take a huge amount of GPU memory at the moment (#32). I'm slowly working on reducing the memory footprint (#47 for example as probably one of the biggest bottlenecks) so we can train it the same as in the paper, but at the moment, I am unable to. I have more of the data available on HuggingFace for training it, but haven't been able to train it fully yet.

[vitusbenson]

Ahh cool! Thanks for the clarification:) I don't know if you use the Pytorch lightning script or not, but there is a bug:

https://github.com/openclimatefix/graph_weather/blob/3cc040aed704940a60a4e3b8c730bf61663a43f9/train/pl_graph_weather.py#L280

This makes the edge processor very deep, which kills gradients..

[jacobbieker]

Ah, good spot, thanks!

[peterdudfield]

@all-contributors please add @vitusbenson for bug

[allcontributors[bot]]

@peterdudfield

I've put up a pull request to add @vitusbenson! :tada: