Try 2d, 3d CNN before attention

[JackKelly]

trafficstars

related: #35

3D cnn: sees two timesteps at once

[peterdudfield]

https://github.com/openclimatefix/predict_pv_yield/blob/issue/85-perciever-nwp/predict_pv_yield/models/perceiver/perceiver_conv3d_nwp_sat.py

This model does now

[JackKelly]

Nice! How well does the new model perform?! :)

[peterdudfield]

I dont have a direct comparision to running it without the cnn.

results: https://github.com/openclimatefix/predict_pv_yield/blob/issue/85-perciever-nwp/experiments/2021-09/2021-09-28/experiments.txt

[JackKelly]

Cool beans. No rush, but it might be nice to have a direct comparison, so we can get a feel for whether it's useful to place a CNN before the attention. What do you think?

[peterdudfield]

yea, I think it would be a useful to do. Could also see if 01, ... N layers of CNN are useful?

[JackKelly]

sounds great!