CI error when gadm server is down: add GADM file to datakits

[davide-f]

Checklist

• [x] I am using the current main branch or the latest release. Please indicate.
• [x] I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

Please provide a description of what the bug is and add a minimal example/command for reproducing the bug.

With the latest updates, the CI bundles should contain the appropriate gadm data and as such the workflow should not attempt to download them again and trigger the error. This clearly doesn't work and shall be checked.

In particular, there is the need to verify this line: https://github.com/pypsa-meets-earth/pypsa-earth/blob/b205e83c3b631448a778e29257eed551790b9d5c/scripts/build_shapes.py#L100 In the CI case, that condition should be false and so no access to the Davis server should be requested.

@yerbol-akhmetov this is related to what you are experiencing

It should be a quick fix also for new users in case

[davide-f]

Update: In the tutorial databundles, unfortunately we don't have the gadm files, that is the reason why the CI fails when the server is down.

To solve this issue, the volunteer shall:

1. download the only 3 databundles: bundle_tutorial_NGBJ/BW/MA and unzip them
2. download the corresponding gadm data, available at https://drive.google.com/drive/folders/1mAFe0HVVZXGH8ZBgh8_PPGlC3DGwe8jM?usp=drive_link
3. to each bundle in point 1, add the corresponding gadm files, having path "gadm/gadm41_{3 letter code country}/gadm41_{3 letter code country}.gpkg", basically a smart copy and paste from the shared folder. Note, the bundle NGBJ shall contain 2 folders for both NG and BJ, while those for BW and MA just one
4. re-zip the bundles, using the file at https://github.com/pypsa-meets-earth/pypsa-earth/blob/main/scripts/non_workflow/zip_folder.py
5. upload the zip files into gdrive and share them to us

[davide-f]

Revised title to make it clearer

[ekatef]

Hello @DhanshreeA! thanks a lot for offering help 🙂 Please feel free to tackle this one, if you are interested.

[davide-f]

Btw, we first need the permission to proceed. I'd suggest to keep this issue in stand-by

[ekatef]

Btw, we first need the permission to proceed. I'd suggest to keep this issue in stand-by

@davide-f thanks a lot for notifying. Yeah, if we don't obtained yet a permission from GADM team, then we need to put it on hold at the moment.

@DhanshreeA as a clarification: the issue implies re-uploading GADM data, which needs approval from GADM authors. We have requested a permission from them, but haven't obtained a response yet. So, let's wait until that will be resolved. Sorry for this!