Add optional rule for updating network topology from entsoe map

[coroa]

Must be optional, since it is based on GridKit and requires a working Postgres database with PostGIS. Based on https://github.com/PyPSA/GridKit/tree/master/entsoe and https://rustyb.github.io/ee_transmap/.

[nworbmot]

Need to also update heuristic for reading line tags (e.g. multiple circuit types).

[coroa]

To make some headway here, there are 4 steps this entails:

1. Getting the raw geojson data from their new mapbox endpoint
2. Importing them into a postgres database with the postgis extension
3. Running Gridkit on them
4. Exporting them

All steps have to be run on the https://github.com/PyPSA/GridKit repo in the entsoe directory.

Step 1: Getting the raw GeoJSON data

This is in principle accomplished by running the download.sh script. You can remove the two rusty handles from the while loop, those are the previous datasets (as used in pypsa-eur, atm). One of the entsoe.* files will contain buses, one will contain line segments.

The problem is, that the underlying vt-geojson library hasn't received any love in the last year and the rapidly changing javascript ecosystem has long evolved away.

I found it is possible to get it to run anyway with the help of an ancient docker image for node:

docker run -it -v $PWD:/app:rw -w /app node:8.16.0-alpine sh download.sh

Steps 2-4: Import, running and export

Are governed in order by the shell-scripts import.sh, run.sh and export.sh (creative isn't it).

These depend on a postgres database server with the PostGIS extension and a python installation with several packages. It's probably possible with the correct docker image again, but previously I just installed the postgres things with apt.

Once postgres is running, you must create a new database (maybe install the PostGIS extension there?) and set the environment variables PG_USER, PG_PASSWORD and PG_DATABASE in your shell.

The python environment (don't know whether python3 works or you'll have to go back to 2.7) should contain:

psycopg2
geopandas
fiona
geojson

Then, you should (brace for impact) be able to execute the shell scripts in order and finally get all the csv files which you need for pypsa-eur.

Note that the ids and oids differ from the version in pypsa now, so that you will have to re-write the data/parameter_corrections.yaml file, maybe what is in there has been already fixed in the upstream map, though.

I'll try watching this thread, if you need further instructions.

[pz-max]

We discussed this today. We won't add an optional rule but aim to document the process how to update the network. @martacki is speaking to @coroa to get some information on the process (if understood correctly)