Cluster by arbitrary shapes (NUTS1, NUTS2, NUTS3, GADM)

[fneum]

trafficstars

It is already possible to specify a custom busmap to the cluster_network rule to support arbitrary clustering. But that is so far a manual process.

There are many use cases for clustering according to administrative areas (NUTS, GADM, etc).

These are usually available as shapefiles (e.g. https://gisco-services.ec.europa.eu/distribution/v2/nuts/download/#nuts21).

I would like to automate clustering to arbitrary shapefiles. That could look something like this:

To create (note that you need not to supply a number of clusters):

snakemake -j all "networks/elec_s_shapes.nc"

Provide a shapefile in config.yaml:

clustering:
  optional_inputs:
    shapefile: "nuts/NUTS_RG_01M_2016_4326_LEVL_0.geojson"

That becomes and optional input to the snakemake rule in the Snakefile:

wildcard_constraints:
    clusters="[0-9]+m?|all|shapes",

rule cluster_network:
    input:
        **config["clustering"].get("optional_inputs", {})

I think the busmap should be relatively straightforward to create using geopandas.sjoin(). One would add a function to scripts/cluster_network.py:

def busmap_from_shapes():
    ...

Two difficulties I can think of right now:

• The mapping of a bus' Voronoi cell from the base network to the regions datafile is not always 100% clear; i.e. the borders of regions will run through Voronoi cells which need to be split/disaggregated. Vice versa, it can also occur that a region is fully contained inside a Voronoi cell of the base network (presumably less likely).
• Offshore nodes need to be mapped to onshore buses (that are covered by shapefile)

[fneum]

Example for NUTS1 and region shapes from elec_s.nc (color codes the NUTS1 region):

[pz-max]

Hi Fabian, the build_bus_region.py creates the Voronoi cells to which all information are aggregated. Maybe it is worth adjusting the build_bus_region.py instead of the cluster_network.py function?

Or is there a particular reason for adjusting cluster_network.py? FYI, if we have sometime, we would like to implement the GADM zones (which would make NUTS3 probably also easy to add)

[FabianHofmann]

But the Voronoi cells are build on basis of the ENTSOE gridmap, i.e. their nodes. I think the problem remains the same, as the nodes of the ENTOSE maps have to be aligned to the NUTS3 regions...

[pz-max]

Yes, true. The problem stays the same. What was nice with the Voronoi bus map technique is that each region had a clear connection. One node was connected to another one with a line. The main question is now with NUTS/GADM or other arbitrary shapes. How are these regions connected? Which region is linked to each with which magnitude?

In general what can happen:

• A region is isolated (only transmission/distribution infrastructure in the region itself)
• A region is connected with other regions

We would do the following:

1. Create ID's for any shape i.e. Voronoi_busmap/GADM/NUTS.
2. If buses/lines are in the same shape -> Associate the capacities and cost at the region as i.e. "distribution"
3. If two buses, connected by a line, are in two different regions. Define a connection between the region with the given transmission capacity and costs.

One could then also develop some nice search algorithms i.e.

• Connect from different locations to the isolated regions and assess if it reduces the total system costs (also DC lines connected from far distant regions) Or a non-search approach/option:
• Define all regions as connected to the neighbouring ones (This is a good default which I guess is applied in Calliope)

[fneum]

Hmmm, I don't really see a good way of defining the clustered regions further upstream without a tremendous amount of work. Everything is tied around the ENTSO-E grid map.

• The network simplification (aligning voltage level, removing stubs etc) in simplify_network would have to be moved further upstream, too.
• If the initial regions become too large build_renewable_profiles will likely get a performance hit because the rasters need to be chunked for calculating the renewable potentials.

[FabianHofmann]

• If the initial regions become too large build_renewable_profiles will likely get a performance hit because the rasters need to be chunked for calculating the renewable potentials.

This should not be a problem, the performance with large regions is pretty solid. But anyhow, it would require a lot of rearrangement considering the simplify_network and correct assignment of load time series.

[fneum]

@Hazem-IEG, were you planning to transfer this also to PyPSA-Eur at some point?

[hazemakhalek]

@fneum, yes, I already started working on it locally and did some initial tests a while back but stopped to focus on the sector coupled version of PyPSA-Earth.. soon I'll continue working on it

[fneum]

@Hazem-IEG any updates? This would be a nice feature to add to PyPSA-Eur.

[fneum]

This is in PyPSA-Earth:

Look for "alternative clustering":

https://github.com/pypsa-meets-earth/pypsa-earth/blob/dbca213b43bf96fa25ec5b4785cddadec3a85f5c/scripts/build_bus_regions.py#L200-L207

https://github.com/pypsa-meets-earth/pypsa-earth/blob/main/scripts/cluster_network.py

https://github.com/pypsa-meets-earth/pypsa-earth/blob/main/scripts/simplify_network.py