pypsa-eur icon indicating copy to clipboard operation
pypsa-eur copied to clipboard

Support Ukraine and Moldova in the country list and in the dataset.

Open euronion opened this issue 3 years ago • 11 comments

TODO list

New country keys to be supported:

TODOs

  • [x] Crimea: Is automatically included or not when using "UA" key (country_shapes.json)? (Crimea not included
  • [x] Keep Crimea as part of "UA" key or not?
  • [x] Updated power plants list incl. Moldova @FabianHofmann
  • [x] OPSD (load demand) @martacki
  • [x] ENTSO-E grid extract
  • [x] 750 kV transmission electrical params @fneum @p-glaum
  • [x] 750 kV lines types -> point release PyPSA to use with pypsa-eur @fneum -> 0.19.2 (conda may take 24h)
  • [x] 750 kV transmission include in simplify_network & enable in pypsa-eur config line type & enforce min. version in environment.yaml for pypsa point release @p-glaum (alt.: 380kV assumptions)
  • [x] GDP/POP data for assigning load data to bus regions. @euronion @martacki (decided to use GDP data only, i.e. no population)
  • [x] No pop data considered for add_electricity for MD, UA (simlification)
  • [x] Cutout containing Ukraine: ERA5 @euronion @fneum
  • [x] Cutout containing Ukraine: SARAH @euronion
  • [x] Hydro profiles for Ukraine (Moldova has only ~100 MW hydro if seen correctly, so less relevant) @euronion [Needs update to databundle, see below]
  • [ ] Renewable potentials?

Additions to databundle

  • [x] EIA data (https://github.com/PyPSA/pypsa-eur/pull/321#issuecomment-1059728440)
  • [x] @martacki gdp data for UA, MD (Link: https://github.com/martacki/pypsa-ua_gdp_data)
  • [x] Update cutouts from here @fneum https://zenodo.org/record/6350001#.Yi3p9Lko8UE

Release

  • [ ] Release notes

Checklist

  • [ ] I tested my contribution locally and it seems to work fine.
  • [ ] Code and workflow changes are sufficiently documented.
  • [ ] Newly introduced dependencies are added to envs/environment.yaml and envs/environment.docs.yaml.
  • [ ] Changes in configuration options are added in all of config.default.yaml, config.tutorial.yaml, and test/config.test1.yaml.
  • [ ] Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
  • [ ] A note for the release notes doc/release_notes.rst is amended in the format of previous release notes.

euronion avatar Mar 04 '22 13:03 euronion

So one problem currently is that PyPSA-Eur thinks UA is synchronized with continental Europe, which it is not. Well not all of it, there is a small Island in the border region that is synchronized with continental Europe. I tried to find out where the B2B converters are, but I couldn't find them. As all studies EU + UA I can think of will assume synchronisation, I'd propose to keep synchronised.

If somebody wants to extend this investigation:

  • https://www.lowcarbonukraine.com/wp-content/uploads/Synchronising-Ukraine%C2%B4s-and-Europe%C2%B4s-electricity-grids.pdf
  • https://op.europa.eu/en/publication-detail/-/publication/785f224b-93cd-11e9-9369-01aa75ed71a1
  • https://en.wikipedia.org/wiki/Burshtyn_TES
  • https://en.wikipedia.org/wiki/Rzesz%C3%B3w%E2%80%93Khmelnytskyi_powerline
  • https://en.wikipedia.org/wiki/Vetrino%E2%80%93Isaccea%E2%80%93Yuzhnoukrainsk_powerline
  • https://en.wikipedia.org/wiki/Albertirsa%E2%80%93Zakhidnoukrainska%E2%80%93Vinnytsia_powerline

fneum avatar Mar 04 '22 15:03 fneum

PR for new 750kV standard line type in PyPSA created at https://github.com/PyPSA/PyPSA/pull/369

p-glaum avatar Mar 04 '22 18:03 p-glaum

build_hydro requires input from the EIA. The last time we retrieved the data from there was 2017, they changed their data access and csv format since then quite abit. For prototyping, instead of adjusting all scripts and rule, I've added the values for Ukraine and Moldova to the EIA data file. See attached.

EIA_hydro_generation_2000_2014.csv

  • [x] Needs to be added to the databundle.

edit:

With the updated EIA data build_hydro runs without issues for MD. I have no reason to believe it would not work for UA, I can' test it atm as I don't have access to the new cutouts right now.

Two things we should track (seperate issues):

Quality Plotting inflow from build_hydro reveals unrealistic peaks for MD. A similar peak is also present for DE, other countries were not investigated. The peaks in both cases exceed what is possible with existing hydro capacities (DE: factor 2 ca. 4.5 GW installed, MD: factor 5-10, ca. 50 MW installed)

image

Since these are only temporary peaks they probably don't affect the results too much, also because they are relatively low.

Workflow The EIA data format from our datasource changed. Suggestions

  • Automatic download of EIA data OR update the EIA .csv file with hydro generation to new dataformat
  • substitute the vresutils dependency in build_hydro by writing the pd.read_csv statement directl.y

euronion avatar Mar 05 '22 09:03 euronion

if you want to run the last commits yourself (add_electricity), you need to generate or download the input data GDP_PPP_30arcsec_v3_mapped.csv once and move the file into pypsa-eur/data/bundle. edit: in case you download the data and don't re-generate it, you should rename GDP_PPP_30arcsec_v3_mapped_default.csv to GDP_PPP_30arcsec_v3_mapped.csv

Available here: https://github.com/martacki/pypsa-ua_gdp_data

Caution: Quick&Dirty solution; in case you execute the script - it runs very slow!! (~2h on my local machine)

martacki avatar Mar 05 '22 10:03 martacki

if you want to run the last commits yourself (add_electricity), you need to generate or download the input data GDP_PPP_30arcsec_v3_mapped.csv once and move the file into pypsa-eur/data/bundle. edit: in case you download the data and don't re-generate it, you should rename GDP_PPP_30arcsec_v3_mapped_default.csv to GDP_PPP_30arcsec_v3_mapped.csv

Available here: https://github.com/martacki/pypsa-ua_gdp_data

Caution: Quick&Dirty solution; in case you execute the script - it runs very slow!! (~2h on my local machine)

How does the quality of distributing load based on this look? We two already talked about it, but we could use the same datasets and roll-out the same approach for everywhere and instead drop the NUTS3 approach currently used.

euronion avatar Mar 07 '22 09:03 euronion

PyPSA 0.19.2 is out on PyPI with 750kV line type, conda may take 24h.

fneum avatar Mar 07 '22 15:03 fneum

new cutouts now available at: https://zenodo.org/record/6350001#.Yi3p9Lko8UE

fneum avatar Mar 13 '22 12:03 fneum

Looking at the potentials for renewables for UA, MD, @fneum was right: The potentials are currently 0, as Corine is not extending to there.

I quickly used the Copernicus Land Cover dataset to see how well that could substitute Corine, here the results (onshore wind only):

image

The match is pretty poor for densily populated areas. I believe that is because the Copernicus dataset does not differentiate between build-on areas, while the Corine dataset differentiates between different types of buildings.

euronion avatar Mar 17 '22 07:03 euronion

We could set the renewable potentials to 0 (current status), set them to np.inf (hacky, but ok for now) or manually set them to values based on the Copernicus dataset (quick and dirty, hardcoded, probably the most accurate).

What do you think?

euronion avatar Mar 17 '22 07:03 euronion

+1 for hardcoded Copernicus for UA/MD profiles look ok?

fneum avatar Mar 17 '22 07:03 fneum

EDIT: Forget my comment, the copernicus land cover dataset available from https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-land-cover appears to be a repackaged version of what i called ESA-CCI LC. They're the same.

~~Just a shout-out that in principle one could also switch to using the ESA-CCI land cover dataset which has global coverage: https://www.esa-landcover-cci.org/. It does not have different categories of build-on land.~~

~~http://maps.elie.ucl.ac.be/CCI/viewer/index.php for a quick view, http://maps.elie.ucl.ac.be/CCI/viewer/download/CCI-LC_Maps_Legend.pdf for a legend.~~

coroa avatar Mar 17 '22 11:03 coroa

@fneum do you still need someone to test run this on a fresh pypsa-eur clone? Could do it this week

energyLS avatar Jan 30 '23 17:01 energyLS

would be great! Thanks @energyLS !

fneum avatar Jan 30 '23 17:01 fneum

Successful. Steps I have done: Fresh repo: git clone https://github.com/PyPSA/pypsa-eur.git Switch to branch ukraine_hackathon Changed config to clusters: [39, 128] and opts: [Co2L-144H] to reduce solving time Run: snakemake -c1 solve_all_networks

Worked all fine, did some additional plotting of the solved network and looks good. Ready to merge from my point of view @fneum

energyLS avatar Feb 01 '23 15:02 energyLS

Just for reference. This PR is merged because of ENTSO-E news:

"Continental European TSOs announce completion of synchronisation project with Ukrenergo and significant increase in export capacity from Continental Europe to Ukraine. https://www.entsoe.eu/news/2023/11/28/continental-european-tsos-announce-completion-of-synchronisation-project-with-ukrenergo-and-significant-increase-in-export-capacity-from-continental-europe-to-ukraine/" :eu:

pz-max avatar Dec 11 '23 17:12 pz-max