pypsa-eur icon indicating copy to clipboard operation
pypsa-eur copied to clipboard

Eurostat 2023 data for energy totals

Open toniseibold opened this issue 1 year ago • 3 comments

Changes proposed in this Pull Request

Reads in the latest eurostat report from 2023 and uses it for building energy totals. Since the JRC IDEES data is only available for 2015, the energy balances are scaled by the factor of eurostat[report_year]/eurostat[2015] data as well. The latest swiss energy totals are added to ensure consistent energy totals.

To complement the PR, please add the eurostat data from 2023 to the zenodo data.

Missing is only the passenger car efficiency and the number of passenger cars that are initially in the JRC IDEES but not in the eurostat data set.

Comparison between 2023 and 2016 reports for the year 2013 show no difference except for Norway and Switzerland. energy_totals_comparison.csv

Checklist

  • [x] I tested my contribution locally and it seems to work fine.
  • [x] Code and workflow changes are sufficiently documented.
  • [x] Changed dependencies are added to envs/environment.yaml.
  • [x] Changes in configuration options are added in all of config.default.yaml.
  • [x] Changes in configuration options are also documented in doc/configtables/*.csv.
  • [ ] A release note doc/release_notes.rst is added.

toniseibold avatar Feb 23 '24 12:02 toniseibold

@toniseibold great! Just a few remarks:

  1. in the long-term we would like to move away from the Zenodo sector-bundle repository. Can you add to the retrieve_sector_databundle.py some lines which download the necessary files from Eurostat directly?
  2. The new data set is still also containing the previous years and the UK, so there is no need to keep the old ones, right? This simplifies the script, you can use as an input only the new one an remove the if-else statement depending on which Eurostat input you use.
  3. Can you do some comparison of the output files between previous Eurostat and the new?

lisazeyen avatar Feb 26 '24 06:02 lisazeyen

  1. in the long-term we would like to move away from the Zenodo sector-bundle repository. Can you add to the retrieve_sector_databundle.py some lines which download the necessary files from Eurostat directly?

I think, a separate rule retrieve_eurostat would be better to have a clear distinction

FabianHofmann avatar Feb 26 '24 09:02 FabianHofmann

@toniseibold Have you done some comparison of the output files between previous Eurostat and the new? Can you plot it here or confirm that the outputs are similar? The CI is failing for the script build_energy_totals.py can you make sure that it is running through? It seems to be connected with the swiss data and the year 2019

lisazeyen avatar Feb 28 '24 14:02 lisazeyen

Checked output of energy_totals.csv before and after and value changes are in an acceptable range.

fneum avatar Mar 05 '24 17:03 fneum