open-MaStR icon indicating copy to clipboard operation
open-MaStR copied to clipboard

Key error `geloeschteunddeaktivierteeinheiten`

Open chrwm opened this issue 2 years ago • 9 comments

I get a key error when I download with bulk from scratch while parsing

Table 'hydro_extended' is filled with data 'einheitenwasser' from the bulk download.
File 'EinheitenWasser.xml' is parsed.
Data is cleansed.
Table 'wind_extended' is filled with data 'einheitenwind' from the bulk download.
File 'EinheitenWind.xml' is parsed.
Data is cleansed.
Traceback (most recent call last):
  File "/home/cwm/Python/open-mastr/main.py", line 72, in <module>
    db.download(method="bulk", data=data_bulk, date=bulk_date, bulk_cleansing=True)
  File "/home/cwm/Python/open-mastr/open_mastr/mastr.py", line 219, in download
    write_mastr_xml_to_database(
  File "/home/cwm/Python/open-mastr/open_mastr/xml_download/utils_write_to_database.py", line 33, in write_mastr_xml_to_database
    if is_table_relevant(
  File "/home/cwm/Python/open-mastr/open_mastr/xml_download/utils_write_to_database.py", line 75, in is_table_relevant
    tablename_mapping[xml_tablename]["__class__"] is not None
KeyError: 'geloeschteunddeaktivierteeinheiten'

Process finished with exit code 1


chrwm avatar Nov 03 '22 19:11 chrwm

Which bulk download date do you have?

FlorianK13 avatar Nov 04 '22 07:11 FlorianK13

Have you deleted your database before the download to get the newest data model?

FlorianK13 avatar Nov 04 '22 07:11 FlorianK13

Ok, I get the same error with the newest data model and todays bulk download. The error did not occur with the download from two weeks ago.

FlorianK13 avatar Nov 04 '22 07:11 FlorianK13

GeloeschteUndDekativierteEinheiten.xml is now a file that comes with the bulk download.

FlorianK13 avatar Nov 04 '22 07:11 FlorianK13

@deniztepe the column Einheitentyp is still filled with integer numbers, even though I put it into the replace_columns list. Can you check why this does not work?

FlorianK13 avatar Nov 04 '22 08:11 FlorianK13

There was a typo in columns_to_replace. Einheitentyp -> Einheittyp should resolve it. I am testing it currently.

deniztepe avatar Nov 04 '22 08:11 deniztepe

I'm testing it too

chrwm avatar Nov 04 '22 09:11 chrwm

There is still a mistake: Einheittyp is not to be replaced with values from Katalogwerte.

FlorianK13 avatar Nov 04 '22 10:11 FlorianK13

I got the actual values we can use to replace the numbers in Einheittyp from deleted_units:

Vielen Dank für den Hinweis. Wir werden den Katalog im entsprechenden XML Objekt ergänzen.

Folgende Werte Relationen sind möglich: Id ETY_Bezeichnung 1 Solareinheit 2 Windeinheit 3 Biomasse 4 Wasser 5 Geothermie 6 Verbrennung 7 Kernenergie 8 Stromspeichereinheit 9 Stromverbrauchseinheit 10 Gasverbrauchseinheit 11 Gaserzeugungseinheit 12 Gasspeichereinheit 13 Stromerzeugungseinheit

FlorianK13 avatar Nov 14 '22 09:11 FlorianK13