open-MaStR icon indicating copy to clipboard operation
open-MaStR copied to clipboard

"_extended" tabels empty for selected bulk download data

Open clwehner opened this issue 7 months ago • 5 comments

Description of the issue

First, thank you for developing this package. I have been using the latest 0.14.4 version of open_mastr in combination with a self hosted postgresql. When I do a bulk download to dump the latest data into my db, everything works fine for the wind_eeg and solar_eeg table. The wind_extended and solar_extended table on the other hand remain empty.

Steps to Reproduce

  1. init mastr
db = open_mastr.Mastr(engine=my_psql_engine)
  1. start bulk download
db.download(
        method=params["mastr_method"], # "bulk"
        data=params["mastr_data"], # ["wind","solar"]
        date=params["mastr_date"], # "today"
    )

Ideas of solution

I looked into the donwloaded Gesamtdatenexport_20240715.zip file and found the following .xml files. I reduced the numbered ones to save space. Going through your code, I missed EinheitenSolar.xml and EinheitenWind.xml in my extraced Gesamtdatenexport_20240715.zip.

Archive:  Gesamtdatenexport_20240715.zip
  inflating: mastr/AnlagenEegBiomasse.xml  
  inflating: mastr/AnlagenEegGeothermieGrubengasDruckentspannung.xml  
  inflating: mastr/AnlagenEegSolar_1.xml  
...
  inflating: mastr/AnlagenEegSpeicher_1.xml  
...
  inflating: mastr/AnlagenEegWasser.xml  
  inflating: mastr/AnlagenEegWind.xml  
  inflating: mastr/AnlagenGasSpeicher.xml  
  inflating: mastr/AnlagenKwk.xml    
  inflating: mastr/AnlagenStromSpeicher_1.xml  
...
  inflating: mastr/Bilanzierungsgebiete.xml  
  inflating: mastr/EinheitenAenderungNetzbetreiberzuordnungen.xml  
  inflating: mastr/EinheitenGasErzeuger.xml  
  inflating: mastr/EinheitenGasSpeicher.xml  
  inflating: mastr/EinheitenGasverbraucher.xml  
  inflating: mastr/EinheitenGenehmigung.xml  
  inflating: mastr/EinheitenStromVerbraucher.xml  
  inflating: mastr/Einheitentypen.xml  
  inflating: mastr/Ertuechtigungen.xml  
  inflating: mastr/GeloeschteUndDeaktivierteEinheiten_1.xml  
  inflating: mastr/GeloeschteUndDeaktivierteEinheiten_2.xml  
  inflating: mastr/Katalogkategorien.xml  
  inflating: mastr/Katalogwerte.xml  
  inflating: mastr/Lokationen_1.xml  
...
  inflating: mastr/Marktakteure_1.xml  
...
  inflating: mastr/Marktrollen.xml   
  inflating: mastr/Netzanschlusspunkte_1.xml  
...
  inflating: mastr/Netze.xml

Any idea on what might went wrong?

Context and Environment

  • Version used: 0.14.04
  • Operating system: Ubuntu 22.04
  • Environment setup and (python) version: 3.10.12

Workflow checklist

clwehner avatar Jul 15 '24 13:07 clwehner