open-MaStR icon indicating copy to clipboard operation
open-MaStR copied to clipboard

Split postprocessing into data cleansing and enrichment

Open gplssm opened this issue 3 years ago • 1 comments

Data cleansing

  • Unplausible nominal power correction
  • Filtering duplicates (expects raw data including all StatistikFlag A and B units)

Enrichment

  • Add geom column and fill with data from lat/lon and PLZ

By doing this, parts of the code might be translated to python if that eases the process (as done for geom column creation).

gplssm avatar Mar 01 '21 17:03 gplssm

With v 0.11.0, cleansing is done when reading data into the database. Seperately, postprocessing scripts can then be applied later. They can be found in the github repository, postprocessing folder.

FlorianK13 avatar May 03 '22 13:05 FlorianK13