gridpath icon indicating copy to clipboard operation
gridpath copied to clipboard

Hydro horizons missing from temporal causes tab file to disappear

Open misyntropy opened this issue 1 year ago • 0 comments

Relevant to #819 and ultimately #6 .

gen_hydro projects are linked to horizons in inputs_project_hydro_operational_characteristics. The way hydro data is read from the database is to pick the intersection of these horizons, with those specified in the inputs_temporal_horizons_timepoints table : https://github.com/blue-marble/gridpath/blob/ad4ffd1d066dc76797fc2f2d755feff671e392e8/gridpath/project/operations/operational_types/common_functions.py#L824-L835

When horizons are missing from the temporal table, the intersection is also the null set and therefore the tab files aren't even written: https://github.com/blue-marble/gridpath/blob/ad4ffd1d066dc76797fc2f2d755feff671e392e8/gridpath/project/operations/operational_types/common_functions.py#L446-L449

This results in a file not found error:

ERROR:root:[Errno 2] No such file or directory: '../scenarios_etc-dev/2horizons_w_hydro_w_balancing_types/inputs/hydro_conventional_horizon_params.tab' and FileNotFoundError: [Errno 2] No such file or directory: '../scenarios_etc-dev/2horizons_w_hydro_w_balancing_types/inputs/hydro_conventional_horizon_params.tab' .

I see three equally valid ways to handle this:

  1. Make sure all empty files are written, with (perhaps?) the result that the hydro projects are never simulated. This aligns with existing practice of silently dropping mismatched horizons.
  2. Make mismatched horizons an error, and use LEFT JOIN instead of INNER JOIN while fetching the hydro opchar data. This entails a lot of changes throughout the test suite.
  3. Keep existing behaviour as by-design.

I am in the process of writing validation to catch this case, but before that wanted to understand if current operational behaviour is intended.

misyntropy avatar Feb 27 '24 05:02 misyntropy