equinor
Validate results after SingleTestRun/EnsembleExperiment
Sometimes it is a bit hard for users to tell if all the required observations and parameters are loaded in the callback. In AHM we allow some observations to be missing, and only crash if all observations are deactivated.
The REFCASE has some functionality to check that the correct dates are present for summary data, however there has historically been a problem that it is not necessarily in sync with the schedule file, and as such, might give false positives or negatives.
One way we can improve this, is by validating that SingleTestRun and EnsembleExperiment validate the results for missing parameters and responses. This can be as simple as raising an exception when it has completed if something is missing, and everything should show up correctly in the gui.
We can do something similar to here: https://github.com/equinor/ert/blob/7fc50c7d1ff401e684a04bb8b8b6cd1f62d08a60/src/ert/analysis/_es_update.py#L153, i.e. load all the observations and check if any are missing, similar to what we do in the update report: https://github.com/equinor/ert/blob/7fc50c7d1ff401e684a04bb8b8b6cd1f62d08a60/src/ert/analysis/snapshots.py#L37
We already validate that the forward model is running as expected, with this we also validate all the observations.
We should get some feedback on forward_model_ok state when hovering over the realization. Let's see if this issue is still valid.
Nevertheless this does not check whether it contains the correct data. Next step: contact @oyvindeide for more info.
So take snake oil as an example, it has summary data as responses, and if some of them are missing with an update algorithm you get this (note the Deactivated: missing response):
Can be reproduced by adding:
if day in [76,77,78,79,80]:
continue
in: test-data/ert/snake_oil/forward_models/snake_oil_simulator.py
and running snake oil with smoothing.
The goal of this issue is that when running single_test_run and ensemble_experiment you would also be notified that these observations are missing.
This requires some more explanation what the actual warning (after running testrun and ensemble experiment) should be.