ert icon indicating copy to clipboard operation
ert copied to clipboard

Combine datasets & refactor observations to response configs

Open yngve-sk opened this issue 1 year ago • 2 comments

squash of https://github.com/equinor/ert/pull/7495

Requires change in semeio: https://github.com/equinor/semeio/pull/609

yngve-sk avatar Apr 04 '24 11:04 yngve-sk

Codecov Report

Attention: Patch coverage is 93.44262% with 56 lines in your changes are missing coverage. Please review.

Project coverage is 85.90%. Comparing base (140e5d7) to head (b2a7932). Report is 8 commits behind head on main.

Files Patch % Lines
src/ert/storage/local_ensemble.py 95.73% 11 Missing :warning:
src/ert/storage/migration/to6.py 90.69% 8 Missing :warning:
src/ert/config/observations.py 90.00% 7 Missing :warning:
src/ert/libres_facade.py 40.00% 6 Missing :warning:
src/ert/config/summary_config.py 94.18% 5 Missing :warning:
src/ert/config/obs_commons.py 94.02% 4 Missing :warning:
src/ert/dark_storage/common.py 90.69% 4 Missing :warning:
src/ert/dark_storage/endpoints/records.py 71.42% 4 Missing :warning:
src/ert/config/gen_data_config.py 95.83% 3 Missing :warning:
src/ert/config/commons.py 94.44% 1 Missing :warning:
... and 3 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7563      +/-   ##
==========================================
+ Coverage   85.76%   85.90%   +0.13%     
==========================================
  Files         379      381       +2     
  Lines       23117    23441     +324     
  Branches      624      630       +6     
==========================================
+ Hits        19827    20136     +309     
- Misses       3209     3232      +23     
+ Partials       81       73       -8     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Apr 10 '24 08:04 codecov-commenter

Known issues 2fix:

  • Combination is not done after final iteration the way it is done now, due to no update
  • (On local MacOS M1) Running performance_utils example takes about 1h20min, combining datasets about 10mins per iteration (being with 200 * 3400 gen_data entries, and 200 summary files at 1.6MB each). Memory usage at most around 1.8GB during the update.

yngve-sk avatar Apr 12 '24 09:04 yngve-sk

closes #5065

eivindjahren avatar May 24 '24 10:05 eivindjahren