Write access to dark storage

[oyvindeide]

We would like to be able to write directly from ert to dark storage which is currently read only. That will require some refactoring of the way ert stores data today.

To achive that we need to refactor how ert interfaces with the storage system currently, where the main contact points are the callbacks and the saving/loading of parameters. While doing this we would like to add meta data to the stored information which will in the future allow us to:

• Only require that summary vectors have responses at dates where where there are observations, and not all aligned
• Simplify loading parameters and responses

We will make the storage solution independend from EnKFMain, and will require:

• Test performance of off-the-shelf file formats for saving loading
• Refactor how parameters are sampled and written
• Refactor loading and saving parameters in the analysis module

• [x] https://github.com/equinor/ert/issues/3398
• [x] https://github.com/equinor/ert/issues/3397
• [x] https://github.com/equinor/ert/issues/3396
• [x] https://github.com/equinor/ert/issues/3377
• [x] https://github.com/equinor/ert/issues/3389
• [x] https://github.com/equinor/ert/issues/3217
• [ ] #3465
• [x] #3485
• [x] #3486
• [ ] #3487
• [ ] #3488
• [x] #3516
• [ ] #3515
• [x] #3514

First iteration datamodel:

graph TD
    Experiment
    Experiment --- Observations
    Observations --- Observations_transformation
    Experiment --- Ensemble
    Ensemble --- Record
    Record --- Record_info
    Ensemble --> Update
    Update --> Ensemble
    Observations_transformation --- Update