OPM
[testing] tightening tolerance wells to be 1.e-8
Did not manage to check all the regression tests. For the many of the ones I checked, it does not look causing any running failures. And also, for many, it avoids some time step cuts.
Maybe it is time to tighten the well tolerance. But it should really depends on the tests for the field cases.
There are cases related to network also call for tighter tolerance of 1.e-8, and 1.e-6 (1.e-4 by default) was not sufficient.
Benchmark result overview:
| Test | Configuration | Relative |
|---|---|---|
| opm-git | OPM Benchmark: drogon - Threads: 1 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: drogon - Threads: 8 - FOPT (Total Oil Production At End Of Run) | 0.999 |
| opm-git | OPM Benchmark: punqs3 - Threads: 1 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: punqs3 - Threads: 8 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: smeaheia - Threads: 1 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: smeaheia - Threads: 8 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 1 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 8 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 1 - FOIT (Total Oil Injection At End Of Run) | 1.01 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 8 - FOIT (Total Oil Injection At End Of Run) | 0.998 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 1 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 8 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 1 - FOPT (Total Oil Production At End Of Run) | 1 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 8 - FOPT (Total Oil Production At End Of Run) | 0.978 |
- Speed-up = Total time master / Total time pull request. Above 1.0 is an improvement. *
View result details @ https://www.ytelses.com/opm/?page=result&id=2435
Benchmark result overview:
| Test | Configuration | Relative |
|---|---|---|
| opm-git | OPM Benchmark: drogon - Threads: 1 | 0.995 |
| opm-git | OPM Benchmark: drogon - Threads: 8 | 1.014 |
| opm-git | OPM Benchmark: punqs3 - Threads: 1 | 1.024 |
| opm-git | OPM Benchmark: punqs3 - Threads: 8 | 0.997 |
| opm-git | OPM Benchmark: smeaheia - Threads: 1 | 1.022 |
| opm-git | OPM Benchmark: smeaheia - Threads: 8 | 1.008 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 1 | 1.003 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 8 | 1 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 1 | 1.023 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 8 | 0.998 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 1 | 1.002 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 8 | 0.981 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 1 | 1.001 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 8 | 0.985 |
- Speed-up = Total time master / Total time pull request. Above 1.0 is an improvement. *
View result details @ https://www.ytelses.com/opm/?page=result&id=2480
If the benchmark is reliable, it looks that we do not need to worry about the performance impact.
Benchmark result overview:
| Test | Configuration | Relative |
|---|---|---|
| opm-git | OPM Benchmark: drogon - Threads: 1 | 0.997 |
| opm-git | OPM Benchmark: drogon - Threads: 8 | 1.025 |
| opm-git | OPM Benchmark: punqs3 - Threads: 1 | 1.002 |
| opm-git | OPM Benchmark: punqs3 - Threads: 8 | 0.997 |
| opm-git | OPM Benchmark: smeaheia - Threads: 1 | 1.02 |
| opm-git | OPM Benchmark: smeaheia - Threads: 8 | 1.006 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 1 | 0.97 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 8 | 0.998 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 1 | 1.023 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 8 | 1.004 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 1 | 0.998 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 8 | 0.99 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 1 | 1.002 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 8 | 0.999 |
- Speed-up = Total time master / Total time pull request. Above 1.0 is an improvement. *
View result details @ https://www.ytelses.com/opm/?page=result&id=2481
checking how the regression looks like after the massive update of the reference in opm-tests.
If this PR can go in, we will be able to simplify some logic for the zero rate/stoppped wells. And there are some tests calling for tighter tolerance.
Will trigger a jenkins test to evaluate the efforts needed.
Test Result (195 failures / +195) mpi.compareECLFiles_flowexp_blackoil+SPE1CASE2 mpi.compareECLFiles_flow+SPE1CASE2_2P mpi.compareECLFiles_flow+SPE1CASE1 mpi.compareECLFiles_flow+SPE1CASE1_IMPORT mpi.compareECLFiles_flow+SPE1CASE2_THERMAL mpi.compareECLFiles_flow+SPE1CASE2_THERMAL_WATVISC mpi.compareECLFiles_flow+SPE1CASE1_BRINE mpi.compareECLFiles_flow+SPE1CASE1_PRECSALT mpi.compareECLFiles_flow+SPE1CASE2_ROCK2DTR mpi.compareECLFiles_flow+NETWORK-01 Show all failed tests >>>
It still looks the same. Will evaluate when finding time. And adding the code that can be removed due to this change to the PR.
Benchmark result overview:
| Test | Configuration | Relative |
|---|---|---|
| opm-git | OPM Benchmark: drogon - Threads: 1 | 0.974 |
| opm-git | OPM Benchmark: drogon - Threads: 8 | 0.855 |
| opm-git | OPM Benchmark: punqs3 - Threads: 1 | 0.986 |
| opm-git | OPM Benchmark: punqs3 - Threads: 8 | 0.998 |
| opm-git | OPM Benchmark: smeaheia - Threads: 1 | 0.976 |
| opm-git | OPM Benchmark: smeaheia - Threads: 8 | 0.905 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 1 | 1.018 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 8 | 1.011 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 1 | 1 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 8 | 0.94 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 1 | 0.983 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 8 | 0.96 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 1 | 0.978 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 8 | 0.937 |
- Speed-up = Total time master / Total time pull request. Above 1.0 is an improvement. *
View result details @ https://www.ytelses.com/opm/?page=result&id=2591
The benchmark looks very different from the previous ones. Will trigger again to confirm.
Benchmark result overview:
| Test | Configuration | Relative |
|---|---|---|
| opm-git | OPM Benchmark: drogon - Threads: 1 | 0.849 |
| opm-git | OPM Benchmark: drogon - Threads: 8 | 0.985 |
| opm-git | OPM Benchmark: punqs3 - Threads: 1 | 0.992 |
| opm-git | OPM Benchmark: punqs3 - Threads: 8 | 0.998 |
| opm-git | OPM Benchmark: smeaheia - Threads: 1 | 0.993 |
| opm-git | OPM Benchmark: smeaheia - Threads: 8 | 0.991 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 1 | 1.016 |
| opm-git | OPM Benchmark: spe10_model_1 - Threads: 8 | 0.999 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 1 | 0.949 |
| opm-git | OPM Benchmark: flow_mpi_extra - Threads: 8 | 1.014 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 1 | 0.987 |
| opm-git | OPM Benchmark: flow_mpi_norne - Threads: 8 | 0.993 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 1 | 0.998 |
| opm-git | OPM Benchmark: flow_mpi_norne_4c_msw - Threads: 8 | 0.974 |
- Speed-up = Total time master / Total time pull request. Above 1.0 is an improvement. *
View result details @ https://www.ytelses.com/opm/?page=result&id=2592
the benchmark appears not very stable. If needed, maybe we should test the cases on own machine to check the iteration numbers.