Antares_Simulator icon indicating copy to clipboard operation
Antares_Simulator copied to clipboard

Random failures in windows CI in adequacy patch tests

Open sylvlecl opened this issue 1 year ago • 1 comments

Description From time to time, some adequacy patch tests fail on windows CI, due to differences in results compared to reference.

We should investigate where those instabilities could come from.

See logs in step "Run tests for adequacy patch":

1/1 Test #2: json .............................***Failed   10.43 sec
============================= test session starts =============================
platform win32 -- Python 3.12.0, pytest-7.4.3, pluggy-1.3.0 -- C:\hostedtoolcache\windows\Python\3.12.0\x64\python.exe
cachedir: .pytest_cache
rootdir: D:\a\Antares_Simulator\Antares_Simulator\src\tests\run-study-tests
configfile: pytest.ini
plugins: xdist-3.3.1
collecting ... collected 6 items / 4 deselected / 2 selected

test_from_json.py::test[test_1  (adequacy-patch-CSR / adq-patch-CSR-test-case-v01)] 
Running the study
Running check : output compare
Running check : integrity compare
PASSED
test_from_json.py::test[test_1  (adequacy-patch-CSR / adq-patch-CSR-test-case-v02)] 
Running the study
Running check : output compare

mc-all\links\areain-1 - areain-2\values-annual.txt : CONG. FEE (ABS.).1
   reference  results  diff
0        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-annual.txt : CONG. FEE (ALG.).1
   reference  results  diff
0        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-daily.txt : CONG. FEE (ABS.).1
   reference  results  diff
7        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-daily.txt : CONG. FEE (ABS.).3
   reference   results  diff
7   119980.0  120000.0  20.0

mc-all\links\areain-1 - areain-2\values-daily.txt : CONG. FEE (ALG.).1
   reference  results  diff
7        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-daily.txt : CONG. FEE (ALG.).2
   reference   results  diff
7  -119980.0 -120000.0 -20.0

mc-all\links\areain-1 - areain-2\values-hourly.txt : CONG. FEE (ABS.).1
     reference  results  diff
168        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-hourly.txt : CONG. FEE (ABS.).3
     reference   results  diff
168   119980.0  120000.0  20.0

mc-all\links\areain-1 - areain-2\values-hourly.txt : CONG. PROB +
     reference  results  diff
168        0.0     50.0  50.0

mc-all\links\areain-1 - areain-2\values-hourly.txt : CONG. FEE (ALG.).1
     reference  results  diff
168        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-hourly.txt : CONG. FEE (ALG.).2
     reference   results  diff
168  -119980.0 -120000.0 -20.0

mc-all\links\areain-1 - areain-2\values-monthly.txt : CONG. FEE (ABS.).1
   reference  results  diff
0        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-monthly.txt : CONG. FEE (ALG.).1
   reference  results  diff
0        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-weekly.txt : CONG. FEE (ABS.).1
   reference  results  diff
1        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-weekly.txt : CONG. FEE (ABS.).3
   reference   results  diff
1   119980.0  120000.0  20.0

mc-all\links\areain-1 - areain-2\values-weekly.txt : CONG. FEE (ALG.).1
   reference  results  diff
1        0.0     10.0  10.0

mc-all\links\areain-1 - areain-2\values-weekly.txt : CONG. FEE (ALG.).2
   reference   results  diff
1  -119980.0 -120000.0 -20.0

00002\links\areain-1 - areain-2\values-daily.txt : CONG. FEE (ABS.)
   reference   results  diff
7   119980.0  120000.0  20.0

00002\links\areain-1 - areain-2\values-daily.txt : CONG. FEE (ALG.)
   reference   results  diff
7  -119980.0 -120000.0 -20.0

00002\links\areain-1 - areain-2\values-hourly.txt : CONG. FEE (ABS.)
     reference   results  diff
168   119980.0  120000.0  20.0

00002\links\areain-1 - areain-2\values-hourly.txt : CONG. FEE (ALG.)
     reference   results  diff
168  -119980.0 -120000.0 -20.0

00002\links\areain-1 - areain-2\values-hourly.txt : CONG. PROB +
     reference  results   diff
168        0.0    100.0  100.0

00002\links\areain-1 - areain-2\values-weekly.txt : CONG. FEE (ABS.)
   reference   results  diff
1   119980.0  120000.0  20.0

00002\links\areain-1 - areain-2\values-weekly.txt : CONG. FEE (ALG.)
   reference   results  diff
1  -119980.0 -120000.0 -20.0
FAILED

OS and version

  • OS: windows
  • Antares Simulator version: 8.8

sylvlecl avatar Oct 30 '23 09:10 sylvlecl