SCLOPF intermittently allows post-contingency line overloads (old "flaky" test issue)

[Irieo]

Version Checks (indicate both or one)

• [ ] I have confirmed this bug exists on the lastest release of PyPSA.

• [x] I have confirmed this bug exists on the current master branch of PyPSA.

Issue Description

See test_sclopf_scigrid.py and SCLOPF code in abstract.py

Security‑constrained optimization on the SciGrid-DE example (with two line outages) sometimes yields LPF contingency flows exceeding thermal limits by up to 7% (e.g. Line 673: 1.073 p.u.).

FAILED test/test_sclopf_scigrid.py::test_optimize_security_constrained - AssertionError: Arrays are not almost equal to 4 decimals Mismatched elements: 2 / 3 (66.7%) Max absolute difference among violations: 0.07296842 Max relative difference among violations: 0.07296842 ACTUAL: array([1. , 1.0246, 1.073 ]) DESIRED: array([1., 1., 1.])

Failures are intermittent and appear irregularly. Rerunning it locally with ubuntu & python 3.13 yields 1 in 30 fails, always with the same overload values.

The big problem here is that it breaks CI, also on master of PyPSA.

Reproduce: pytest -q test/test_sclopf_scigrid.py::test_optimize_security_constrained long enough or run the bash script below with bash test.py N for N runs.

A degeneracy of solution mostly likely plays a role here, but after some checks I think it is that degeneracy highlights there is a problem in core code of abstract.py, rather than degeneracy causes the problem.

It requires a proper look. Maybe @coroa @FabianHofmann

#!/bin/bash

# Check if number of runs is provided
if [ $# -eq 0 ]; then
    echo "Usage: $0 <number_of_runs>"
    exit 1
fi

RUNS=$1
PASSED=0
FAILED=0

echo "Running test $RUNS times..."
echo "================================"

for i in $(seq 1 $RUNS); do
    echo -n "Run $i/$RUNS: "
    
    if uv run pytest test/test_sclopf_scigrid.py::test_optimize_security_constrained -q 2>/dev/null; then
        echo "✓ PASSED"
        ((PASSED++))
    else
        echo "✗ FAILED"
        ((FAILED++))
    fi
done

echo "================================"
echo "Results:"
echo "  Passed: $PASSED/$RUNS"
echo "  Failed: $FAILED/$RUNS"
echo "  Success rate: $(echo "scale=2; $PASSED * 100 / $RUNS" | bc)%"

Reproducible Example

pytest -q test/test_sclopf_scigrid.py::test_optimize_security_constrained

Expected Behavior

passed tests

Installed Versions

Replace this line.

[coroa]

I had a first look and can reproduce a failing test intermittently.

Observations:

1. I am not able to reproduce the error outside of the test environment, ie. running the exact same code the test uses repeatedly in a script, even after 200 iterations i never produced an error. @lkstrp are special pypsa.options settings in effect during testing that are not active otherwise?
2. The lp file and the sol file created in a faulty run are identical to the files in a successful one; ie. this has nothing to do with degeneracy.
3. I was able to export the networks during a faulty run and am trying to compare them now (unfortunately the importer is not happy with sclopf solved networks because they also have an additional dimension for some dual variables that is not called scenario :)).

Will give it another go in a couple of days.

[coroa]

Unfortunately the new components structure is unpickle-able either, @lkstrp . Probably needs __getstate__, __setstate__ definitions in one or two places to fix.

[lkstrp]

I am not able to reproduce the error outside of the test environment, ie. running the exact same code the test uses repeatedly in a script, even after 200 iterations i never produced an error. @lkstrp are special pypsa.options settings in effect during testing that are not active otherwise?

If not explicitly called in the test, just in conf.test:

pypsa.options.debug.runtime_verification = True

...

def pytest_configure(config):
    """Configure pytest session with custom options."""
    if config.getoption("--new-components-api"):
        pypsa.options.api.new_components_api = True

Unfortunately the new components structure is unpickle-able either, @lkstrp . Probably needs getstate, setstate definitions in one or two places to fix.

I have somewhere on my list to move io to components level, but we can fix that quickly and make them pickable before that

[coroa]

I did not use the --new-components-api flag, so it is just the pypsa.options.debug.runtime_verification one. Will test later.

[Cellophil]

Just a wild hunch, one thing that is not necessarily deterministic about sclopf is the bodf calculation. If it's a test network that has lines that split the system the bodf can deliver 0, nan or anything "in between". If it's a well behaved network that cannot be split please ignore!