simsopt Use ubunut 24.04 in place of other ubuntu runners

Apr 25 '25 14:04 mbkumar

Putting the failing tests from CI here for future reference.

======================================================================
ERROR: test_boozer_surface_optimisation_convergence (geo.test_boozersurface.BoozerSurfaceTests) (surfacetype='SurfaceXYZTensorFourier', stellsym=True, optimize_G=True, second_stage='newton_exact', get_data=<function get_ncsx_data at 0x7f6aea688b80>, vectorize=False)
Test to verify the various optimization algorithms that compute
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 238, in test_boozer_surface_optimisation_convergence
    self.subtest_boozer_surface_optimisation_convergence(
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 304, in subtest_boozer_surface_optimisation_convergence
    res = boozer_surface.minimize_boozer_exact_constraints_newton(
  File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/site-packages/simsopt/geo/boozersurface.py", line 641, in minimize_boozer_exact_constraints_newton
    dx = np.linalg.solve(A, b)
  File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/site-packages/numpy/linalg/_linalg.py", line 410, in solve
    r = gufunc(a, b, signature=signature)
  File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/site-packages/numpy/linalg/_linalg.py", line 104, in _raise_linalgerror_singular
    raise LinAlgError("Singular matrix")
numpy.linalg.LinAlgError: Singular matrix

======================================================================
FAIL: test_boozer_surface_optimisation_convergence (geo.test_boozersurface.BoozerSurfaceTests) (surfacetype='SurfaceXYZTensorFourier', stellsym=True, optimize_G=True, second_stage='newton_exact', get_data=<function get_hsx_data at 0x7f6aea688dc0>, vectorize=False)
Test to verify the various optimization algorithms that compute
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 238, in test_boozer_surface_optimisation_convergence
    self.subtest_boozer_surface_optimisation_convergence(
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 311, in subtest_boozer_surface_optimisation_convergence
    assert res['success']
AssertionError

======================================================================
FAIL: test_boozer_surface_optimisation_convergence (geo.test_boozersurface.BoozerSurfaceTests) (surfacetype='SurfaceXYZTensorFourier', stellsym=True, optimize_G=True, second_stage='newton', get_data=<function get_ncsx_data at 0x7f6aea688b80>, vectorize=False)
Test to verify the various optimization algorithms that compute
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 238, in test_boozer_surface_optimisation_convergence
    self.subtest_boozer_surface_optimisation_convergence(
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 311, in subtest_boozer_surface_optimisation_convergence
    assert res['success']
AssertionError

======================================================================
FAIL: test_boozer_surface_optimisation_convergence (geo.test_boozersurface.BoozerSurfaceTests) (surfacetype='SurfaceXYZTensorFourier', stellsym=True, optimize_G=True, second_stage='newton', get_data=<function get_giuliani_data at 0x7f6aea688af0>, vectorize=False)
Test to verify the various optimization algorithms that compute
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 238, in test_boozer_surface_optimisation_convergence
    self.subtest_boozer_surface_optimisation_convergence(
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 311, in subtest_boozer_surface_optimisation_convergence
    assert res['success']
AssertionError

======================================================================
FAIL: test_boozer_surface_optimisation_convergence (geo.test_boozersurface.BoozerSurfaceTests) (surfacetype='SurfaceXYZFourier', stellsym=True, optimize_G=False, second_stage='ls', get_data=<function get_ncsx_data at 0x7f6aea688b80>, vectorize=False)
Test to verify the various optimization algorithms that compute
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 238, in test_boozer_surface_optimisation_convergence
    self.subtest_boozer_surface_optimisation_convergence(
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 312, in subtest_boozer_surface_optimisation_convergence
    assert not boozer_surface.surface.is_self_intersecting(thetas=100)
AssertionError

======================================================================
FAIL: test_convergence_cpp_and_notcpp_same (geo.test_boozersurface.BoozerSurfaceTests)
This unit test verifies that that the cpp and not cpp implementations converge to
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 392, in test_convergence_cpp_and_notcpp_same
    x_nonvec = self.subtest_convergence_cpp_and_notcpp_same(False)
  File "/home/runner/work/simsopt/simsopt/tests/geo/test_boozersurface.py", line 426, in subtest_convergence_cpp_and_notcpp_same
    assert res['success']
AssertionError

May 05 '25 15:05 mbkumar

Happy to get this over the finish line, but what is the advantage of this change?

Seems like it addresses (https://github.com/hiddenSymmetries/simsopt/pull/467) although I don't think there is understanding still of why different ubuntu versions break certain tests in the code?

May 12 '25 14:05 akaptano

Happy to get this over the finish line, but what is the advantage of this change?

No advantage, but a necessity. ubuntu-latest points to ubuntu 24.04 on Github runners where the CI was failing. On CI, we changed the runners back to 22.04. But we have to transition to Ubuntu 24.04 eventually when 22.04 reaches EOL in 2027. Many will install 24.04 even before that and we want to have no test failures.

May 12 '25 14:05 mbkumar

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 92.02%. Comparing base (706cb1d) to head (f63dfc8).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #486   +/-   ##
=======================================
  Coverage   92.02%   92.02%           
=======================================
  Files          82       82           
  Lines       16015    16015           
=======================================
  Hits        14738    14738           
  Misses       1277     1277

Flag	Coverage Δ
unittests	`92.02% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

:snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

May 12 '25 15:05 codecov[bot]

@mbkumar weren't the failing tests happening on Ubuntu 24.04? why would forcing 24.04 fix the issue?

May 12 '25 19:05 andrewgiuliani

We are not forcing 24.04 on people. It is the other way, people want to run simsopt on 24.04. So we need to resolve the issues. This PR is to address that.

On Mon, May 12, 2025 at 3:59 PM Andrew Giuliani @.***> wrote:

andrewgiuliani left a comment (hiddenSymmetries/simsopt#486) https://github.com/hiddenSymmetries/simsopt/pull/486#issuecomment-2873821658

@mbkumar https://github.com/mbkumar weren't the failing tests happening on Ubuntu 24.04? why would forcing 24.04 fix the issue?

— Reply to this email directly, view it on GitHub https://github.com/hiddenSymmetries/simsopt/pull/486#issuecomment-2873821658, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA62VEBGAP3GRDUYKGEZGW326D4SPAVCNFSM6AAAAAB33ZHE2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNZTHAZDCNRVHA . You are receiving this because you were mentioned.Message ID: @.***>

May 12 '25 20:05 mbkumar