Trixi.jl icon indicating copy to clipboard operation
Trixi.jl copied to clipboard

CI results are inconsistent for `DGMulti` MHD elixirs with `approximation_type=SBP()`

Open jlchan opened this issue 3 years ago • 5 comments

As noted in https://github.com/trixi-framework/Trixi.jl/pull/880, CI tests currently do not pass when running elixir_mhd_weak_blast_wave.jl with approximation_type=SBP(). To get tests to pass, a constant GLM cleaning speed was introduced in elixir_mhd_weak_blast_wave_SBP.jl.

However, the root cause of this issue is still unclear.

jlchan avatar Sep 23 '21 22:09 jlchan

Summary of #880:

  • A constant GLM cleaning speed does not help
  • Error-based step size control does not help reliably
  • Tests may pass locally but fail in some CI runs
  • With error-based step size control, the failing CI test used 44 time steps while I got 46 time steps locally
  • The problem still persists - #880 just disabled the tests that fail sometimes

ranocha avatar Sep 27 '21 05:09 ranocha

Todo: replace MHD with SWE for failing DGMulti SBP tests.

jlchan avatar Sep 27 '21 15:09 jlchan

Does reducing the error tolerances help? This was the case in https://github.com/trixi-framework/Trixi.jl/pull/889#issuecomment-929279443 and https://github.com/trixi-framework/Trixi.jl/pull/889#issuecomment-929287262. There, floating point differences lead to a similar behavior.

ranocha avatar Sep 29 '21 17:09 ranocha

It should, but the difference between computed/expected errors in failing tests was larger than 1e-7. https://github.com/trixi-framework/Trixi.jl/runs/3695911625?check_suite_focus=true gives

elixir_mhd_weak_blast_wave_SBP.jl (Quad): Test Failed at /home/runner/work/Trixi.jl/Trixi.jl/test/test_trixi.jl:54
  Expression: isapprox(l2_expected, l2_actual, atol = 1.1102230246251565e-13, rtol = 1.4901161193847656e-8)
   Evaluated: isapprox(0.20037359518825335, 0.19913012207747086; atol = 1.1102230246251565e-13, rtol = 1.4901161193847656e-8)

That should be a relative error of around 1e-3, which seems like it would be pretty high for a tolerance?

jlchan avatar Sep 29 '21 17:09 jlchan

Yes, indeed...

ranocha avatar Sep 29 '21 17:09 ranocha