Speedup CI :magic_wand:

[kellertuer]

trafficstars

This is an idea to speed up CI, I did a small benchmark, see #144 and it seems from

https://github.com/JuliaDiff/ChainRules.jl/pull/499/files#diff-3b9314a6f9f2d7eec1d0ef69fa76cfabafdbe6d0df923768f9ec32f27a249c63R27-R32

one can set the compiler flags, though I am not sure this works after running a few (ambiguity & diff) tests. Let's see.

[codecov[bot]]

Codecov Report

Merging #507 (873e95e) into master (63e8ce6) will decrease coverage by 71.44%. The diff coverage is n/a.

:exclamation: Current head 873e95e differs from pull request most recent head be19c89. Consider uploading reports for the commit be19c89 to get more accurate results

@@             Coverage Diff             @@
##           master     #507       +/-   ##
===========================================
- Coverage   98.87%   27.43%   -71.45%     
===========================================
  Files          96       96               
  Lines        8646     8629       -17     
===========================================
- Hits         8549     2367     -6182     
- Misses         97     6262     +6165     

Impacted Files	Coverage Δ
src/groups/unitary.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/manifolds/Torus.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/manifolds/Circle.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/groups/heisenberg.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/groups/orthogonal.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/manifolds/Oblique.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/manifolds/Unitary.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/groups/connections.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/groups/power_group.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
src/manifolds/Elliptope.jl	0.00% <0.00%> (-100.00%)	:arrow_down:
... and 77 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[kellertuer]

Ok, this will end up as a looooooong guesswork of randomly combining things from other yaml files without me having any clue what I am doing.

[mateuszbaran]

That doesn't sound particularly encouraging :slightly_frowning_face:

[kellertuer]

Well I have two examples (the actual runtest action https://github.com/julia-actions/julia-runtest/blob/master/action.yml one can try to extract manual test runs from for example like https://github.com/timholy/SnoopCompile.jl/blob/3d34c0cc5d4ccdb7fc23e1e3627728f86ba55500/.github/workflows/ci.yml but since nothing is documented I could also try to brew some magic potion.

[kellertuer]

So to report some speedups (Just checked 1.7)

• Lie groups from 46 minutes to 24.5
• manifolds from 64 minutes to 30 minutes
• and the additional test is currently at 20 minutes (statistics and some manifolds that were faster on normal compilation=

[kellertuer]

So times are good, code coverage reports not yet 🙄

[mateuszbaran]

Cool, that's significantly faster.

[mateuszbaran]

I wonder if compilation mode affects coverage.

[kellertuer]

Hm, all 3 runs generate code coverage all 3 are uploaded but 2 seem to be ignored at codecov and just this newest group is reported. I have (again!) no clue why, not even where to start looking, since the logs report successful upload of all 3 reports.

[kellertuer]

I wonder if compilation mode affects coverage.

If it does then this day of work with magic here was completely useless.

[kellertuer]

So there were 3 uploads for the last commit to codecov I am just not sure whether the CI confuses something and reports 3 times the Lie group one, it seemed a little bit like that. Let's try again (I updated nightly to follow the same scheme).

[Affie]

Hi, don't know if you have seen this: https://discourse.julialang.org/t/implications-of-compile-min-and-optimize-0-for-dummies/56894 Hope it helps

[kellertuer]

Sure I am aware of that and as far as I can see it should not affect code cov.

[kellertuer]

I am out of ideas why 2 of 3 tests report to code coverage, they reach codecov.io but it does not count (at least there is three reports arriving at codecov) – so. If anyone has ideas left – otherwise this is just faster but useless.

[kellertuer]

It seems my manual starts of Julia (compared to the runtest action) does not generate the codecov files (that are collected in the next step), but I have no clue why, since I activate that.

Edit: For example the manifold test (on new compiler) should find files for Euclidean https://github.com/JuliaManifolds/Manifolds.jl/runs/7471335409?check_suite_focus=true#step:7:147

[kellertuer]

Last idea: I copied the Julia command from the log file (of the runtests action that reports tests) – now both call the same stuff, but also this does not seem to work.

So when even copying the command yields that the action does something different (though there is just an if that now runs two identical commands), I am really at the en of my ideas.

Any help welcome, otherwise, this will just stall for now.

[kellertuer]

I think I'll close this for now, since there is already one newer PR to work on a new CI.