DESC icon indicating copy to clipboard operation
DESC copied to clipboard

Reduce testing time/memory

Open f0uriest opened this issue 1 year ago • 2 comments

Running our basic unit tests now takes 1.5+ hrs for me locally, and fairly often it will get killed due to OOM issues if I'm trying to do anything else at the same time.

A few suggestions:

  • Reduce resolution as much as possible wherever possible
  • Drop resolution after doing desc.examples.get since in most cases we don't need all the detail
  • for VMEC IO tests, make new baseline cases with like, 10 surfaces, since all we're testing is that things get loaded and saved correctly. We already know DESC and VMEC both work.
  • See if we can eliminate some tests are are now redundant with others. There's ways to see which tests cover which lines, so we can cut tests that don't cover new ground: https://pytest-cov.readthedocs.io/en/latest/contexts.html

f0uriest avatar Feb 29 '24 07:02 f0uriest

one could combine all these tests into one function, computing everything at once to save time: https://github.com/PlasmaControl/DESC/blob/664a5622fdee06b4e785c5737549ff5ee2124cd4/tests/test_stability_funs.py#L71 https://github.com/PlasmaControl/DESC/blob/664a5622fdee06b4e785c5737549ff5ee2124cd4/tests/test_stability_funs.py#L174 https://github.com/PlasmaControl/DESC/blob/664a5622fdee06b4e785c5737549ff5ee2124cd4/tests/test_stability_funs.py#L127 etc.

unalmis avatar Feb 29 '24 19:02 unalmis

For reference, here's a breakdown of all the unit tests and how long they take https://gist.github.com/f0uriest/dc9f2b7633dae5b84bd0d5d69e74a2f5

total time is over 2hrs

f0uriest avatar Mar 06 '24 06:03 f0uriest

https://github.com/PlasmaControl/DESC/blob/95612fa625bd5ba939569004c767ed02594586f0/tests/test_objective_funs.py#L984

we could make this test use the conftest coils instead of making new ones, which is more expensive bc of intersection checks

dpanici avatar Nov 18 '24 20:11 dpanici

image Here are some of the longest regression(bottom) and unit(top) tests. Taken from one of the CI runs on December 3rd.

YigitElma avatar Dec 04 '24 07:12 YigitElma

maybe we could reduce the number of examples we have? or move some to "not tested" part of examples so it still exists but people know not to expect it to work all the time?

Or edit input file to run faster (avoid continuation method, reduce res,, etc)

dpanici avatar Dec 04 '24 18:12 dpanici

We don't do that many actual examples, I think its just DSHAPE, HELIOTRON and ATF. I don't think we can get rid of any of those without missing some important code paths

f0uriest avatar Dec 04 '24 22:12 f0uriest