openmc icon indicating copy to clipboard operation
openmc copied to clipboard

Swap test order

Open gonuke opened this issue 1 year ago • 2 comments

Description

Change the directory name of python unit tests so that they run before regression tests. This assumes that the test order is determined by the alphabetical order of the test directories.

Fixes #2828

Checklist

  • [x] I have performed a self-review of my own code
  • [x] I have run clang-format (version 15) on any C++ source files (if applicable)
  • [x] I have followed the style guidelines for Python source files (if applicable)
  • [x] I have made corresponding changes to the documentation (if applicable)
  • [x] I have added tests that prove my fix is effective or that my feature works (if applicable)

gonuke avatar Oct 05 '24 16:10 gonuke

This definitely results in the unit tests being run first, which is certainly helpful in getting earlier failures.

For reasons I don't fully understand, changing the order of the regression tests causes universe numbers to be off by about 6390. I am guessing that the test harness is somehow generating unique universe IDs across many test problems, and the unit tests consume roughly 6390 of these IDs before the regression tests begin.

There is a mode to update the reference data, but I'm not sure I understand all of this enough to be sure it's the right step at this time.

Perhaps @paulromano could offer some insight and/or chime in on whether this is worth pursuing.

gonuke avatar Oct 05 '24 17:10 gonuke

Note: While this PR shows 117 files that have changed, most of them are just directory name changes. Four (4) files have been updated because they have python imports that refer to that directory name.

gonuke avatar Oct 05 '24 19:10 gonuke

Fixed by #3533.

GuySten avatar Oct 22 '25 15:10 GuySten