gz-usd icon indicating copy to clipboard operation
gz-usd copied to clipboard

USD component: handle early exits in unit tests

Open adlarkin opened this issue 3 years ago • 0 comments

Description

https://github.com/ignitionrobotics/sdformat/pull/796#pullrequestreview-842491424 points out that USD has a known bug (https://github.com/PixarAnimationStudios/USD/issues/86) where fatal errors do not abort cleanly. This could be an issue in unit tests if an error occurs mid-test from something like trying to access an invalid stage - USD will "gracefully exit" without signifying that the stage was invalid, which could give a false positive of tests succeeding.

Steps to reproduce

The following program can be tested (you need to install USD). You'll notice that the program exits cleanly (exit code 0 instead of 1) without throwing an error:

#include <pxr/usd/usd/stage.h>

int main(int argc, char **argv) {
  pxr::UsdStageRefPtr stage;
  stage->Save(); // try to dereference invalid pointer
  throw std::runtime_error("bad stage reference occurred");
  return 1;
}

It should be worth knowing that in the unit tests that are being written for the USD component, we are not intentionally doing things like trying to dereference an invalid stage pointer. However, in order to protect ourselves from bad exits we are unaware of and false positive test results, we should address this issue to make sure that all unit tests run to completion. https://github.com/ignitionrobotics/sdformat/pull/796#pullrequestreview-842491424 lists a few ideas for solving this, and in particular, I think the third idea given could work: "make a custom test runner which either captures and check stdout of the tests, or have each test write some artifacts and have the runner check them at the end."

adlarkin avatar Jan 20 '22 18:01 adlarkin