MOE
MOE copied to clipboard
[Python] set up a common file for test markers/decorators (e.g., to mark flaky, excluded, slow, etc)
pytest provides a nice framework for skipping tests. It can handle conditionals, expected failures, and much more: http://pytest.org/latest/skipping.html
e.g., skipping tests that are marked slow is as easy as: http://pytest.org/latest/example/simple.html#control-skipping-of-tests-according-to-command-line-option
The system relies on devs decorating tests consistently with pytest.mark
subclasses. They suggest:
For larger test suites it’s usually a good idea to have one file where you define the markers which you then consistently apply throughout your test suite.
We should set up such a file for ourselves. At the minimum it should have a standard way of marking tests as "excluded" so we can quickly disable them. (There's even frameworks for rerunning flaky tests automatically: https://github.com/box/flaky)