ice
ice copied to clipboard
Skipping slow tests
When I run ./scripts/run-tests.sh
, all tests show PASSED except two:
tests/test_metrics.py::test_nubia FAILED
tests/test_metrics.py::test_gold_paragraphs SKIPPED (Parses all PDFs - very slow)
-
test_nubia
fails with an obscure pydantic validation error, I had to add a print to understand the problem. How about adding@mark.skipif(not settings.OUGHT_INFERENCE_API_KEY, reason='...')
? - Both tests have
@mark.slow
but this doesn't do anything by default. I assume this is whytest_gold_paragraphs
has a hard@mark.skip
. This doesn't seem great - what do you do when you actually want to run the test? Should a way to skip slow tests by default (with or without using@mark.slow
) be investigated? -
-m 'not slow'
correctly deselects the slow tests, but only if I directly run docker compose and pytest../scripts/run-tests.sh -m 'not slow'
doesn't work, and the shell (tested both bash and zsh) shows it running with-m 'not\' slow
. Looks like this isn't working:
https://github.com/oughtinc/ice/blob/7eab90d22b3bd8ca68f944213e6f9425ff188b3a/scripts/run-tests.sh#L8-L9
This definitely seems worth fixing as part of an overhaul of all the scripts: https://github.com/oughtinc/ice/issues/7#issuecomment-1256665522