tarpaulin
tarpaulin copied to clipboard
Test filtering is slow
Describe the bug I use tarpaulin in a bit of an abusive way (as noted here) - I want a breakdown of coverage per test and so i get cargo to give me a list of tests and call tarpaulin for each of those tests. This works fine but i recently noticed that the process of filtering out the excluded tests is very slow.
Example I can't provide the code to generate this (though if required i can spend some time making a non-private example) but i can provide an output showing the timestamps posted below.
We do have a weird test setup where we set a bunch of stuff to ignored and then trigger them 'manually' by specifying the test file and using the ignored flag.
The command that was used to generate the data below is
cargo tarpaulin --all --skip-clean --ignore-tests -o json html --ignored --features <feature_flag> -- test_camera
Two things I don't understand: why some appear repeated and why the filtering is slow.
[2mApr 27 19:06:38.795[0m [32m INFO[0m cargo_tarpaulin::config: Creating config
[2mApr 27 19:06:38.882[0m [32m INFO[0m cargo_tarpaulin: Running Tarpaulin
[2mApr 27 19:06:38.882[0m [32m INFO[0m cargo_tarpaulin: Building project
<COMPILING>
Finished test [unoptimized + debuginfo] target(s) in 2m 13s
[2mApr 27 19:08:53.014[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:08:53.014[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/netman-79ed60d51e9a80f4
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 24 filtered out; finished in 0.00s
[2mApr 27 19:08:53.724[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:08:53.724[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/netman-79ed60d51e9a80f4
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 24 filtered out; finished in 0.00s
[2mApr 27 19:08:54.293[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:08:54.294[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/camera-a5491540fae07128
running 1 test
test test_camera ... ignored
test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 1 filtered out; finished in 0.00s
[2mApr 27 19:09:01.124[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:01.125[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/camera-a5491540fae07128
running 1 test
test test_camera ... <TEST RUN>
ok
a
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 14.60s
[2mApr 27 19:09:22.135[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:22.135[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/uploader-06582b41884ef3cc
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 8 filtered out; finished in 0.00s
[2mApr 27 19:09:28.360[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:28.360[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/uploader-06582b41884ef3cc
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 8 filtered out; finished in 0.00s
[2mApr 27 19:09:34.249[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:34.249[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/threads-4b930ccc0434f7ad
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s
[2mApr 27 19:09:40.564[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:40.564[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/threads-4b930ccc0434f7ad
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s
[2mApr 27 19:09:46.655[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:46.655[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/PROJECT-dfbed683b4108eae
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[2mApr 27 19:09:52.744[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:52.745[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/PROJECT-dfbed683b4108eae
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[2mApr 27 19:09:58.704[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:09:58.704[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/serial-101dc2dd4995a0ef
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.00s
[2mApr 27 19:10:04.880[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:04.881[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/serial-101dc2dd4995a0ef
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.00s
[2mApr 27 19:10:10.859[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:10.859[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/migration-fed54f68cc83e80a
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 3 filtered out; finished in 0.00s
[2mApr 27 19:10:16.988[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:16.989[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/migration-fed54f68cc83e80a
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 3 filtered out; finished in 0.00s
[2mApr 27 19:10:22.976[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:22.976[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/PROJECT-44afaec8f6bab996
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 21 filtered out; finished in 0.00s
[2mApr 27 19:10:29.032[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:29.033[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/PROJECT-44afaec8f6bab996
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 21 filtered out; finished in 0.00s
[2mApr 27 19:10:35.044[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:35.044[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/system-f0d2d1d4b0f682aa
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s
[2mApr 27 19:10:41.060[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:41.061[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/system-f0d2d1d4b0f682aa
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s
[2mApr 27 19:10:47.048[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:47.048[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/logging-266777b2b1b3f891
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 4 filtered out; finished in 0.00s
[2mApr 27 19:10:53.064[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:53.065[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/logging-266777b2b1b3f891
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 4 filtered out; finished in 0.00s
[2mApr 27 19:10:59.052[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:10:59.052[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/wifi-bf08ebb1cfdc6145
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.00s
[2mApr 27 19:11:05.068[0m [32m INFO[0m cargo_tarpaulin::process_handling::linux: Launching test
[2mApr 27 19:11:05.069[0m [32m INFO[0m cargo_tarpaulin::process_handling: running /CODE_PATH/target/debug/deps/wifi-bf08ebb1cfdc6145
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.00s
[2mApr 27 19:11:11.118[0m [32m INFO[0m cargo_tarpaulin::report: Coverage Results:
|| Tested/Total Lines:
|| <RESULTS>
||
25.82% coverage, 1351/5233 lines covered
Expected behavior I would have expected, perhaps naively, that the filtering would be quick. In the above example the filtering is comparable in duration to the test itself.
Possible workaround As noted i know which file each test lives in from getting cargo to list all the tests, is there some way i can feed this information to tarpaulin to tell it to only look in one of the test binaries for each test?
Thanks! Z