macrotest
macrotest copied to clipboard
eupn
Fails under cargo nextest
Cargo Nextest has a different test execution model where each test is compiled into a separate binary. This causes issues with macrotest- it seems that it assumes that each crate will contain at most one call to macrotest from a single binary.
This is reproducible from the tests within this repo:
cargo install cargo-nextest
git clone https://github.com/eupn/macrotest.git
cd macrotest/test-project
cargo nextest run
Output:
cargo nextest run
Finished test [unoptimized + debuginfo] target(s) in 0.04s
Starting 9 tests across 3 binaries (run ID: 0c2815c1-f154-41a9-bf6f-819b59413968, nextest profile: default)
FAIL [ 0.194s] test-project::par_tests_regression parallel_2
--- STDOUT: test-project::par_tests_regression parallel_2 ---
running 1 test
Running 1 macro expansion tests
test parallel_2 ... FAILED
failures:
failures:
parallel_2
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.07s
--- STDERR: test-project::par_tests_regression parallel_2 ---
thread 'parallel_2' panicked at D:\git-repos\github\macrotest\src\expand.rs:128:9:
prepare failed: IoError(
Os {
code: 32,
kind: Uncategorized,
message: "The process cannot access the file because it is being used by another process.",
},
)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Canceling due to test failure: 8 tests still running
PASS [ 0.174s] test-project::tests fail_expect_expanded
PASS [ 0.153s] test-project::tests fail_expect_expanded_args
FAIL [ 0.134s] test-project::tests pass
--- STDOUT: test-project::tests pass ---
running 1 test
Running 4 macro expansion tests
test pass ... FAILED
failures:
failures:
pass
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 6 filtered out; finished in 0.08s
--- STDERR: test-project::tests pass ---
thread 'pass' panicked at D:\git-repos\github\macrotest\src\expand.rs:128:9:
prepare failed: IoError(
Os {
code: 32,
kind: Uncategorized,
message: "The process cannot access the file because it is being used by another process.",
},
)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAIL [ 0.140s] test-project::tests pass_args
--- STDOUT: test-project::tests pass_args ---
running 1 test
Running 1 macro expansion tests
test pass_args ... FAILED
failures:
failures:
pass_args
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 6 filtered out; finished in 0.08s
--- STDERR: test-project::tests pass_args ---
thread 'pass_args' panicked at D:\git-repos\github\macrotest\src\expand.rs:128:9:
prepare failed: IoError(
Os {
code: 32,
kind: Uncategorized,
message: "The process cannot access the file because it is being used by another process.",
},
)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAIL [ 0.133s] test-project::tests pass_expect_expanded
--- STDOUT: test-project::tests pass_expect_expanded ---
running 1 test
Running 4 macro expansion tests
test pass_expect_expanded ... FAILED
failures:
failures:
pass_expect_expanded
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 6 filtered out; finished in 0.08s
--- STDERR: test-project::tests pass_expect_expanded ---
thread 'pass_expect_expanded' panicked at D:\git-repos\github\macrotest\src\expand.rs:128:9:
prepare failed: IoError(
Os {
code: 32,
kind: Uncategorized,
message: "The process cannot access the file because it is being used by another process.",
},
)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAIL [ 0.144s] test-project::tests pass_expect_expanded_args
--- STDOUT: test-project::tests pass_expect_expanded_args ---
running 1 test
Running 1 macro expansion tests
test pass_expect_expanded_args ... FAILED
failures:
failures:
pass_expect_expanded_args
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 6 filtered out; finished in 0.09s
--- STDERR: test-project::tests pass_expect_expanded_args ---
thread 'pass_expect_expanded_args' panicked at D:\git-repos\github\macrotest\src\expand.rs:128:9:
prepare failed: IoError(
Os {
code: 32,
kind: Uncategorized,
message: "The process cannot access the file because it is being used by another process.",
},
)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAIL [ 0.131s] test-project::tests pr61
--- STDOUT: test-project::tests pr61 ---
running 1 test
Running 2 macro expansion tests
test pr61 ... FAILED
failures:
failures:
pr61
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 6 filtered out; finished in 0.08s
--- STDERR: test-project::tests pr61 ---
thread 'pr61' panicked at D:\git-repos\github\macrotest\src\expand.rs:128:9:
prepare failed: IoError(
Os {
code: 32,
kind: Uncategorized,
message: "The process cannot access the file because it is being used by another process.",
},
)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
PASS [ 1.853s] test-project::par_tests_regression parallel_1
------------
Summary [ 1.856s] 9 tests run: 3 passed, 6 failed, 0 skipped
FAIL [ 0.194s] test-project::par_tests_regression parallel_2
FAIL [ 0.134s] test-project::tests pass
FAIL [ 0.140s] test-project::tests pass_args
FAIL [ 0.133s] test-project::tests pass_expect_expanded
FAIL [ 0.144s] test-project::tests pass_expect_expanded_args
FAIL [ 0.131s] test-project::tests pr61
error: test run failed
It looks like its panicking here due to error deleting the target/tests/<CRATE NAME>/macrotest000X directory .
the unique names from COUNT seem to handle multiple calls to expand/expand_args within a single binary, but this won't work for multiple test binaries. I skimmed thru trybuild code and it seems like they avoid this by having a flock on the directory
This is kind of a side effect of https://github.com/eupn/macrotest/pull/69.
That said, since the target directory is shared, it should not work to actually run macrotest in parallel anyway due to the cargo's mechanism.
File lock seems to be a reasonable approach here.
