bytecodealliance
Weird file CI issues with benchmark_effect_size test
On Windows, the benchmark_effect_size test has started intermittently failing since #170. It is unclear to me what could have modified the execution of the code that copies the built engine to a location provided by tempfile::NamedTempFile. I added an assert in hopes that it would trigger the error sooner but the OS always thinks the file exists:
https://github.com/bytecodealliance/sightglass/blob/cd8c11b1da4c7ebe3b5b1c7fc10e3a613a9c3b7f/crates/cli/tests/tests.rs#L213
One example of this failure is here:
thread 'benchmark::benchmark_effect_size' panicked at 'Unexpected failure.
code--1073741819
...
command=`"D:\\a\\sightglass\\sightglass\\target\\debug\\sightglass-cli.exe" "benchmark" "--engine" "\\\\?\\D:\\a\\sightglass\\sightglass\\engines\\wasmtime\\engine.dll" "--engine" "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\.tmpaNVVum" "--processes" "1" "--iterations-per-process" "3" "../../benchmarks-next/noop/benchmark.wasm"`
If this random link is to be believed, the code referenced above is an access violation that might indicate referencing a null pointer, e.g. But why? The original engine library seems to run fine according to the logs; only the copied engine library has the problem.
@fitzgen, should we now consider this closed? If the CI issues with benchmark_effect_size are no longer a problem, maybe my "access violation" hypothesis was incorrect and it was the variance issue you saw?
