[BUG] Uninitialized pointer access when using --benchmark_repetitions in some scenarios

[Sparta142]

Describe the bug An uninitialized pointer is dereferenced when calculating statistics for a repeated benchmark that fails a certain number of times, which causes a segv/access violation/other mayhem.

This doesn't seem to happen for benchmarks that always fail, nor if you run without --benchmark_repetitions=N.

System Which OS, compiler, and compiler version are you using:

• OS: Windows 10 Pro 21H2 (x64)
• Compiler and version: MSVC 19.29.30141.0

In my experience this happens on various compiler (MSVC, GCC) and execution environments (Windows x64, Linux arm64), as well as in both Debug and Release builds.

To reproduce

#include <benchmark/benchmark.h>

static int count = 0;

static void BM_StringCreation(benchmark::State &state)
{
    if (++count == 3)
    {
        state.SkipWithError("Simulate a flaky benchmark");
        return;
    }

    for (auto _ : state)
        std::string empty_string;
}
BENCHMARK(BM_StringCreation);

int main(int argc, char *argv[])
{
    benchmark::Initialize(&argc, argv);
    benchmark::RunSpecifiedBenchmarks();
    benchmark::Shutdown();
    return 0;
}

Steps to reproduce the behavior: Compile the given code above on MSVC 19 with benchmark 1.6.1, and run using:

./example.exe --benchmark_repetitions=5

Expected behavior Benchmark does not crash. Statistics are calculated for only the passing runs. (If this is incorrect usage of the library, it should be documented somewhere.)

Screenshots

Additional context Looks like the uninitialized pointer is reports[0].statistics: https://github.com/google/benchmark/blob/6d50251d8e6ed6b7f6eb1e12c558d2808fb4ddaa/src/statistics.cc#L159

It seems to not be initialized here (on line 98) when a run has an error: https://github.com/google/benchmark/blob/6d50251d8e6ed6b7f6eb1e12c558d2808fb4ddaa/src/benchmark_runner.cc#L88-L99

[dmah42]

i agree, this looks to be a bug. though i would have expected us to skip error reports in the statistics calculations.