benchmark icon indicating copy to clipboard operation
benchmark copied to clipboard

benchmark::MakeUnpredictable

Open Maratyszcza opened this issue 8 years ago • 12 comments

folly::benchmark includes makeUnpredictable function to hide a compile-time constant value of a variable from optimizer. This is a very useful feature, and google::benchmark should provide something similar.

Maratyszcza avatar Feb 11 '17 21:02 Maratyszcza

DoNotOptimize() should do that already.

EricWF avatar Feb 11 '17 21:02 EricWF

DoNotOptimize() is different. Here is my use-case:

static void quotient_uint32_t(benchmark::State& state) {
	const uint32_t divisor = benchmark::MakeUnpredictable(3);
	uint32_t x = 0;
	while (state.KeepRunning()) {
		const uint32_t quotient = x++ / divisor;
		benchmark::DoNotOptimize(quotient);
	}
}
BENCHMARK(quotient_uint32_t);

Normally compilers would replace division by an integer known at compile-time with a multiplication. I want to hide the value of the variable divisor from the optimizing compiler, so that it generates a generic hardware division instruction.

Maratyszcza avatar Feb 11 '17 21:02 Maratyszcza

DoNotOptimize() may do other things, but it will also do this.

const uint32_t divisor x = 3;
DoNotOptimize(&x);

EricWF avatar Feb 11 '17 21:02 EricWF

*Actually don't declare the variable as const. Woops.

If you want to provide MakeUnpredictable as a wrapper for DoNotOptimize with a different interface that would be OK too.

EricWF avatar Feb 11 '17 21:02 EricWF

I see in the assembly code that it gets compiled into division-via-multiplication sequence, with both Apple Clang 8.0.0 and gcc 6.3.0. Removing const doesn't make a difference.

Maratyszcza avatar Feb 11 '17 21:02 Maratyszcza

No, Clang and GCC both generate div instructions. https://godbolt.org/g/76Rz4K

EricWF avatar Feb 11 '17 21:02 EricWF

The difference is I pass divisor, not &divisor no DoNotOptimize. However, the README suggests syntax without ampersand, is it wrong?

Maratyszcza avatar Feb 11 '17 22:02 Maratyszcza

No, because you don't always want to escape the stack address of a variable. Sometimes you just want to escape/consume its value.

DoNotOptimize is tricky, but so is the optimizer.

EricWF avatar Feb 11 '17 22:02 EricWF

Ok, thanks for the solution! Would be good to document it in the README.

Maratyszcza avatar Feb 11 '17 22:02 Maratyszcza

I think the trickiness of the API is a good reason to provide a MakeUnpredictable so we can handle the pitfalls correctly for users.

EricWF avatar Feb 11 '17 22:02 EricWF

It's been a year, but I finally have a patch ready: https://github.com/efcs/benchmark/tree/make-unpredictable

Just waiting on PR #530 to land first.

EricWF avatar Feb 19 '18 23:02 EricWF

It looks like Folly dropped makeUnpredictable

HFTrader avatar Mar 08 '23 15:03 HFTrader