Benching system

[NikVolf]

trafficstars

to address lack of benchmarks, we might want to use special wasm binaries which utilize specific benching api

it should consist of:

1. start benchmark (extern "C" fn start_bench(name_ptr: *const u8, name_len: u32))
2. start iteration (extern "C" fn start_iter())
3. end iteration (extern "C" fn end_iter())
4. blackbox/observe (extern "C" fn bb_observe(ptr: *mut u8, len: u32)) (can be replaced by test::black_box on nightly)

each start_iter should follow end_iter sequentally so there should be no nested iterations or overlaps (higher-level utility library should enforce it via closures/ownership)

bb_observe is used to avoid compiler optimisations

[eira-fransham]

AAIU bb_observe is unnecessary, you can use any function that's unreadable by LLVM (including the test::black_box function).

[NikVolf]

test::black_box

is it available in no-std?

[eira-fransham]

Yes, it is

[eira-fransham]

I'm trying to work on a relatively-transparent benchmarking harness that compiles normal Rust benchmarks to a format that could work when compiled with wasm32-unknown-unknown.

[eira-fransham]

Implemented here

[NikVolf]

Great! Should we utilize it here, on the CI also?

[eira-fransham]

I think so, although I'd have to make it work properly on test failure - i.e., report the actual reason for test failure. Currently I haven't implemented any of the exception mechanisms so the interpreter just panicks complaining about unimplemented functions.

[Robbepop]

We moved to Criterion based benchmarks that allow for more flexibility. I do not agree with a fixed benchmarking API as proposed in the initial comment since benchmarks are way too different to unify them under a single Wasm based API.