anp
META: benchmarks to include
This is a tracking issue for new benchmarks we'd like to add. If you're landing here from hacktoberfest (welcome!), feel free to see the project README for more information :D.
The basic process goes:
- find where the project in question keeps its benchmarks (usually in its repo under
benches) - read the documentation on adding new benchmarks
- add the benchmarks
- send a PR
- at a point in the near future, we'll start seeing results from the benchmarks you ported on https://blog.anp/lol/lolbench-data
unclaimed
please comment on individual issues to claim them!
- aho-corasick
- futures
- backtrace
- smallvec/vecmap
- enjarify
- an arena allocator
- backtrace
- claxon
- benchmark-game
- port ISPC examples
- port SPEC
- latest inflate
finished
- ~csv~ thanks @shssoichiro!
- ~byteorder~ thanks @thomwiggers!
- ~RustCrypto~ thanks @thomwiggers!
- ~clap~ thanks @kitlith!
We should probably also include the benchmark-game programs from https://github.com/TeXitoi/benchmarksgame-rs .
Also I always wanted to port the remaining examples of the rust-lang-nursery/simd crate to rust-lang-nursery/stdsimd, maybe we should include those once they are done. They will probably both have a scalar and vectorized implementation, but catching regressions on LLVM vector type optimizations is worth pursuing.
The ISPC compiler has a bunch of examples that we might want to port to Rust: https://github.com/ispc/ispc/tree/master/examples
From the standard suites, porting SPEC2017 (https://www.spec.org/benchmarks.html) to Rust.
IMO when porting thins, we should port benchmarks to "idiomatic" Rust code.
I agree! :p
Right now the benchmark count bottleneck is having enough machines to run them in a timely fashion, I need to dig in more on the bench timings and get the new server(s) I'm setting up ready.
I think ISPC and SPEC benchmarks would be fantastic to track as well!
per @eddyb on irc, we should also include an equivalent to the inflate 0.3.4 benchmark with a more recent version of the crate.
The very important bit is whether or not the debloating https://github.com/PistonDevelopers/inflate/pull/35 is included.
@eternaleye gave me some reading homework and I'm saving it here for future reference:
https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.html
https://algowiki-project.org/en/Open_Encyclopedia_of_Parallel_Algorithmic_Features
http://superfri.org/superfri/article/view/69
https://github.com/vtsynergy/OpenDwarfs
http://dsc.soic.indiana.edu/publications/OgreFacets.pdf
https://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-144.html
https://www.sciencedirect.com/science/article/pii/S1877050912003353
I've uploaded the ambient occlusion (ray-tracing) benchmarks: https://github.com/gnzlbg/aobench
The readme shows how to run the scalar, vectorized, and parallel versions. Probably it only makes sense to add the scalar and scalar_par benchmarks initially. The vector and vector_par benchmarks use std::simd and that's still in flux.
https://benchmarksgame.alioth.debian.org/u64q/rust.html ?
Alioth is dead, but see https://benchmarksgame-team.pages.debian.net/benchmarksgame/measurements/rust.html
Going to lock this so it can just be a tracking issue. If you have ideas for other benchmarks we should include, please open an issue!