llvm-opt-benchmark icon indicating copy to clipboard operation
llvm-opt-benchmark copied to clipboard

Wishlist for new benchmarks

Open dtcxzyw opened this issue 1 year ago • 15 comments

C/C++

  • [ ] https://github.com/gem5/gem5
  • [ ] https://github.com/daos-stack/daos
  • [ ] https://github.com/ray-project/ray
  • [ ] https://github.com/google/pigweed
  • [ ] https://github.com/pmodels/mpich
  • [ ] https://github.com/RT-Thread/rt-thread
  • [ ] https://github.com/FreeRTOS/FreeRTOS
  • [ ] https://github.com/cloudflare/workerd
  • [ ] https://github.com/bitcoin/bitcoin
  • [ ] https://github.com/MarlinFirmware/Marlin
  • [ ] https://github.com/spotify/pedalboard
  • [ ] https://github.com/vim/vim
  • [ ] https://github.com/wesnoth/wesnoth
  • [ ] https://github.com/chromium/chromium
  • [ ] https://github.com/sonic-pi-net/sonic-pi
  • [ ] https://github.com/google/skia
  • [ ] https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/Benchmarks/MiBench
  • [ ] https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/Benchmarks/mafft
  • [ ] https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/Benchmarks/7zip
  • [ ] https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/Applications/SPASS
  • [ ] https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/Benchmarks/tramp3d-v4
  • [ ] https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/Applications/JM/lencod
  • [ ] https://github.com/scylladb/seastar
  • [ ] https://github.com/nginx/nginx
  • [ ] https://github.com/envoyproxy/envoy
  • [ ] https://github.com/clerkma/ptex-ng
  • [ ] https://gitlab.freedesktop.org/poppler/poppler
  • [ ] https://github.com/blender/blender
  • [ ] https://github.com/falcosecurity/falco
  • [ ] https://github.com/microsoft/SEAL
  • [ ] https://github.com/mamba-org/mamba
  • [ ] https://github.com/emacs-mirror/emacs
  • [ ] https://github.com/zsh-users/zsh
  • [ ] https://github.com/Perl/perl5
  • [ ] https://github.com/randombit/botan
  • [ ] https://github.com/PCRE2Project/pcre2
  • [ ] https://github.com/neovim/neovim
  • [ ] https://repo.or.cz/w/isl.git
  • [ ] https://github.com/aws/s2n-tls
  • [ ] https://github.com/Mozilla-Ocho/llamafile
  • [ ] https://github.com/ThinkParQ/beegfs
  • [ ] https://github.com/mjansson/rpmalloc
  • [ ] https://github.com/arminbiere/kissat
  • [ ] https://github.com/Genivia/ugrep
  • [ ] https://github.com/osqp/osqp
  • [ ] https://github.com/trilinos/Trilinos
  • [ ] https://github.com/Xyce/Xyce
  • [ ] https://github.com/bazelbuild/bazel
  • [ ] https://github.com/RobotLocomotion/drake
  • [ ] https://github.com/NASA-SW-VnV/ikos
  • [ ] https://github.com/manticoresoftware/manticoresearch/
  • [ ] https://github.com/neutrinolabs/xrdp
  • [ ] https://github.com/ArduPilot/ardupilot
  • [ ] https://proxyapps.exascaleproject.org
  • [ ] https://github.com/rime/librime
  • [ ] https://github.com/tcltk/tcl
  • [ ] https://github.com/gwsystems/composite
  • [ ] https://github.com/ArthurSonzogni/Diagon
  • [ ] https://github.com/ArthurSonzogni/FTXUI
  • [ ] https://github.com/ArtifexSoftware/mupdf
  • [ ] https://github.com/adobe-type-tools/afdko
  • [ ] https://gitlab.com/AOMediaCodec/SVT-AV1
  • [ ] https://github.com/nmwsharp/polyscope
  • [ ] https://github.com/pmp-library/pmp-library
  • [ ] https://github.com/nwchemgit/nwchem
  • [ ] https://github.com/cyberbotics/webots
  • [ ] https://github.com/introlab/rtabmap
  • [ ] https://github.com/hyprwm/Hyprland
  • [ ] https://github.com/wine-mirror/wine
  • [ ] https://github.com/unikraft/unikraft
  • [ ] https://github.com/accellera-official/systemc
  • [ ] https://github.com/KhronosGroup/SPIRV-Tools
  • [ ] https://gitlab.freedesktop.org/pixman/pixman
  • [ ] https://github.com/tesseract-ocr/tesseract
  • [ ] https://github.com/PKU-IDEA/OpenPARF
  • [ ] https://github.com/obsproject/obs-studio
  • [ ] https://github.com/OpenFOAM/OpenFOAM-dev
  • [ ] https://github.com/LekKit/RVVM
  • [ ] https://github.com/ValveSoftware/GameNetworkingSockets
  • [ ] https://github.com/ValveSoftware/Proton
  • [ ] https://gitlab.freedesktop.org/mesa/mesa
  • [ ] https://github.com/facebook/CacheLib
  • [ ] https://github.com/The-OpenROAD-Project/OpenSTA
  • [ ] https://github.com/The-OpenROAD-Project/OpenROAD
  • [ ] https://github.com/pierotofy/OpenSplat
  • [ ] https://github.com/LadybirdBrowser/ladybird
  • [ ] https://github.com/gazebosim/gz-sim
  • [ ] https://github.com/vstakhov/libucl
  • [ ] https://github.com/rspamd/rspamd
  • [ ] https://github.com/sbeamer/gapbs
  • [ ] https://github.com/mjwybrow/adaptagrams
  • [ ] https://github.com/scipy/scipy
  • [ ] https://github.com/ceph/ceph
  • [ ] https://github.com/evaleev/libint
  • [ ] https://github.com/vesoft-inc/nebula
  • [ ] https://github.com/zephyrproject-rtos/zephyr
  • [ ] https://github.com/dragonflydb/dragonfly
  • [ ] https://github.com/eclipse-sumo/sumo
  • [ ] https://github.com/OSGeo/gdal
  • [ ] https://github.com/OSGeo/PROJ
  • [ ] https://github.com/OSGeo/grass
  • [ ] https://github.com/cesanta/mongoose
  • [ ] https://github.com/antoinemine/apron
  • [ ] https://github.com/espressomd/espresso
  • [ ] https://github.com/souffle-lang/souffle
  • [ ] https://github.com/chipsalliance/espresso
  • [ ] https://github.com/KhronosGroup/KTX-Software
  • [ ] https://github.com/commonmark/cmark
  • [ ] https://github.com/github/glb-director
  • [ ] https://github.com/dreamworksanimation/openmoonray
  • [ ] https://github.com/openscad/openscad/
  • [ ] https://github.com/ossrs/srs
  • [ ] https://github.com/dun/munge
  • [ ] https://github.com/LadybirdBrowser/ladybird
  • [ ] https://github.com/vsg-dev/VulkanSceneGraph
  • [ ] https://github.com/apache/thrift
  • [ ] https://github.com/wolfpld/etcpak
  • [ ] https://github.com/sofa-framework/sofa
  • [ ] https://github.com/google-deepmind/mujoco
  • [ ] https://gitlab.com/libxc/libxc
  • [ ] https://github.com/Open-Cascade-SAS/OCCT
  • [ ] https://github.com/capnproto/capnproto
  • [ ] https://github.com/xiph/rnnoise
  • [ ] https://github.com/microsoft/onnxruntime
  • [ ] https://github.com/Yubico/libfido2
  • [ ] https://github.com/PipeWire/pipewire
  • [ ] https://github.com/libjxl/libjxl
  • [ ] https://github.com/openocd-org/openocd
  • [ ] https://github.com/shadowsocks/shadowsocks-libev
  • [ ] https://github.com/PointCloudLibrary/pcl
  • [ ] https://github.com/homenc/HElib
  • [ ] https://github.com/mozilla/mozjpeg
  • [ ] https://github.com/mozilla/DeepSpeech
  • [ ] https://github.com/ERGO-Code/HiGHS
  • [ ] https://gitlab.kitware.com/paraview
  • [ ] https://github.com/strukturag/libheif
  • [ ] https://github.com/mfem/mfem
  • [ ] https://github.com/bitwuzla/bitwuzla
  • [ ] https://github.com/LibreOffice/core
  • [ ] https://github.com/lammps/lammps
  • [ ] https://github.com/odygrd/quill
  • [ ] https://github.com/eBay/NuRaft
  • [ ] https://github.com/BenLangmead/bowtie2
  • [ ] https://github.com/mozilla/gecko-dev
  • [ ] https://gtsam.org
  • [ ] https://github.com/SVF-tools/SVF
  • [ ] sage
  • [ ] https://github.com/SoftFever/OrcaSlicer
  • [ ] https://gitlab.cern.ch/explore/projects/
  • [ ] https://github.com/stillwater-sc/universal
  • [ ] https://github.com/nix-community/nixd
  • [ ] https://github.com/capstone-engine/capstone
  • [ ] https://github.com/open-quantum-safe/liboqs
  • [ ] https://github.com/jrouwe/JoltPhysics
  • [ ] https://github.com/intel/isa-l
  • [ ] https://github.com/protocolbuffers/protobuf/tree/main/upb
  • [ ] https://github.com/Tencent/MMKV
  • [ ] https://github.com/google/perfetto

Rust

  • [ ] https://github.com/Keats/tera
  • [ ] https://github.com/risingwavelabs/risingwave
  • [ ] https://github.com/Byron/gitoxide
  • [ ] https://github.com/huggingface/candle
  • [ ] https://github.com/gfx-rs/naga
  • [ ] https://github.com/lancedb/lance
  • [ ] https://github.com/dimforge/nalgebra
  • [ ] https://github.com/rust-ndarray/ndarray
  • [ ] https://github.com/parcel-bundler/lightningcss
  • [ ] https://github.com/facebook/buck2
  • [ ] https://github.com/evmar/n2
  • [ ] https://github.com/github/stack-graphs
  • [ ] https://github.com/tweag/nickel
  • [ ] https://github.com/DioxusLabs/blitz
  • [ ] https://github.com/signalapp/libsignal
  • [ ] https://github.com/cryspen/libcrux
  • [ ] https://github.com/apache/iceberg-rust
  • [ ] https://github.com/shadowsocks/shadowsocks-rust
  • [ ] rust libstd
  • [ ] https://github.com/microsoft/SymCrypt
  • [ ] https://github.com/icedland/iced
  • [ ] https://github.com/chroma-core/chroma
  • [ ] https://github.com/googlefonts/fontc
  • [ ] https://github.com/huggingface/xet-core
  • [ ] https://github.com/facebook/buck2

dtcxzyw avatar Jan 17 '24 12:01 dtcxzyw

~~Request: https://github.com/python/cpython~~ nevermind, realize this is not a list of what you have.

goldsteinn avatar Mar 08 '24 16:03 goldsteinn

~Request: https://github.com/python/cpython~ nevermind, realize this is not a list of what you have.

I will add a list to README :)

dtcxzyw avatar Mar 09 '24 09:03 dtcxzyw

Since I see vim up there: https://savannah.gnu.org/git/?group=emacs

goldsteinn avatar Apr 01 '24 20:04 goldsteinn

Since I see vim up there: https://savannah.gnu.org/git/?group=emacs

Also favor Neovim here: https://github.com/neovim/neovim :)

XChy avatar Apr 06 '24 16:04 XChy

https://github.com/sudo-project/sudo https://github.com/OpenPrinting/cups

Non-compute-intensive applications will not be considered.

dtcxzyw avatar May 08 '24 01:05 dtcxzyw

libev the less bloated and more performant version of libevent.

goldsteinn avatar Oct 08 '24 04:10 goldsteinn

Firefox: https://github.com/mozilla/gecko-dev

mayankleoboy avatar Dec 12 '24 02:12 mayankleoboy

Rust's libcore/liballoc/libstd may be good candidates, to cover the non-inlined parts.

nikic avatar Jan 19 '25 09:01 nikic

I just wanted to propose the Wasmi WebAssembly interpreter for this wishlist but just noticed that it is already on the list. 🙃

Wasmi is heavily influenced by optimizations and optimization heuristics in LLVM (at least as evidenced over the years with major LLVM updates), especially with regards to its internal hot interpreter loop. LLVM once compiled Wasmi's interpreter loop to efficient decentralized computed-goto but since some time it has severely regressed and uses a centralized jump table.

For the LLVM optimizer Wasmi's interpreter loop clearly demonstrates a challenge as Rust does not support explicit tail calls nor computed-gotos which usually are a backbone for efficient interpreters. This means that Rust interpreters have to rely on LLVM to optimize them in the right way. I hope that it might be in the interest of both projects to have Wasmi included. :)

How do I increase the chances?

Robbepop avatar Apr 01 '25 16:04 Robbepop

I just wanted to propose the Wasmi WebAssembly interpreter for this wishlist but just noticed that it is already on the list. 🙃

Wasmi is heavily influenced by optimizations and optimization heuristics in LLVM (at least as evidenced over the years with major LLVM updates), especially with regards to its internal hot interpreter loop. LLVM once compiled Wasmi's interpreter loop to efficient decentralized computed-goto but since some time it has severely regressed and uses a centralized jump table.

For the LLVM optimizer Wasmi's interpreter loop clearly demonstrates a challenge as Rust does not support explicit tail calls nor computed-gotos which usually are a backbone for efficient interpreters. This means that Rust interpreters have to rely on LLVM to optimize them in the right way. I hope that it might be in the interest of both projects to have Wasmi included. :)

How do I increase the chances?

Sorry for the late reply. If possible, I will add wasmi this week :)

dtcxzyw avatar May 19 '25 16:05 dtcxzyw

I just wanted to propose the Wasmi WebAssembly interpreter for this wishlist but just noticed that it is already on the list. 🙃

Wasmi is heavily influenced by optimizations and optimization heuristics in LLVM (at least as evidenced over the years with major LLVM updates), especially with regards to its internal hot interpreter loop. LLVM once compiled Wasmi's interpreter loop to efficient decentralized computed-goto but since some time it has severely regressed and uses a centralized jump table.

For the LLVM optimizer Wasmi's interpreter loop clearly demonstrates a challenge as Rust does not support explicit tail calls nor computed-gotos which usually are a backbone for efficient interpreters. This means that Rust interpreters have to rely on LLVM to optimize them in the right way. I hope that it might be in the interest of both projects to have Wasmi included. :)

How do I increase the chances?

Sorry for the late reply. If possible, I will add wasmi this week :)

Very happy to hear that. Looking forward to this. :)

Robbepop avatar May 19 '25 17:05 Robbepop

I just wanted to propose the Wasmi WebAssembly interpreter for this wishlist but just noticed that it is already on the list. 🙃 Wasmi is heavily influenced by optimizations and optimization heuristics in LLVM (at least as evidenced over the years with major LLVM updates), especially with regards to its internal hot interpreter loop. LLVM once compiled Wasmi's interpreter loop to efficient decentralized computed-goto but since some time it has severely regressed and uses a centralized jump table. For the LLVM optimizer Wasmi's interpreter loop clearly demonstrates a challenge as Rust does not support explicit tail calls nor computed-gotos which usually are a backbone for efficient interpreters. This means that Rust interpreters have to rely on LLVM to optimize them in the right way. I hope that it might be in the interest of both projects to have Wasmi included. :) How do I increase the chances?

Sorry for the late reply. If possible, I will add wasmi this week :)

Very happy to hear that. Looking forward to this. :)

Done.

dtcxzyw avatar May 20 '25 07:05 dtcxzyw

@dtcxzyw thank you! 🚀

Robbepop avatar May 20 '25 08:05 Robbepop

evmone: https://github.com/ipsilon/evmone, the C++ EVM interpreter, which I'm an author of.

chfast avatar Sep 30 '25 11:09 chfast

evmone: https://github.com/ipsilon/evmone, the C++ EVM interpreter, which I'm an author of.

Done.

dtcxzyw avatar Oct 01 '25 08:10 dtcxzyw