asterius icon indicating copy to clipboard operation
asterius copied to clipboard

Add nofib performance testing

Open TerrorJack opened this issue 5 years ago • 1 comments

Is your feature request related to a problem? Please describe. We don't have quantitative comparison of generated code performance against native or ghcjs. A good starting point would be using nofib, since most nofib programs are self-contained and can be compiled with ghc boot libs.

Gathering performance characteristics of nofib opens new chances for generating smaller and faster code (and improving the runtime). We need to answer the following questions:

  • How do we compare to native/ghcjs in terms of code size and performance?
  • What are the bottlenecks in the wasm code or runtime?
  • For a PR that impacts generated code or the runtime, does it make things better or worse? (And by what percentage?)

Describe the solution you'd like

  • [ ] Collect nofib main programs and their arguments from the upstream nofib repo. Upstream uses Makefiles to specify different inputs of programs for fast/normal/slow paths; we should parse those data from Makefiles.
  • [ ] Finish native/asterius comparison first; set this up as a standalone CI job, similar to ghc-testsuite.
  • [ ] Generate V8 CPU/heap profiler report and serve them as CI artifacts, so we can examine the bottlenecks visually.

TerrorJack avatar May 03 '20 20:05 TerrorJack

Blocked by #634.

TerrorJack avatar May 03 '20 20:05 TerrorJack