r7rs-benchmarks

r7rs-benchmarks copied to clipboard

• R7RS Benchmarks Benchmarks for various Scheme implementations. Taken with kind permission from the Larceny project, based on the Gabriel and Gambit benchmarks.

See the results [[http://ecraven.github.io/r7rs-benchmarks/benchmark.html][here]]

• Running the tests Just run #+BEGIN_SRC ./bench #+END_SRC Schemes that should work:

• [[https://www-sop.inria.fr/indes/fp/Bigloo][Bigloo]] (=bigloo=)
• [[http://www.call-with-current-continuation.org/bones][Bones]] (=bones=)
• [[http://scheme.com][Chez]] (=chez=)
• [[http://synthcode.com/scheme/chibi][Chibi]] (=chibi=)
• [[https://www.call-cc.org/][Chicken]] (=chicken=)
• [[http://justinethier.github.io/cyclone/][Cyclone]] (=cyclone=)
• [[https://github.com/leftmike/foment][Foment]] (=foment=)
• [[http://gambitscheme.org/wiki/index.php/Main_Page][GambitC]] (=gambitc=)
• [[http://practical-scheme.net/gauche/][Gauche]] (=gauche=)
• [[http://www.gnu.org/software/guile/][Guile]] (=guile=)
• [[https://github.com/leppie/IronScheme][IronScheme]] (=ironscheme=)
• [[http://www.gnu.org/software/kawa/][Kawa]] (=kawa=)
• [[http://www.larcenists.org/][Larceny]] (=larceny=)
• [[https://scheme.fail/][Loko]] (=loko=)
• [[https://www.gnu.org/software/mit-scheme/][MIT/GNU]] Scheme (=mit=)
• [[http://mosh.monaos.org][Mosh]] (=mosh=)
• [[http://scheme.com/][Petite Chez]] (=chez=)
• [[https://github.com/picrin-scheme/picrin][Picrin]] (=picrin=)
• [[https://racket-lang.org][Racket]] (=racket=)
• [[http://www.kt.rim.or.jp/~qfwfq/rhiz-pi/index-e.html][Rhizome/Pi]] (=rhizome=)
• [[http://www.rscheme.org][RScheme]] (=rscheme=)
• [[https://bitbucket.org/ktakashi/sagittarius-scheme/wiki/Home][Sagittarius]] (=sagittarius=)
• [[http://s48.org][Scheme48]] (=scheme48=)
• [[http://t3x.org/s9fes/][Scheme 9 from Empty Space]] (=s9fes=)
• [[http://marcomaggi.github.io/vicare.html][Vicare]] (=vicare=)
• [[http://www.littlewingpinball.net/mediawiki/index.php/Ypsilon][Ypsilon]] (=ypsilon=) This should result in a file =./results.=.

All of these Schemes are packaged for ArchLinux. The tests are run with whatever that package produces.

Run #+BEGIN_SRC make csv #+END_SRC to produce a file =all.csv= with the test results (=,,=).

** Limiting compile-time and run-time The environment variable =CPU_TIME= can be set, to limit the time for compilation and for running (each individually). #+BEGIN_SRC CPU_TIME=3600 ./bench chez all #+END_SRC ** Path to executables You can set an environment variable to specify the path to each Scheme. #+BEGIN_SRC LARCENY=/tmp/larceny/larceny MIT=/tmp/mit/mit-scheme ./bench "mit larceny" "fib ack" #+END_SRC ** Notes for specific implementations *** Chicken The chicken test code assumes the presence of

• vector-lib
• r7rs

Install them by running #+BEGIN_SRC chicken-install vector-lib r7rs #+END_SRC *** Racket The racket test code assumes the presence of https://github.com/lexi-lambda/racket-r7rs.git.

Install the r7rs package installation-wide.

Run: #+BEGIN_SRC sudo raco pkg install --scope installation r7rs #+END_SRC

*** IronScheme Precompile system libraries (optional). It is still going to be slow. #+BEGIN_SRC echo "(compile-system-libraries)" | mono IronScheme.Console-v4.exe #+END_SRC The default binary for IronScheme (can be changed by exporting IRONSCHEME in your shell before running bench) is called =ironscheme=.

If you unpack IronScheme in your home, you can create a shell script like the following: #+BEGIN_SRC #!/bin/bash mono ~/IronScheme/IronScheme.Console-v4.exe -nologo -emacs $* #+END_SRC and put that somewhere in your =$PATH=. *** Stalin Before calling Stalin, the code is run through [[http://petrofsky.org/src/alexpander.scm][alexpander]] (by default using Chicken's =csi=). Then stalin is invoked on the result. ** Unsupported Schemes *** Owl Lisp Missing a lot of necessary functions, =read= for example. *** TinyScheme Problems with redirecting input, =read= chokes. No timing functions. ** Safety The benchmarking code does not use unsafe optimizations. These make code run even faster (but less safe ;) Specific options that could be used are:

• Chez :: =--optimize-level 3=
• Gerbil / Gambit :: =(declare (not safe))=