API Benchmarks

This repository provides applications in various Ruby web frameworks for the purposes of easy benchmarking. All benchmarks are currently run using wrk, though other benchmarking tools may be used in the future.

Setup

Here's the software used to run these benchmarks:

• Ruby 2.1.2 (with bundler: gem install bundler)
• wrk

If one of these isn't installed:

# Ruby
$ brew install chruby ruby-install
$ ruby-install ruby 2.1.2
$ chruby ruby-2.1.2

# wrk
$ brew install wrk

Running benchmarks

First, set up the benchmarking database:

$ bundle install

Then, you can run benchmarks:

$ rake benchmark:all

# Or, run benchmarks for one framework
$ rake benchmark:rails

To run benchmarks on Heroku (recommended, so that the web servers are not competing with the load generation tool for resources), first set up Heroku:

$ brew install heroku
$ heroku create

Then, you can run benchmarks:

$ HEROKU=true rake benchmark:all

# Or, run benchmarks for one framework
$ HEROKU=true rake benchmark:rails

Benchmarking Information

All applications boot via rackup using the Puma dispatcher. Each application defines two endpoints:

• /empty (renders an empty response body)
• /numbers/:count (renders an array of :count numbers as JSON, defaults to 1000 numbers)

Each endpoint is hit consecutively for 3 minutes using wrk -t 2 -c 10 -d 3m.

Results

Note: I've ordered results by the average number of requests/sec each framework was able to handle. Response time seems to vary, and I'm unsure of whether the response time reported is actually an average or some other metric.

/empty

/numbers/1000

System Information

Load is generated on a 15" Retina MacBook Pro with the following specifications:

• CPU: 2.3 GHz Intel Core i7
• RAM: 16 GB 1600 MHz DDR3
• GPU: Intel HD Graphics 4000 1024 MB
• OS: Mac OS X 10.9.2

All web applications were run on Heroku, using a single free dyno.