rust-web-frameworks-benchmark
rust-web-frameworks-benchmark copied to clipboard
Published
20 hours ago •
rousan
rousan
Rocket v5 (async) bencmark
I try to run the benchmark against the master branch of https://github.com/SergioBenitez/Rocket (v5). To see if there is any speed improvement. Because it's already async.
I run the benchmark to warp, to see my machine capability.
warp
❯ cd warp
❯ cargo build --release
❯ ./target/release/warp
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:3030
Running 8s test @ http://127.0.0.1:3030
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 817.75us 1.11ms 17.93ms 93.77%
Req/Sec 72.54k 7.78k 95.50k 68.12%
Latency Distribution
50% 519.00us
75% 844.00us
90% 1.41ms
99% 6.21ms
2314469 requests in 8.09s, 180.99MB read
Non-2xx or 3xx responses: 2314469
Requests/sec: 286126.89
Transfer/sec: 22.38MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:3030
Running 8s test @ http://127.0.0.1:3030
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.86ms 1.03ms 16.82ms 93.42%
Req/Sec 65.72k 6.99k 83.28k 63.75%
Latency Distribution
50% 580.00us
75% 0.93ms
90% 1.48ms
99% 5.55ms
2097890 requests in 8.09s, 164.06MB read
Non-2xx or 3xx responses: 2097890
Requests/sec: 259381.43
Transfer/sec: 20.28MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:3030
Running 8s test @ http://127.0.0.1:3030
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.93ms 1.10ms 17.71ms 93.35%
Req/Sec 60.40k 7.81k 82.04k 65.00%
Latency Distribution
50% 632.00us
75% 1.01ms
90% 1.62ms
99% 6.10ms
1926600 requests in 8.09s, 150.66MB read
Non-2xx or 3xx responses: 1926600
Requests/sec: 238068.17
Transfer/sec: 18.62MB
Rocket v4
❯ cd rocket
❯ cargo +nightly build --release
❯ ./target/release/rocket
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 786.81us 445.55us 14.41ms 70.86%
Req/Sec 18.22k 734.43 22.02k 78.75%
Latency Distribution
50% 799.00us
75% 1.09ms
90% 1.28ms
99% 1.96ms
580046 requests in 8.05s, 80.76MB read
Socket errors: connect 0, read 580030, write 0, timeout 0
Requests/sec: 72045.45
Transfer/sec: 10.03MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.92ms 3.56ms 205.67ms 99.80%
Req/Sec 17.09k 0.91k 19.49k 68.44%
Latency Distribution
50% 809.00us
75% 1.16ms
90% 1.38ms
99% 2.08ms
544051 requests in 8.04s, 75.75MB read
Socket errors: connect 0, read 544036, write 0, timeout 0
Requests/sec: 67683.32
Transfer/sec: 9.42MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.98ms 2.43ms 209.81ms 99.43%
Req/Sec 15.55k 1.48k 19.28k 90.62%
Latency Distribution
50% 0.91ms
75% 1.29ms
90% 1.53ms
99% 2.69ms
495140 requests in 8.04s, 68.94MB read
Socket errors: connect 0, read 495120, write 0, timeout 0
Requests/sec: 61560.47
Transfer/sec: 8.57MB
Rocket v5
❯ cd rocket
❯ # change Cargo.toml
❯ # change main.rs to work with v5
❯ cargo build --release
❯ ./target/release/rocket
@@ -1,5 +1,3 @@
-#![feature(proc_macro_hygiene, decl_macro)]
-
#[macro_use]
extern crate rocket;
@@ -8,6 +6,8 @@ fn index() -> &'static str {
"Hello, world!"
}
-fn main() {
- rocket::ignite().mount("/", routes![index]).launch();
+#[launch]
+fn rocket() -> _ {
+ rocket::build()
+ .mount("/", routes![index])
}
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.30ms 1.03ms 26.44ms 86.44%
Req/Sec 39.56k 3.79k 48.86k 67.19%
Latency Distribution
50% 1.08ms
75% 1.61ms
90% 2.33ms
99% 5.05ms
1261745 requests in 8.08s, 175.68MB read
Requests/sec: 156093.58
Transfer/sec: 21.73MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.38ms 0.95ms 28.54ms 79.82%
Req/Sec 36.52k 3.16k 45.96k 67.81%
Latency Distribution
50% 1.18ms
75% 1.74ms
90% 2.46ms
99% 4.83ms
1164811 requests in 8.07s, 162.18MB read
Requests/sec: 144278.36
Transfer/sec: 20.09MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.43ms 0.99ms 21.53ms 79.25%
Req/Sec 35.24k 3.21k 48.04k 69.38%
Latency Distribution
50% 1.22ms
75% 1.81ms
90% 2.57ms
99% 5.05ms
1124770 requests in 8.08s, 156.61MB read
Requests/sec: 139211.46
Transfer/sec: 19.38MB
Rocket v5 (ROCKET_ENV=prod)
❯ cd rocket
❯ # change Cargo.toml
❯ # change main.rs to work with v5
❯ cargo build --release
❯ env ROCKET_ENV=prod ./target/release/rocket
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.21ms 0.92ms 21.33ms 84.57%
Req/Sec 42.26k 3.27k 50.97k 74.38%
Latency Distribution
50% 1.02ms
75% 1.50ms
90% 2.15ms
99% 4.61ms
1348461 requests in 8.08s, 187.75MB read
Requests/sec: 166937.68
Transfer/sec: 23.24MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.32ms 0.94ms 23.26ms 81.14%
Req/Sec 38.34k 3.62k 47.96k 69.38%
Latency Distribution
50% 1.12ms
75% 1.66ms
90% 2.37ms
99% 4.79ms
1223178 requests in 8.08s, 170.31MB read
Requests/sec: 151402.49
Transfer/sec: 21.08MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.45ms 1.00ms 19.32ms 79.40%
Req/Sec 34.91k 3.04k 44.97k 71.25%
Latency Distribution
50% 1.24ms
75% 1.83ms
90% 2.59ms
99% 5.15ms
1113370 requests in 8.06s, 155.02MB read
Requests/sec: 138150.06
Transfer/sec: 19.24MB
Rocket v5 (witch async keyword)
❯ cd rocket
❯ # change Cargo.toml
❯ # change main.rs to work with v5
❯ cargo build --release
❯ env ROCKET_ENV=prod ./target/release/rocket
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.26ms 0.94ms 18.77ms 83.92%
Req/Sec 40.41k 3.72k 49.53k 72.50%
Latency Distribution
50% 1.07ms
75% 1.57ms
90% 2.23ms
99% 4.80ms
1289197 requests in 8.08s, 179.50MB read
Requests/sec: 159601.93
Transfer/sec: 22.22MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.41ms 0.99ms 18.31ms 80.39%
Req/Sec 35.79k 3.46k 46.59k 73.44%
Latency Distribution
50% 1.21ms
75% 1.78ms
90% 2.52ms
99% 5.11ms
1142153 requests in 8.08s, 159.03MB read
Requests/sec: 141319.52
Transfer/sec: 19.68MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.52ms 1.09ms 29.17ms 80.94%
Req/Sec 33.37k 3.86k 45.75k 82.81%
Latency Distribution
50% 1.29ms
75% 1.92ms
90% 2.73ms
99% 5.45ms
1065163 requests in 8.07s, 148.31MB read
Requests/sec: 131921.73
Transfer/sec: 18.37MB
Summary
warp: 286_126.89, 259_381.43, 238_068.17
Rocket v4: 72_045.45, 67_683.32, 61_560.47
Rocket v5: 156_093.58, 144_278.36, 139_211.46
Rocket v5 (ROCKET_ENV=prod): 166_937.68, 151_402.49, 138_150.06
Rocket v5 (witch async keyword): 159_601.93, 141_319.52, 131_921.73
Machine
I have made my machine quite before running the benchmark.
❯ macchina
Machine — LENOVO ThinkPad L390 20NSS0TA00
Kernel — Linux 4.19.0-14-amd64
Distro — Debian GNU/Linux
WM — i3
CPU — Intel® Core™ i7-8565U CPP
Memory — 3.3 GB/15.9 GB
Notes
IMHO, to make the command wrk --latency -t4 -c200 -d8s http://127.0.0.1:8080 consistent. We need to set each port of the web framework to 8080
At this point I think we are testing different things. So I change warp codebase to match Rocket.
modified warp/src/main.rs
@@ -2,8 +2,8 @@ use warp::Filter;
#[tokio::main]
async fn main() {
- // GET /hello/warp => 200 OK with body "Hello, warp!"
- let hello = warp::path!("hello" / String).map(|name| format!("Hello, {}!", name));
+ // GET / => 200 OK with body "Hello, wolrd!"
+ let hello = warp::any().map(|| "Hello, world!");
- warp::serve(hello).run(([127, 0, 0, 1], 3030)).await;
+ warp::serve(hello).run(([127, 0, 0, 1], 8000)).await;
}
The result is:
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 780.43us 0.99ms 15.08ms 93.28%
Req/Sec 73.99k 7.81k 98.49k 69.69%
Latency Distribution
50% 505.00us
75% 821.00us
90% 1.37ms
99% 5.50ms
2361359 requests in 8.08s, 292.76MB read
Requests/sec: 292284.41
Transfer/sec: 36.24MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.87ms 1.08ms 17.02ms 93.52%
Req/Sec 66.45k 7.54k 91.47k 71.16%
Latency Distribution
50% 574.00us
75% 0.92ms
90% 1.50ms
99% 6.12ms
2114053 requests in 8.08s, 262.10MB read
Requests/sec: 261637.18
Transfer/sec: 32.44MB
❯ wrk --latency -t4 -c200 -d8s http://127.0.0.1:8000
Running 8s test @ http://127.0.0.1:8000
4 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.94ms 1.15ms 19.01ms 93.58%
Req/Sec 60.90k 7.59k 80.80k 76.88%
Latency Distribution
50% 630.00us
75% 1.01ms
90% 1.61ms
99% 6.33ms
1942062 requests in 8.08s, 240.77MB read
Requests/sec: 240399.79
Transfer/sec: 29.80MB