[Ruby] Should we update the Rails/Ruby tests to use YJIT?

[crimson-knight]

Since we recently got all of the Rails tests to run using the "optimized" production settings, should we also include YJIT to showcase the optimal Ruby configuration?

I think it would be interesting to see if it makes a difference.

[waghanza]

I think there was a gap down for 3.3, when you compare https://web-frameworks-benchmark.netlify.app/result?f=rails-api,railsfor december 2023 and january 2024.

Worth to investigate before enabling jit

[waghanza]

Maybe we should not @crimson-knight

Let me explain ...

Here is some figures

• rails without yjit

wrk -H 'Connection: keep-alive' --connections 64 --threads 8 --duration 15 --timeout 1 --script /home/waghanza/workspace/benchmark/web/pipeline.lua http://127.0.0.1:3000
Running 15s test @ http://127.0.0.1:3000
  8 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.82ms   11.31ms 169.36ms   84.08%
    Req/Sec   770.87    103.75     1.08k    79.40%
  92103 requests in 15.02s, 29.59MB read
Requests/sec:   6133.22
Transfer/sec:      1.97MB

• rails with yjit

wrk -H 'Connection: keep-alive' --connections 64 --threads 8 --duration 15 --timeout 1 --script /home/waghanza/workspace/benchmark/web/pipeline.lua http://127.0.0.1:3000
Running 15s test @ http://127.0.0.1:3000
  8 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.82ms    9.21ms 194.68ms   84.53%
    Req/Sec     1.19k   139.76     1.72k    91.25%
  141770 requests in 15.01s, 45.54MB read
Requests/sec:   9443.41
Transfer/sec:      3.03MB

• rails (api mode) without yjit

wrk -H 'Connection: keep-alive' --connections 64 --threads 8 --duration 15 --timeout 1 --script /home/waghanza/workspace/benchmark/web/pipeline.lua http://127.0.0.1:3000
Running 15s test @ http://127.0.0.1:3000
  8 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.82ms   15.89ms 160.35ms   83.51%
    Req/Sec     1.10k   169.65     1.80k    75.83%
  131897 requests in 15.01s, 42.28MB read
Requests/sec:   8786.10
Transfer/sec:      2.82MB

• rails (api mode) with yjit

wrk -H 'Connection: keep-alive' --connections 64 --threads 8 --duration 15 --timeout 1 --script /home/waghanza/workspace/benchmark/web/pipeline.lua http://127.0.0.1:3000
Running 15s test @ http://127.0.0.1:3000
  8 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.70ms   10.87ms 140.24ms   82.88%
    Req/Sec     1.69k   186.22     2.66k    77.90%
  201473 requests in 15.01s, 64.58MB read
Requests/sec:  13424.50
Transfer/sec:      4.30MB

YJIT is a game changer, but not sure if this is our responsibility (I mean in this project) to override default.

At least in rails 7.2, yjit will be enabled by default.

Will it be the same for other, I do not kown

cc @jodosha @jeremyevans @adamluzsi

[jeremyevans]

I think it's a good idea to enable yjit by default.

[jodosha]

@waghanza Since now on, please ping @timriley for Hanami related questions. Thanks 🙂 .