Parameterize resources charts

[botandrose]

Hello, thank you very much for rails_performance! You've done the Rails world a great service by open sourcing this, and I really appreciate it!

Summary

This PR is a refactoring of the resources tab, aimed at enabling adding arbitrary charts to it from outside.

Motivation

My app is running the Phusion Passenger http server, and a key metric of Passenger to monitor is the http request queue. If its ever more than 0, that means that the server is being overloaded and requests are backing up. I want to add this graph to the resource tab so I can correlate it to CPU and RAM usage.

Most users of this gem are probably not running Passenger, so it probably doesn't make sense to add this chart directly to the rails_performance gem. Instead, I think it'd be better to add the chart at the app configuration level. However, the resource charts are currently hard-coded, so its not practical to monkeypatch new ones in.

Example

Here is how this PR enables monkeypatching a new chart in the app's initializer:

# config/initializers/rails_performance.rb
if defined?(RailsPerformance)
  RailsPerformance.setup do |config|
    ...

    config.system_monitor_charts = [
      "HTTPQueueChart",
      "CPUChart",
      "MemoryChart",
      "DiskChart",
    ]
  end

  class HTTPQueueChart < RailsPerformance::Reports::ResourcesReport::Chart
    def initialize server
      super(
        server:,
        key: :http_queue,
        type: "Usage",
        subtitle: "HTTP Queue",
        description: "HTTP queue length",
        legend: "Length"
      )
    end

    def signal e
      e[:http_queue].to_i
    end
  end

  RailsPerformance::Extensions::ResourceMonitor.prepend Module.new {
    def payload
      super.merge(http_queue: fetch_http_queue_length)
    end

    def fetch_http_queue_length
      `sudo passenger-status`.match(/Requests in queue: (\d+)/)[1].to_i
    rescue => e
      ::Rails.logger.error "Error fetching http queue length: #{e.message}"
      0
    end
  }
end

Final thoughts

This is mostly a proof-of-concept to get a conversation started with you. I imagine you probably have opinions about how best to enable something like this, and they likely differ from what I've done here.

So what do you think? Is this level of configuration and extension a direction you'd like to see rails_performance going?

[botandrose]

I've just pushed another two commits that enable configuration of the resource charts, and edited my above comment to reflect that change. This also allows easy reordering, removal, or replacement of the default resource charts.

[igorkasyanchuk]

Hi @botandrose I think this is an interesting idea, maybe it can be configurable with different "monitors" (even existing)

Can you please share screenshot how it looks on production?

[botandrose]

Sure! Here's a screenshot of the entire resources page, zoomed out to 50% so you can see the whole thing. I've customized this page in the initializer in three ways:

1. New HTTP Queue chart, as described above
2. Tweaked the CPU Average chart description with better wording and added a count of current vCPUs
3. Replaced the per-process memory chart with a new system memory chart

[botandrose]

Here's the initializer for the above, in case you're curious:

# config/initializers/rails_performance.rb
if defined?(RailsPerformance)
  RailsPerformance.setup do |config|
    ...
    config.system_monitor_charts = [
      "HTTPQueueChart",
      "MyCPUChart",
      "SystemMemoryChart",
      "DiskChart",
    ]
  end

  class HTTPQueueChart < RailsPerformance::Reports::ResourcesReport::Chart
    def initialize server
      super(
        server:,
        key: :http_queue,
        type: "Usage",
        subtitle: "HTTP Queue",
        description: "HTTP queue length",
        legend: "Length"
      )
    end

    def signal e
      e[:http_queue].to_i
    end
  end

  # tweaks
  class MyCPUChart < RailsPerformance::Reports::ResourcesReport::CPUChart
    # def type = "Usage" # why does this break things

    def description
      @@count ||= Sys::CPU.processors.length
      "CPU load average (1 min), for #{@@count} vCPUs"
    end
  end

  class SystemMemoryChart < RailsPerformance::Reports::ResourcesReport::Chart
    def initialize server
      super(
        server:,
        key: :system_memory,
        type: "Percentage",
        subtitle: "System Memory",
        description: "System memory usage (percentage)",
        legend: "Percentage"
      )
    end

    def signal e
      return 0 unless e[:system_memory]
      total = e[:system_memory]["mem_total"]
      used = total - e[:system_memory]["mem_available"]
      percentage = used.to_f / total * 100
      percentage.round(2)
    end
  end

  RailsPerformance::Extensions::ResourceMonitor.prepend Module.new {
    def payload
      super.merge({
        http_queue: fetch_http_queue_length,
        system_memory: fetch_system_memory,
      })
    end

    def fetch_http_queue_length
      `sudo passenger-status`.match(/Requests in queue: (\d+)/)[1].to_i
    rescue => e
      ::Rails.logger.error "Error fetching http queue length: #{e.message}"
      0
    end

    def fetch_system_memory
      File.open("/proc/meminfo").reduce({}) do |info, line|
        key, value = line.split(":").map(&:strip)
        key = key.underscore.to_sym
        value = value.to_i / 1024 # convert from KB to MB
        info.merge(key => value)
      end
    end
  }
end

[igorkasyanchuk]

@botandrose could you please update your branch? I tried it locally with your config

And I have this.

And also

So it would be great to have some instructions in the readme with possible options.

---

Finally maybe you can try to improve default charts (change labels, etc, but with default gems that are mentioned already in readme. To avoid issues like I posted with CPU count or memory info).

[botandrose]

@igorkasyanchuk Hi Igor, sorry for the late reply here.

I've rebased this branch on current master and its green.

Yeah that exact config I shared is my very application-specifc config, so I'm not surprised it didn't work for you. For example, its calling out to sudo passenger-status to get the number of http requests that are backed up in the Phusion Passenger http request queue. If you're not using Phusion Passenger as your app server, this will not work, as you have experienced! Also yes, it's using the sys-cpu ruby gem, which needs to be in the Gemfile. Again, this is specific to my exact application.

But that's the whole point of this PR, right? It allows people to customize rails_performance with application-specific monitors and graphs that are not generic enough to ship in the repo itself. I'm sure that nearly everyone wants to measure something unique to their app, and this PR is designed to make that possible.

But it sounds like you like this idea enough, so I'll move forward with two things:

1. Create a way to register new monitors, so we don't have to monkeypatch with RailsPerformance::Extensions::ResourceMonitor.prepend
2. Update the README to document these extension methods

[botandrose]

I've made some progress here, and here is my updated initializer. This is now done without any monkeypatching. However, I'm seeing a pattern here that would be nice to reify: setting up a Chart and its corresponding Monitor. Seems like maybe this can be simplified to something like a ResourceChart, which would define both how to gather the data and also how to display it in one single class. This would also simplify configuration, too. I'm going to try to work that out.

if defined?(RailsPerformance)
  RailsPerformance.setup do |config|
    # ...

    # If enabled, the system monitor will be displayed on the dashboard
    # to enabled add required gems (see README)
    config.system_monitor_duration = 24.hours
    config.system_monitor_charts = [
      "HTTPQueueChart",
      "MyCPUChart",
      "SystemMemoryChart",
      "DiskChart",
    ]

    config.resource_monitors = [
      "PumaHttpQueueMonitor",
      "CPUMonitor",
      "MyMemoryMonitor",
      "DiskMonitor"
    ]
  end

  class HTTPQueueChart < RailsPerformance::Reports::ResourcesReport::Chart
    def initialize server
      super(
        server:,
        key: :http_queue,
        type: "Usage",
        subtitle: "HTTP Queue",
        description: "HTTP queue length",
        legend: "Length"
      )
    end

    def signal e
      e[:http_queue].to_i
    end
  end

  class PumaHttpQueueMonitor
    def call
      { http_queue: fetch_http_queue_length }
    end

    def fetch_http_queue_length
      lines = `pumactl stats`.split("\n")
      json_line = lines.find { |line| line.strip.start_with?('{') }
      stats = JSON.parse(json_line)
      stats['worker_status'].sum { |worker| worker['last_status']['backlog'] }
    rescue => e
      ::Rails.logger.error "Error fetching http queue length: #{e.message}"
      0
    end
  end

  require "sys-cpu"
  class MyCPUChart < RailsPerformance::Reports::ResourcesReport::CPUChart
    # def type = "Usage" # why does this break things

    def description
      @@count ||= Sys::CPU.processors.length
      "CPU load average (1 min), for #{@@count} vCPUs"
    end
  end

  class SystemMemoryChart < RailsPerformance::Reports::ResourcesReport::Chart
    def initialize server
      super(
        server:,
        key: :system_memory,
        type: "Percentage",
        subtitle: "System Memory",
        description: "System memory usage (percentage)",
        legend: "Percentage"
      )
    end

    def signal e
      return 0 unless e[:system_memory]
      total = e[:system_memory]["mem_total"]
      used = total - e[:system_memory]["mem_available"]
      percentage = used.to_f / total * 100
      percentage.round(2)
    end
  end

  class MyMemoryMonitor
    def call
      { system_memory: fetch_system_memory }
    end

    def fetch_system_memory
      File.open("/proc/meminfo").reduce({}) do |info, line|
        key, value = line.split(":").map(&:strip)
        key = key.underscore.to_sym
        value = value.to_i / 1024 # convert from KB to MB
        info.merge(key => value)
      end
    end
  end
end

[botandrose]

@igorkasyanchuk Okay, I got something working that I mostly like. Chart and Monitor have been merged into ResourceChart. I think this probably could use some minor tweaks to naming, and maybe some to the configuration API as well. And I don't love the call to super in the initializer. Also I think it probably won't work if there's more than one server involved. But I think its a solid proof-of-concept for an making the system resources totally data-driven and extensible. What do you think about this direction?

if defined?(RailsPerformance)
  RailsPerformance.setup do |config|
    # ...

    # If enabled, the system monitor will be displayed on the dashboard
    # to enabled add required gems (see README)
    config.system_monitor_duration = 24.hours
    config.system_monitors = [
      "PumaHTTPQueueLength",
      "MyCPULoad",
      "SystemMemory",
      "DiskUsage",
    ]
  end

  class PumaHTTPQueueLength < RailsPerformance::SystemMonitor::ResourceChart
    def initialize server
      super(
        server:,
        key: :http_queue,
        type: "Usage",
        subtitle: "HTTP Queue",
        description: "HTTP queue length",
        legend: "Length"
      )
    end

    def format data
      data.to_i
    end

    def measure
      lines = `pumactl stats`.split("\n")
      json_line = lines.find { |line| line.strip.start_with?('{') }
      stats = JSON.parse(json_line)
      stats['worker_status'].sum { |worker| worker['last_status']['backlog'] }
    rescue => e
      ::Rails.logger.error "Error fetching http queue length: #{e.message}"
      0
    end
  end

  require "sys-cpu"
  class MyCPULoad < RailsPerformance::SystemMonitor::CPULoad
    def description
      @@count ||= Sys::CPU.processors.length
      "CPU load average (1 min), for #{@@count} vCPUs"
    end
  end

  class SystemMemory < RailsPerformance::SystemMonitor::ResourceChart
    def initialize server
      super(
        server:,
        key: :system_memory,
        type: "Percentage",
        subtitle: "System Memory",
        description: "System memory usage (percentage)",
        legend: "Percentage"
      )
    end

    def format data
      return 0 unless data
      total = data["mem_total"]
      used = total - data["mem_available"]
      percentage = used.to_f / total * 100
      percentage.round(2)
    end

    def measure
      File.open("/proc/meminfo").reduce({}) do |info, line|
        key, value = line.split(":").map(&:strip)
        key = key.underscore.to_sym
        value = value.to_i / 1024 # convert from KB to MB
        info.merge(key => value)
      end
    end
  end
end

[botandrose]

@igorkasyanchuk Its probably worth pointing out that the above config demonstrates four things:

1. Removing a default system monitor simply by leaving it out of the replacement array: MemoryUsage,
2. Tweaking an existing default system monitor by subclassing the original and using that instead: MyCpuLoad
3. Retaining a default system monitor: DiskUsage
4. Adding a new system monitor from scratch: PumaHTTPQueueLength

[igorkasyanchuk]

@botandrose thanks for the changes, but please give me a few days, I need to read it all, understand, and check. Hopefully will do it during this weekend.

[botandrose]

@igorkasyanchuk No rush!

[igorkasyanchuk]

@botandrose big sorry for a delay. Now it's merged, plan to release it today.

Thank you for your contribution.

[botandrose]

@igorkasyanchuk Awesome, https://bardtracker.com is now off of my fork, and back to using the mainline rails_performance gem! Thanks very much!

[igorkasyanchuk]

Great. I plan to add a few more features soon. Stay tuned) if you want to help I would appreciate that

[botandrose]

I'm interested! I'm not overflowing with free time at the moment, but I'd be happy to help review and maybe even pitch in.

[igorkasyanchuk]

@botandrose please check new feature if it works for you https://github.com/igorkasyanchuk/rails_performance?tab=readme-ov-file#deployment-events--custom-events-on-the-charts

I don't know how you deploy your app, maybe you can contribute with examples if this is anything except Kamal.

Plus we can extend the feature to support not only Deployment events (it's actually right now can be used for other purposes too)

[botandrose]

This is a great idea! I'll try it out on BARD Tracker

[igorkasyanchuk]

btw you can also try this gem https://github.com/igorkasyanchuk/async_render, released it recently and it can help with performance. If you really have this problem.

[botandrose]

Whoa, that's a super cool idea! I'm surprised I haven't seen it tried before.

I'm guessing it needs the main thread to have I/O wait, so that the GVL can switch. So it needs to wait for the controller action to do something like a slow db query to be worth it. Is that right? If so, do you think using Ractors could make it faster by increasing the parallelism?

On Fri, Aug 29, 2025, 11:21 Igor Kasyanchuk @.***> wrote:

igorkasyanchuk left a comment (igorkasyanchuk/rails_performance#134) https://github.com/igorkasyanchuk/rails_performance/pull/134#issuecomment-3236351436

btw you can also try this gem https://github.com/igorkasyanchuk/async_render, released it recently and it can help with performance. If you really have this problem.

— Reply to this email directly, view it on GitHub https://github.com/igorkasyanchuk/rails_performance/pull/134#issuecomment-3236351436, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAEH3YOECFOOKAYMR3KAF33QALPZAVCNFSM6AAAAAB2TJGLNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMZWGM2TCNBTGY . You are receiving this because you were mentioned.Message ID: @.***>

[igorkasyanchuk]

I tried to use a similar approach to what Rails internally uses for .load_async (and other async queries). To be honest don't have a lot of experience with parallelism in Ruby, and I think cost of creating a thread is acceptable if your partial is taking >50ms for example, to render. And if you have multiple like this - it will improve performance even more.

You can try to test it using this snippet

class ApplicationController < ActionController::Base before_action do AsyncRender.enabled = params[:skip_async].blank? end end

when disabled, it will use regular "render" under the hood.