riemann-tools icon indicating copy to clipboard operation
riemann-tools copied to clipboard

Add a wrapper to run multiple tools in a single process

Open smortex opened this issue 2 years ago • 0 comments

A Ruby process has an entry memory cost of about 15 Mb in resident memory size (RSS). This is for a process that does nothing and wait for input on stdin. When running actual code, this grows with each file loading and object creation.

Currently, a single riemann-tool process consume about 30 Mb of RSS. When multiple tools are running simultaneously, these RSS sizes sum up.

Add a generic wrapper tool that allows to run multiple tools from a single Ruby process, reducing the memory footprint required for the same workload.

Profiling

Before

bundle exec bin/riemann-diskstats &
bundle exec bin/riemann-fd &
bundle exec bin/riemann-health &
bundle exec bin/riemann-net &
bundle exec bin/riemann-ntp &
bundle exec bin/riemann-proc &

Relevant ps(1) output:

F   UID     PID    PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
0  1000   28617   28553  20   0  96168 31104 -      S+   pts/13     0:02 bin/riemann-proc
0  1000   28625   28571  20   0  95296 29956 -      S+   pts/14     0:00 bin/riemann-ntp
0  1000   28630   28495  20   0  95684 30712 -      S+   pts/10     0:00 bin/riemann-net
0  1000   28633   25039  20   0  95688 30384 -      S+   pts/9      0:00 bin/riemann-diskstats
0  1000   28635   28535  20   0  94388 29044 -      S+   pts/12     0:00 bin/riemann-fd
0  1000   28642   28517  20   0  96076 30792 -      S+   pts/11     0:00 bin/riemann-health

After

bundle exec bin/riemann-wrapper -- diskstats -- fd -- health -- net  -- ntp -- proc

Relevant ps(1) output:

F   UID     PID    PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
0  1000   28955   25039  20   0 500660 31424 -      Sl+  pts/9      0:00 bin/riemann-wrapper

This PR include:

  • #224

smortex avatar Aug 09 '22 21:08 smortex