beats icon indicating copy to clipboard operation
beats copied to clipboard

[agentbeat/linux] shirou/gopsutil causes high CPU usage

Open florianl opened this issue 1 year ago • 5 comments

agentbeat uses shirou/gopsutil to fetch network information.

When testing a current release, the following profile from agentbeat was collected.

20240426-091526

From this on CPU profile, agentbeat spends most of its CPU resources on traversing /proc/net and /proc/<PID>/net in order to read files to fetch networking information.

Long standing issues on shirou/gopsutil to address performance issues when collecting network information:

  • https://github.com/shirou/gopsutil/issues/784
  • https://github.com/shirou/gopsutil/issues/695

florianl avatar Apr 29 '24 06:04 florianl

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

elasticmachine avatar Apr 29 '24 09:04 elasticmachine

Thx @florianl for your feedback. Did you observe a worse behavior than standalone Beats or are we having the same CPU consumption? cc @blakerouse

pierrehilbert avatar Apr 29 '24 13:04 pierrehilbert

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

elasticmachine avatar Apr 29 '24 13:04 elasticmachine

It is hard to compare the standalone agentbeat with a collection of multiple beats. Before the change to agentbeat each beat was more independent and could use its available resources. Now all resources are channeled and internals of agentbeat are competing with each other for these resources. From that perspective, I would say, the behavior is worse than before.

florianl avatar Apr 29 '24 13:04 florianl

Interesting, we haven't changed the number of processes (and in theory the work done by those processes), only the executable used to implement those processes.

cmacknz avatar Apr 29 '24 13:04 cmacknz

As @cmacknz please compare this to native to metricbeat, because the number of processes and the work each process does has not changed.

blakerouse avatar May 09 '24 19:05 blakerouse