appsignal-ruby icon indicating copy to clipboard operation
appsignal-ruby copied to clipboard

Published 20 hours ago • verified publisher icon appsignal

appsignal agent process utilises 100% of CPU when using rails console

Open shairyar opened this issue 4 years ago • 16 comments

When running the rails console on the container, the appsignal-agent process begins to utilize 100%+ of CPU,

The customer using version 2.11.4 Diagnostic report: https://appsignal.com/admin/diagnose_reports/diagdaa54708f6b8b159c6b5 appsignal.log: https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/conversation/16410700023825#part_id=comment-16410700023825-7789530617

shairyar avatar Feb 01 '21 10:02 shairyar

https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/4356044/conversations/16410700023825

shairyar avatar Feb 01 '21 10:02 shairyar

The customer is now running into this even when they are not using rails console.

shairyar avatar Apr 12 '21 10:04 shairyar

Thijs says, we read the socket but it's empty, so it keeps retrying very quickly, causing a 100% CPU usage. Let's see if we can add a small sleep in that retry action to avoid the 100% CPU. It wouldn't solve the communication issue that's occurring here.

tombruijn avatar Apr 19 '21 11:04 tombruijn

@jeffkreeftmeijer you can also try to set up a similar test setup as for the FreeBSD issue which recreated the mio create issue to try and reproduce it: https://github.com/appsignal/rust-library-mio-freebsd-test-setup You'll probably need to copy-paste in the code that probably causes it, and run it on a Ubuntu VM instead. This issue is not specific to the FreeBSD OS, the other issue https://github.com/appsignal/support/issues/118 is.

tombruijn avatar May 10 '21 11:05 tombruijn

I have another customer who ran into this issue where the usage shoots up and the host metrics load avg shows 0, they are using the latest version of Ruby integration.

Intercom: https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/4356044/conversations/16410700079256 Diagnose report: https://appsignal.com/admin/diagnose_reports/diag78b5be17f88bf3e70375

I am also seeing the same error in their report which I see in the report of a customer who reported this issue earlier.

[2021-12-13T23:59:59 (agent) #10][Error] Error getting disk IO stats: No such file or directory (os error 2) for /proc/diskstats
[2021-12-14T00:00:00 (agent) #10][Error] Error getting disk IO stats: No such file or directory (os error 2) for /proc/diskstats

shairyar avatar Dec 14 '21 12:12 shairyar

After a chat with @thijsc, this is due to a bug from mio Rust crate.

We'll cover it with https://github.com/appsignal/appsignal-agent/issues/708

luismiramirez avatar Jan 12 '22 13:01 luismiramirez

While performing the daily checks some issues were found with this issue.

  • This issue has not had any activity in 14 days. Please provide a status update if it is still relevant. Closed it if it is no longer relevant. Or move it to another column if it's blocked or requires another look at it. - (More info)

New issue guide | Backlog management | Rules | Feedback

backlog-helper[bot] avatar Jan 27 '22 08:01 backlog-helper[bot]

https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/4356044/conversations/16410700090078

shairyar avatar Feb 08 '22 08:02 shairyar

While performing the daily checks some issues were found with this issue.

  • This issue has not had any activity in 14 days. Please provide a status update if it is still relevant. Closed it if it is no longer relevant. Or move it to another column if it's blocked or requires another look at it. - (More info)

New issue guide | Backlog management | Rules | Feedback

backlog-helper[bot] avatar Feb 23 '22 08:02 backlog-helper[bot]

While performing the daily checks some issues were found with this issue.

  • This issue has not had any activity in 14 days. Please provide a status update if it is still relevant. Closed it if it is no longer relevant. Or move it to another column if it's blocked or requires another look at it. - (More info)

New issue guide | Backlog management | Rules | Feedback

backlog-helper[bot] avatar Mar 17 '22 08:03 backlog-helper[bot]

While performing the daily checks some issues were found with this issue.

  • This issue has not had any activity in 14 days. Please provide a status update if it is still relevant. Closed it if it is no longer relevant. Or move it to another column if it's blocked or requires another look at it. - (More info)

New issue guide | Backlog management | Rules | Feedback

backlog-helper[bot] avatar Apr 04 '22 07:04 backlog-helper[bot]

https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/4356044/conversations/16410700105029

shairyar avatar Apr 13 '22 05:04 shairyar

https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/4356044/conversations/16410700105480

The only thing I could consider to be related is that I triggered a manual garbage collection (GC.start) shortly before it happened

shairyar avatar Apr 14 '22 09:04 shairyar

https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/unassigned/conversations/16410700116700

shairyar avatar Apr 28 '22 08:04 shairyar

@luismiramirez can you follow up with the user? Do they still have the problem? They have upgraded the Ruby gem recently, maybe also the Ruby version itself. That may have resolved the issue.

tombruijn avatar May 10 '22 08:05 tombruijn

https://app.intercom.com/a/inbox/yzor8gyw/inbox/admin/4356044/conversation/16410700132061

shairyar avatar Jul 22 '22 05:07 shairyar

@shairyar do you need to follow up with the following conversations, or can we close the issue?

  • https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/conversation/16410700116700 / https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/conversation/16410700105480
  • https://app.intercom.com/a/apps/yzor8gyw/inbox/inbox/conversation/16410700079256

tombruijn avatar Aug 16 '22 07:08 tombruijn

This can be closed

shairyar avatar Aug 16 '22 08:08 shairyar