yabeda-prometheus
yabeda-prometheus copied to clipboard
Support Yabeda server in a new process
Should fix #21
I've been working on getting yabeda into our app which runs on passenger. I finally have it running, but faced a couple of issues that I've been able to fix manually in our app but wanted to upstream some of those changes Issue 1: Thread gets killed by passenger as a long-running request Issue 2 β : Metrics become pretty bloated when we haven't deployed in a while I imagine this would be a problem on weekends where the app is up for days. The addition of rack deflater shrunk the size of the metrics logged by close to 90% in my testing:
With Rack deflater | Without Rack deflater |
---|---|
1355 | 15918 |
1538 | 16851 |
1582 | 17784 |
Depending on your scrape timeout on your prometheus server this could or could not be a problem ours is set pretty low but has worked fine for other exporters we have.
Issue 3[Just a minor annoyance π
]: When running in clustered mode if app isn't preloaded the other processes or threads error out with Errno::EADDRINUSE
doesn't kill the app just pollutes the start up logs a bit
Changes
- add support for turning on rack deflater for the rack app
- add support for starting the metrics server in a separate process
- add support for suppressing Errno::EADDRINUSE when running in clustered mode in production apps
Gearing up to go live Monday with these changes and a couple others others in relation to resque on Monday so any feedback would be greatly appreciated.
Shameless plug: because we use resque I wrote a small extension for metrics around activejob for us to get some insight similar to what yβall have done for sidekiq and faktory. Check it out here
I could also split this up into multiple PRs if that makes it easier to review unsure who to add as a reviewer though @Envek are you the person to review? π
Also added your yabeda-activejob to the Yabeda plugin list. Thanks for sharing!
Awesome! Apologies for taking so long to get back to this was at a work conference for the past couple days getting those changes in, test some things around calling configure!
over the next little bit. I'll ping you once it's ready for another