puppeteer-ruby icon indicating copy to clipboard operation
puppeteer-ruby copied to clipboard

Use a global thread executor

Open Overload119 opened this issue 2 years ago • 1 comments

Simple description about the feature

Introduce a Concurrent::FixedThreadPool for async tasks.

Usecase / Motivation

I've been using this library in a background job on Sidekiq. Sometimes the jobs hang and Sidekiq stops. I posted about this on StackOverflow.

One of the problems is I have a ridiculous number of threads (500+) on my Ruby process that's long running.

The source of those threads I believe is this library. It uses the default thread pools for concurrent operations.

See the following

500.times { Concurrent::Promise.execute { puts "Hi" } }
puts Thread.list.count # ~500
# Restart Ruby process and now try:
pool = Concurrent::FixedThreadPool.new(5)
500.times { Concurrent::Promise.execute(executor: pool) { puts "Hi" } }
puts Thread.list.count # ~13

This is far more efficient for the library.

Overload119 avatar Jul 15 '22 17:07 Overload119

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Aug 11 '22 22:08 stale[bot]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Sep 21 '22 01:09 stale[bot]