poco icon indicating copy to clipboard operation
poco copied to clipboard

Motivated by "The load balancing issue in Poco::ActiveThreadPool" #4544

Open bas524 opened this issue 9 months ago • 6 comments

Optimization allows redistribute tasks to the idle threads When ActiveThread dequeue next Runnable, it checks is optimization is true and if yes it try to find idle thread and resend task By default optimization is false

bas524 avatar May 07 '24 08:05 bas524

Thanks for the contribution.

IMO partial refactoring of ActiveThreadPool (and related classes) to have a single task queue in the pool where the threads take the work from would automatically create optimal load-balancing and simplify the implementation at the same time.

matejk avatar May 07 '24 09:05 matejk

Thanks for the contribution.

IMO partial refactoring of ActiveThreadPool (and related classes) to have a single task queue in the pool where the threads take the work from would automatically create optimal load-balancing and simplify the implementation at the same time.

Hi! I thougt about it. If notificationqueue was lock-free than you proposition shoud be correct. In our case realisation with single queue would contains many locks and bad performance. May be flat combinig implementation is better choise for us...

bas524 avatar May 07 '24 10:05 bas524

This change appears to fail with the Thread Sanitizer. The actual log has much more in it https://github.com/pocoproject/poco/actions/runs/8982526627/job/24670333088?pr=4548.

testActiveThreadLoadBalancing: ==================
WARNING: ThreadSanitizer: data race (pid=33153)
  Read of size 8 at 0x7ffe63aa0b60 by thread T16:
    #0 Poco::ActiveThread::run() <null> (libPocoFoundation.so.103+0xf9197)
    #1 <null> <null> (libPocoFoundation.so.103+0x1cc80e)
    #2 Poco::ThreadImpl::runnableEntry(void*) <null> (libPocoFoundation.so.103+0x1cdebd)

andrewauclair avatar May 07 '24 19:05 andrewauclair

@bas524 Does the PR #4624 with different implementation that was merged to main satisfy the needs?

matejk avatar Aug 31 '24 13:08 matejk

@bas524 Does the PR #4624 with different implementation that was merged to main satisfy the needs?

Hi, I'll check it next week

bas524 avatar Sep 03 '24 07:09 bas524

@bas524 Does the PR #4624 with different implementation that was merged to main satisfy the needs?

@matejk I answered into https://github.com/pocoproject/poco/pull/4624#issuecomment-2408475465

bas524 avatar Oct 12 '24 09:10 bas524