php-resque icon indicating copy to clipboard operation
php-resque copied to clipboard

Published 20 hours ago • verified publisher icon resque

Implement Child Batching

Open danhunsaker opened this issue 6 years ago • 4 comments

See chrisboulton/php-resque#35 for details and an early implementation.

In a nutshell, reuse job processes for up to X jobs rather than forking for every one.

danhunsaker avatar Dec 11 '18 17:12 danhunsaker

In a nutshell, reuse job processes for up to X jobs rather than forking for every one.

The X-factor could be multiple conditions:

  • number of jobs executed
  • total runtime of the child
  • memory usage (i.e. only execute jobs as long as certain threshold isn't met)

Such a change is definitely a game changer for performance.

mfn avatar Dec 11 '18 19:12 mfn

Indeed so! Looking forward to seeing what comes of this one.

danhunsaker avatar Dec 11 '18 20:12 danhunsaker

Hi @danhunsaker, I am busy these day. I will have install everything and start working on it from this weekend. Can we create slack or discord channel to talk realtime ?

rajibahmed avatar Dec 12 '18 09:12 rajibahmed

That sounds like an excellent idea. I'll get you the details once that's created.

danhunsaker avatar Dec 12 '18 12:12 danhunsaker