deliciousbrains
Added check for is_process_running() in dispatch()
Instead of checking $this->is_process_running() in maybe_handle(), it makes more sense for performance to check before dispatching the async request at all.
In our case, using WP Background Processing to handle incoming webhooks from CRMs, we're using
$process->push_to_queue( $webhook );
$process->save()->dispatch();
to store each webhook. So each incoming requests spawned a new async request, and most of them then died immediately because the process was already running.. With 20+ requests in a minute, we'd start to get timeout errors.
This little change has reduced the server load by about 90% and now we can easily handle 100+ incoming webhooks in a minute 😎
Does this update require any special configuration? Or would it be generally safe to update for anyone using this library?
@arobbins It should be safe for everyone. I can't think of a scenario where you'd want to dispatch a new request when the background worker is already running. With the original way, it does get killed right away (https://github.com/deliciousbrains/wp-background-processing/blob/master/classes/wp-background-process.php#L165) but that uses up an extra request... it can eat up quite a lot of resources.
We tested with 100 incoming API requests in a minute, each one handled like
$process->push_to_queue( $webhook );
$process->save()->dispatch();
With the original WP Background Processing the server times out after 17 requests, since each request is spawning a new async request:
With this change it handles all 100 with no problem:
Thanks! Very interesting. What about the call to is_process_running inside handle_cron_healthcheck?
I'm guessing it would still be necessary to check it here?
Yeah. It's still necessary there. Because the cron job runs every 5 mins.... it'd be possible in theory for a long-running operation to still be running by the time the first cron kicks in, so checking is_process_running at that point ensures the cron task doesn't start processing the same records in parallel 👍
