chrisboulton
Execute jobs via php-fpm rather than forking
After a discussion with a co-worker i decided to write up a quick hack which executes jobs over php5-fpm instead of forking. This provides all the same protections as forking in regards to failures while reducing job execution overhead and providing a more natural php execution environment(vs fork). The Resque_Worker::fork method was adjusted to always return false when running the fastcgi tests.
This is just a POC, not intended for usage outside benchmarking but i think it shows promise.
https://gist.github.com/4253706
Initial benchmarks on an intel i5-2500k (4 cores no HT), ubuntu 12.04, php 5.4.8
WITH FORK:
COUNT=1 INTERVAL=0.01 QUEUE=default APP_INCLUDE=examples/worker_include.php php vendor/.../resque.php 1 Worker, no arguments Completed 2000 jobs in 9.704s ~4.8ms per job per core Completed 2000 jobs in 9.960s Completed 2000 jobs in 10.014s Completed 2000 jobs in 10.022s Completed 2000 jobs in 10.000s Completed 2000 jobs in 9.992s Completed 2000 jobs in 10.310s Completed 2000 jobs in 9.991s Completed 2000 jobs in 10.002s Completed 2000 jobs in 10.016s
COUNT=4 INTERVAL=0.01 QUEUE=default APP_INCLUDE=examples/worker_include.php php vendor/.../resque.php 4 Workers, no arguments, ~6.4ms per job per core Completed 2000 jobs in 3.235s Completed 2000 jobs in 3.250s Completed 2000 jobs in 3.252s Completed 2000 jobs in 3.247s Completed 2000 jobs in 3.278s Completed 2000 jobs in 3.282s Completed 2000 jobs in 3.298s Completed 2000 jobs in 3.292s Completed 2000 jobs in 3.297s Completed 2000 jobs in 3.304s
WITH FASTCGI:
COUNT=1 INTERVAL=0.01 QUEUE=default APP_INCLUDE=examples/worker_fcgi_include.php php vendor/.../resque.php 1 worker, no arguments Completed 2000 jobs in 3.082s ~1.5ms per job per core Completed 2000 jobs in 3.098s Completed 2000 jobs in 3.102s Completed 2000 jobs in 3.084s Completed 2000 jobs in 3.093s Completed 2000 jobs in 3.081s Completed 2000 jobs in 3.111s Completed 2000 jobs in 3.098s Completed 2000 jobs in 3.101s Completed 2000 jobs in 3.079s
COUNT=4 INTERVAL=0.01 QUEUE=default APP_INCLUDE=examples/worker_fcgi_include.php php vendor/.../resque.php 4 workers, no arguments Completed 2000 jobs in 0.982s ~1.9ms per job per core Completed 2000 jobs in 0.981s Completed 2000 jobs in 0.988s Completed 2000 jobs in 0.988s Completed 2000 jobs in 0.989s Completed 2000 jobs in 0.995s Completed 2000 jobs in 0.987s Completed 2000 jobs in 0.996s Completed 2000 jobs in 0.983s Completed 2000 jobs in 0.986s
To summarize:
FORK one core: 4.8ms/job/core four cores: 6.4ms/job/core FASTCGI one core: 1.5ms/job/core four cores: 1.9ms/job/core
Naturally this is a synthetic benchmark with the actual job doing nothing. It is intended only to measure the amount of overhead involved in starting resque jobs. Additionally this was run over the loopback interface, as the provided FCGIClient class does not support unix sockets. It is unknown at this point how much overhead there is between tcp and unix sockets for our usage.
Thoughts? Would there be any interest in flushing out to a proper implementation?
EDIT: fixed bad math, 4 * 0.982/2000 makes for 1.9ms/job/core on 4 core fastcgi
Ok, this is seriously cool. :sunglasses:
I'd definitely be interested in having this built in, and it might help others such as @salimane who have been trying to work with the overhead of a forking model. It will also improve the error handling/catching for failed jobs as you'll be able to get the actual error message for a job that fatal errors out and would normally leave you with a "DirtyExit" exception.
It'd be super cool to get this working via sockets at some stage as well.
I can sort of see there being different strategies for job execution - Resque_JobStrategy_Fork, Resque_JobStrategy_Fastcgi, and you can configure one at runtime. Thoughts?
hi @ebernhardson @chrisboulton this is interesting. one quick problem I could see is long running jobs over fastcgi. May be something could be done to adjust for this. But as long as this is space and resource efficient than fork, I'm all for it. Thanks
Job strategies sound like a good approach to this. I took a stab at refactoring the current code into a strategy pattern, let me know what you think. I'm not sure that perform is the right name for the method on Resque_JobStrategy_Interface, suggestions welcome.
I also have some worry about Resque_Worker::killChild, in that the InProcess strategy has no way to kill the job(when triggered from SIGUSR1), and closing the fpm socket in the Fastcgi strategy wont necessarily stop fpm from finishing the job either. Perhaps a few notes in the documentation is all that is required though.
ebernhardson/php-resque@36f53532130e2864dc9ad84067d4f62634ccf4c2
@salimane Long running tasks are certainly a valid concern. We can always set max_execution_time to 0 from the worker code so it matches the CLI environment, but running some specific testing will be worth the time invested. The time and space efficiency really need to be measured to have any certainty. My intuition tells me it wont be as clear cut as the initial benchmarks show, and will depend on the use case.
@ebernhardson
I quickly went through the changes at ebernhardson/php-resque@36f5353 . If I understand this correctly, this proposal comes down to instead of running the jobs within a child of the worker process, run the jobs directly in the worker process. is this so ?
@salimane The changes in the fork dont represent fcgi yet. The first step here was just to see if @chrisboulton is ok with the direction of the refactor before i start plugging in and writing tests for fastcgi. The linked refactor passes all the original tests and should have exactly the same functionality as the current php-resque code. With the fastcgi in all jobs will be executed by the php-fpm manager, similar to how nginx would run run webpages through php-fpm. Connecting and running scripts through fpm is demonstrated from the gist in the first post.
So... How's this coming along? I see PR #81, but it doesn't have any recent updates, either...
