delayed_job
delayed_job copied to clipboard
collectiveidea
monitor children and re-fork when they exit
Adding to wkonkel's pull request https://github.com/collectiveidea/delayed_job/pull/565 - adding code to monitor children and re-fork them when they exit.
From experience trying to do this, I can say there are a bunch of edge cases this does not cover. Just use the DJ binary.
what is the dj binary/does it cover the issue in https://github.com/collectiveidea/delayed_job/pull/565 ?
bin/delayed_job or script/delayed_job for older versions of rails
Sent from my iPhone
On Sep 19, 2014, at 5:42 PM, David Gobaud [email protected] wrote:
what is the dj binary/does it cover the issue in #565 ?
— Reply to this email directly or view it on GitHub.
but those don't support NUM_PROCESSES for running multiple workers on a single dyno correct?
You are correct -n is not supported in such a way that heroku can support it.
The problem is that while your code will possibly work for your use case, it will not work in general. If we included this in the release, we would receive a bunch of issue reports of this not working in all situations.
In order to be included in the main release, the code needs to work for most cases, not just one.
Got it - I understand it doesn't make sense to include if it causes problems. What are the problems/bugs? I'd like to fix them if possible.
Lets clean this up and pull it in as a use at your own risk feature.
- [ ] Maintain worker name. Store the pids and detect which worker died.
waitwill return the pid and exit status of the dead process. - [ ] Add logging for worker died
- [ ] Cleanup forked workers on exit
- [ ] Move the
fork_delayeddefinition out into the namespace - [ ]
whiledoesn't need do
Coverage decreased (-1.31%) when pulling a6c79c7c5a2508e624bddfc39b26218fdbc1bbaa on dgobaud:master into 42b14892321e42a9c41725378b5f57172a9663b0 on collectiveidea:master.
Sounds good. I've made all the changes except "Cleanup forked workers on exit" - I'm not sure how best to do that.
+! here for a solution to this for heroku. I tried a workaround using a shell script, but for some reason logging gets lost. The logic is to start a bunch of daemons, then give the parent process something to do as in 'rake jobs:work' so you end up with some running as daemons and one running in the foreground to keep the dyno alive. However, the logging gets lost as STDOUT doesnt pass thru.
Here's my shell script (joshco_jobs)
#!/usr/bin/env bash
# dj spawner
bundle exec bin/delayed_job -n $DJ_DAEMONS start
bundle exec rake jobs:work
My procfile entry
jobber: bundle exec bin/joshco_jobs
