resque-pool
resque-pool copied to clipboard
resque
monitor child memory usage
monit style limits, e.g. 2 cycles at 500MB and QUIT, 6 cycles at 600MB and TERM, 8 cycles at 600MB and KILL.
must be settable in the config file (and the WebUI). must be settable per worker type, as well as for default.
will need to extend config file format to support this.
I have a hard coded version on the backupify fork. I'll clean it up. We may be able to get away with not changing the config file by making them command line vars.
I'd prefer to make it configurable per worker-type, and that would get messy if we used environment variables. We could certainly implement the feature in smaller pieces (e.g. stage 1: default for all workers in the pool using env vars; stage 2: configurable per worker type from config file; stage 3: Web UI, etc). But I'd prefer to leave it as a separate branch from the main release until we're at least to stage 2.
Until it's complete, someone who only needs the partial functionality can easily enough install the branched version from their gem file via :git. What do you think?
that's reasonable. probably never get to stage 3.
You could also merge in stage 1 and continue to support an env var through stage 2. It's additive functionality and then you don't have people baking branch names into their bundler Gemfile
Oh, we'll get to stage 3 all right. I'm tired of the tweak yml and /etc/init.d/app_resque reload cycle. :)
On second thought, I think I will merge your "stage one" changes into master before I get around to stage two (which would be this issue).
I merged your branch into my memory_usage branch, reorganised the code a bit, and (most importantly) changed the configuration. See https://github.com/nevans/resque-pool/blob/memory_usage/ExperimentalFeatures.md for info on how to configure it now.
However, I have not done any significant testing on the memory management code nor the orphan offset code. I've simply confirmed that they don't mess with anything else unless they are configured in.
I'm fairly certain there are some bugs in the orphan watcher code, but I might just be misunderstanding what you are trying to do. :-)
Again, I haven't really tested this much yet, but I've just pushed 0.3.0.beta.1 which includes your memory management code (and my changes for configuration). It might be a few weeks before I get around to properly testing this out personally, so any feedback is welcome.