delayed_job icon indicating copy to clipboard operation
delayed_job copied to clipboard

[WIP] Copy-on-Write forking (logic copied from Puma)

Open johnnyshields opened this issue 4 years ago • 0 comments

This PR further improves #1138 by adding copy-on-write forking which forks from the first worker child to the other children. After a configurable timeout (e.g. 1 hour), it will re-fork its children from the first child.

The reason is to reduce memory usage and our AWS bill at TableCheck :)

Much of the code is copied from Puma 5, with proper attribution. (Puma is MIT licensed.)

TODOS:

  • [ ] Fix phased restart (seems to be doing multiple stop/starts of worker1)
  • [ ] Fix TTIN/TTOU --> not being detected?
  • [ ] Determine why code is hot-reloading. Is this just in development mode?
  • [ ] Fix process names (showing as "ruby")
  • [ ] Write pidfile
  • [ ] Test single mode + fix bugs
  • [ ] Test cluster mode + fix bugs
  • [ ] "pooled" mode not yet supported. I am thinking to do N-reforks round robin, where the base worker of each pool is forked from once. This will ensure that all code loaded is compacted.
  • [ ] "pooled" mode TTIN and TTOU signals for worker scaling
  • [ ] Hooks for re-fork, shutdown, etc
  • [ ] Support the restart, stop, etc. script commands.
  • [ ] Cleanup logger (remove Loggable mixin)

johnnyshields avatar Jan 09 '22 06:01 johnnyshields