Distributing job code updates

[mattchan-tencent]

trafficstars

Hi,

What is the recommended way to distribute functions for jobs when updating the job?

I currently distribute the code via scp and have to restart the workers so they update the imports but this is inconvenient when there is another job in progress.

Is there a way to tie the data and the job code together, instead of distributing code out-of-band and then restarting the workers?

Thanks!

[selwin]

I currently distribute the code via scp and have to restart the workers so they update the imports but this is inconvenient when there is another job in progress.

This is what I currently do (but not through SCP, we update source codes on the servers via other means). When you shutdown a worker, if it's in the midst of a job, it will be given time to finish the job before shutting down so no jobs should be lost (provided that the job finishes within the grace peirod).