jupyter-scheduler icon indicating copy to clipboard operation
jupyter-scheduler copied to clipboard

Clarify input file immutability in the docs, explain what happens with a notebook during job or job definition lifecycle

Open andrii-i opened this issue 1 year ago • 0 comments

Problem

As https://github.com/jupyter-server/jupyter-scheduler/issues/504#issuecomment-2043542988 points out, docs do not clearly explain input file immutability.

Moreover, sentence When the scheduler runs your notebook, it makes a copy of the input file (https://jupyter-scheduler.readthedocs.io/en/latest/users/index.html#use) could be interpreted in a way that implies that every scheduled execution makes a copy of the input file notebook including mutations.

Proposed Solution

Clarify input file immutability in the docs, explain what happens with a notebook during job or job definition lifecycle.

Inputs are immutable: every time you create a schedule for an input file (job definition), jupyter scheduler server creates a snapshot of your input file and then runs it / creates jobs from the snapshot accordingly to the schedule. Therefore mutation of the original input notebook would not propagate, new job / job definition needs to be created based on the mutated notebook. https://github.com/jupyter-server/jupyter-scheduler/issues/504#issuecomment-2043297861

andrii-i avatar Apr 09 '24 01:04 andrii-i