jupyter-scheduler
jupyter-scheduler copied to clipboard
Add cell execution tracking during notebook execution
References
Resolves #586
Code changes
This PR adds a completed_cells field to the Job model to track the number of cells executed during notebook execution in real-time. The implementation includes:
Model Changes:
- Added nullable
completed_cellscolumn to the Job table in ORM (jupyter_scheduler/orm.py) - Added completed_cells field to DescribeJob and UpdateJob models (
jupyter_scheduler/models.py)
Execution Tracking:
- Added
JobFeature.track_cell_executionfeature flag to enable/disable cell tracking - Modified
DefaultExecutionManagerto use nbconvert's nativeon_cell_executedhook. Note that this was found to be cleaner than the subclassing approach defined in #586. - Hook updates the database with ep.code_cells_executed after each cell execution
User-facing changes
- GET
/jobs/{job_id}now returns the current count of completed cells - PATCH
/jobs/{job_id}accepts completed_cells updates for manual corrections
Backwards-incompatible changes
None