jupyter-scheduler icon indicating copy to clipboard operation
jupyter-scheduler copied to clipboard

Add cell execution tracking during notebook execution

Open agupta01 opened this issue 4 months ago • 0 comments

References

Resolves #586

Code changes

This PR adds a completed_cells field to the Job model to track the number of cells executed during notebook execution in real-time. The implementation includes:

Model Changes:

  • Added nullable completed_cells column to the Job table in ORM (jupyter_scheduler/orm.py)
  • Added completed_cells field to DescribeJob and UpdateJob models (jupyter_scheduler/models.py)

Execution Tracking:

  • Added JobFeature.track_cell_execution feature flag to enable/disable cell tracking
  • Modified DefaultExecutionManager to use nbconvert's native on_cell_executed hook. Note that this was found to be cleaner than the subclassing approach defined in #586.
  • Hook updates the database with ep.code_cells_executed after each cell execution

User-facing changes

  • GET /jobs/{job_id} now returns the current count of completed cells
  • PATCH /jobs/{job_id} accepts completed_cells updates for manual corrections

Backwards-incompatible changes

None

agupta01 avatar Jul 23 '25 01:07 agupta01