jupyterlab-sos icon indicating copy to clipboard operation
jupyterlab-sos copied to clipboard

Use dedicated panel for task and workflow status

Open BoPeng opened this issue 6 years ago • 1 comments

This is very complicated and could only be implemented for jupyterlab.

Right now, when we submit tasks, we create task status tables in the cell output. This has a number of problems:

  1. The status table is "transient" in nature, and should perhaps not be displayed in the notebook.
  2. When we rerun the cell, perhaps with a different set of tasks, the running tasks becomes missing from the notebook, but are still running. In the worst case, the new tasks appears to be pending forever because the task engine is saturated with old tasks, that are not supposed to be running.

I believe that the tasks stuff should go to a dedicated panel, as in the case of dask. When tasks are submitted, they are displayed in the panel, and be controlled there. Re-running the current cell could submit new batches of tasks, but will not overwrite existing ones, which could be terminated from the panel.

BoPeng avatar Oct 04 '19 03:10 BoPeng

What makes more sense might be celery/flower rq/rq-dashboard like page, that is dedicated to status update.

BoPeng avatar Oct 04 '19 20:10 BoPeng