cylc-flow icon indicating copy to clipboard operation
cylc-flow copied to clipboard

Restart fails after removing a task from flow.cylc

Open hjoliver opened this issue 2 years ago • 0 comments

Ref: https://cylc.discourse.group/t/task-wrangling-issues/517/9

Datastore issue.

[scheduling]
   [[graph]]
      R1 = a & b
[runtime]
   [[a, b]]
      script = false
  • install, play, stop or kill once stalled
  • remove b from the graph, reinstall, and play
2022-08-12T17:05:12+12:00 INFO - LOADING task proxies
2022-08-12T17:05:12+12:00 INFO - + 1/a failed
2022-08-12T17:05:12+12:00 INFO - [1/a failed(runahead) job:01 flows:1] => failed
2022-08-12T17:05:12+12:00 INFO - Runahead limit: 1
2022-08-12T17:05:12+12:00 INFO - + 1/b failed
2022-08-12T17:05:12+12:00 ERROR - '~oliverh/bug/run1//$namespace|b'
    Traceback (most recent call last):
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/data_store_mgr.py", line 824, in generate_ghost_task
        task_def = self.data[self.workflow_id][TASKS][t_id]
    KeyError: '~oliverh/bug/run1//$namespace|b'
    During handling of the above exception, another exception occurred:
    Traceback (most recent call last):
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/scheduler.py", line 645, in start
        await self.configure()
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/scheduler.py", line 464, in configure
        self._load_pool_from_db()
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/scheduler.py", line 695, in _load_pool_from_db
        self.workflow_db_mgr.pri_dao.select_task_pool_for_restart(
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/rundb.py", line 859, in select_task_pool_for_restart
        callback(row_idx, list(row))
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/task_pool.py", line 496, in load_db_task_pool_for_restart
        self.add_to_pool(itask, is_new=False)
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/task_pool.py", line 211, in add_to_pool
        self.create_data_store_elements(itask)
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/task_pool.py", line 237, in create_data_store_elements
        self.data_store_mgr.increment_graph_window(itask)
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/data_store_mgr.py", line 654, in increment_graph_window
        is_orphan = self.generate_ghost_task(s_tokens.id, itask, is_parent)
      File "/home/oliverh/cylc/cylc-flow/cylc/flow/data_store_mgr.py", line 826, in generate_ghost_task
        task_def = self.added[TASKS][t_id]
    KeyError: '~oliverh/bug/run1//$namespace|b'
2022-08-12T17:05:12+12:00 CRITICAL - Workflow shutting down - '~oliverh/bug/run1//$namespace|b'

hjoliver avatar Aug 12 '22 05:08 hjoliver