activitysim icon indicating copy to clipboard operation
activitysim copied to clipboard

Setting "resume_after" with a completed run results in "write_data_dictionary" step failing with multiprocessing

Open Syonv opened this issue 2 months ago • 0 comments

Describe the bug When running a partial model run using resume_after with multiprocessing enabled, and reusing the pipeline files of a completed model run, the write_data_dictionary step fails with the following error: RuntimeError: checkpoint 'write_data_dictionary' not in checkpoints. See traceback for more details: mp_summarize-activitysim.log. This error seems to stem from the fact that the checkpoints table (checkpoints.parquet in pipeline.parquetpipeline) is not overwritten from the resume_after model onwards and still holds references to checkpoints from beyond the mp_households step.

To Reproduce Steps to reproduce the behavior:

  1. Run the prototype_mtc model with multiprocessing
  2. Set resume_after to some model
  3. Re-run model in the same output directory as in step 1
  4. See error

Expected behavior The write_data_dictionary step should not load tables tagged with itself or beyond.

Additional context This was tested with version 1.3.4 and the MWCOG model, as well as version 1.5.0 and prototype_mtc.

Syonv avatar Oct 03 '25 17:10 Syonv