Chunk log assert causes model run failures if trip P&D moves past chunk cache iterations

[AndrewTheTM]

trafficstars

In the case that chunk training does not iterate far enough in trip purpose and destination, an assert statement causes a model run failure if the model run extends past the chunk training and the model is in production mode. Is there a way we can have ActivitySim not throw this (non) error and either use the last iteration's chunk settings (or not store chunk settings by iteration for this model)?

Log file:

27/09/2022 10:08:50 - INFO - activitysim.abm.models.trip_purpose - assign purpose to 1 last inbound trips
27/09/2022 10:08:50 - INFO - activitysim.abm.models.trip_purpose - assign purpose to 1 intermediate trips
27/09/2022 10:08:50 - DEBUG - activitysim.core.expressions - trip_purpose_and_destination.i4.purpose.trip_purpose_annotate_trips_preprocessor compute_columns using expression spec file trip_purpose_annotate_trips_preprocessor.csv
27/09/2022 10:08:50 - INFO - activitysim.core.chunk - trip_purpose Running adaptive_chunked_choosers with 1 choosers
27/09/2022 10:08:51 - WARNING - activitysim.core.mp_tasks - AssertionError exception running trip_purpose_and_destination model: 
27/09/2022 10:08:52 - ERROR - activitysim.core.mp_tasks - mp_tasks - mp_households_9 - AssertionError exception caught in mp_run_simulation: 
27/09/2022 10:08:52 - ERROR - activitysim.core.mp_tasks - 
---
Traceback (most recent call last):
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\core\mp_tasks.py", line 915, in mp_run_simulation
    run_simulation(queue, step_info, resume_after, shared_data_buffer)
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\core\mp_tasks.py", line 867, in run_simulation
    raise e
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\core\mp_tasks.py", line 864, in run_simulation
    pipeline.run_model(model)
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\core\pipeline.py", line 502, in run_model
    orca.run([step_name])
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\orca\orca.py", line 2168, in run
    step()
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\orca\orca.py", line 973, in __call__
    return self._func(**kwargs)
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\abm\models\trip_purpose_and_destination.py", line 132, in trip_purpose_and_destination
    trips_df, save_sample_df = run_trip_purpose_and_destination(
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\abm\models\trip_purpose_and_destination.py", line 27, in run_trip_purpose_and_destination
    choices = run_trip_purpose(
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\abm\models\trip_purpose.py", line 233, in run_trip_purpose
    for i, trips_chunk, chunk_trace_label in chunk.adaptive_chunked_choosers(
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\core\chunk.py", line 1088, in adaptive_chunked_choosers
    rows_per_chunk, estimated_number_of_chunks = chunk_sizer.initial_rows_per_chunk()
  File "C:\Users\andrew.rohne\.conda\envs\gen3_model\lib\site-packages\activitysim\core\chunk.py", line 849, in initial_rows_per_chunk
    assert chunk_training_mode() != MODE_PRODUCTION
AssertionError
---

Original chunk cache (relevant lines):

trip_purpose_and_destination.i1.purpose,25,557056,442368,9356,17695,17695,375
trip_purpose_and_destination.i2.purpose,6,77824,77824,2128,12971,12971,355
trip_purpose_and_destination.i3.purpose,3,53248,53248,960,17750,17750,320

I'm testing a temporary fix where I copied the i3 line and pasted it twice as i4 and i5, but it's not a great fix.