Leaner and Improved ActivitySim Pipeline

[joecastiglione]

trafficstars

There seems to be room for improvement and use of the ActivitySim pipeline. The primary goal of this task would be to substantially reduce the disk space taken with an ActivitySim run, via reducing the size of the pipeline. One key suggested way to get at this would be to reduce/remove duplicated fields and identical information stored throughout the pipeline. A secondary aspect to this task would be to simplify the design and understanding of the pipeline, to improve the logic for how unique information is stored and to make accessing information from the pipeline easy and intuitive.

Additional Description: Partially complete since you can now optionally not save intermediate results to the pipeline for each submodel step. Additional pipeline oriented improvements can be incorporated into the reporting task.

[joecastiglione]

Agency Comments:

SANDAG: agreed. Maybe we can think about a data model with storage, effective querying, visualization, reporting and QAQC in mind (see item 39).

[lmz]

It seems like this is related to https://github.com/ActivitySim/activitysim/issues/728