ActivitySim
Estimation mode inconsistencies between estimated and revised coefficients
Describe the bug If you change a parameter in the SPEC file, LARCH will correctly estimate the new coefficients. However, when you save the estimated parameters, the revised coefficients file will only show coefficients that were in the original coefficients file.
- If you remove a variable in the estimation that was in the original coefficient file, it will reappear in the revised coefficients file with its original parameter value, along with updated values for all the other parameters that were estimated.
- If you add a new variable in the estimation that was not in the original coefficient file, then it will not appear in the revised coefficients file.
To Reproduce For example, in the Auto Ownership model if you follow the notebook example and change the segmentation by HH income, the original coefficients (cars2 as example) are:
coef_cars2_hh_income_0_30k coef_cars2_hh_income_30_up
The newly estimate coefficients are:
coef_cars2_hh_income_0_25k coef_cars2_hh_income_25_50 coef_cars2_hh_income_50_75 coef_cars2_hh_income_75_up
The notebook correctly updates the SPEC file and estimates new parameters. If you then save the new parameters (not in the current notebook example, so I added), the new parameters do not appear in the revised coefficients file. The old income parameters persist with their old values. All of the other parameters are updated based on the new estimation.
I discovered this when reviewing PR 917 comparing estimated coefficients to the revised outputs. For Auto Ownership I found that SPEC. coef_cars2_density_10_up_workers was not in the SPEC but was in the original coefficients file. In the SPEC it should have been applied but, probably due to a manual entry error, coef_cars2_density_10_up_no_workers was also applied to workers. coef_cars2_density_10_up_workers appeared in the revised coefficients with its original input value, even though it was not part of the estimation in LARCH or the SPEC. This means that the coef_cars2_density_10_up_workers was not re-estimated, but the unwitting user might assume it had been. They may also not know that the wrong coefficient is being applied to the worker market segment.
Expected behavior The workaround is to modify the input coefficients file by deleting coefficients you no longer want and adding placeholders for new coefficients you do want, This must be done in addition to modifying the SPEC file. However, there does not appear to be user instructions for this. Moreover, it would be strongly preferred to modify just one file, SPEC since that is where variables are defined, and have the revised coefficient file updated based on the coefficients in SPEC.
In addition, it would be a good idea to identify instances where there is a difference in coefficient names between the input coefficients file and the SPEC file and warn the user. This would help trap unintentional changes like the one described above.
Additional context I checked this on a handful of other model components, and it is an issue for all of the regular utility expressions. For location choice models, it only affects the regular utilities, not the size terms.
