Superfluous COEFFICIENTS and SPEC model settings using SPEC_SEGMENTS
While developing the Settings Checker, it became clear that spurious model configurations (which are happily ignored by some of the actual processing steps) are allowed in the model settings YAML file.
This is mostly a problem for the settings checker itself -- it isn't always clear what settings are permitted, required, or conditionally required. But will also be a problem for users who want to write settings files that pass muster.
In particular, the MWCOG model tests were failing because the non_mandatory_tour_scheduling settings contained the following top-level values:
SPEC: tour_scheduling_nonmandatory.csv
COEFFICIENTS: tour_scheduling_nonmandatory_coeffs.csv
...
The Settings Checker (correctly) threw an error because these files do not exist. But they aren't supposed to exist, as the actual spec and coefficients data used by the model are separated by purpose, each with its own set of CSV files:
SPEC_SEGMENTS:
escort:
'SPEC': tour_scheduling_nonmandatory_escort.csv
'COEFFICIENTS': tour_scheduling_nonmandatory_escort_coefficients.csv
shopping:
'SPEC': tour_scheduling_nonmandatory_shopping.csv
'COEFFICIENTS': tour_scheduling_nonmandatory_shopping_coefficients.csv
...
# etc
For now, the top-level settings have been commented out with a note in the relevant files in the prototype_mwcog test model.
I'd suggest that all the Pydantic models be revisited, since that's really the appropriate place for validating the type and structure of these data.
Would there be an instance where both SPEC and SPEC_SEGMENTS[segment]['SPEC'] would have valid files?