changes in parameters

[RossBoylan]

As the application evolves the parameters change, in 2 ways:

1. The set of parameter names changes, mostly by growing.
2. The best estimated values for the parameters change based on evolving research.

This is a challenge for those who have started working with earlier versions of the code and parameter files. We took an earlier version of the file and then customized it for the local simulations we are doing. So we can't just copy the new version over the old version.

How can these changes be documented and communicated? Recent changes, related to #98, have sought to keep the documentation current with the code (apparently still some rough spots #130). But that doesn't directly address the issue of how someone using an old version of the code can discover what has changed.

Second, once the changes are communicated, is there a way that someone can easily update their existing parameter files?

I started work on a program to assist with the upgrade process. It is still very preliminary, but I thought it might be useful to share it: https://github.com/RossBoylan/OpenABM-Covid19/blob/upgrade/python/upgrade.py. In particular, create_output_file_dictionaries.py in the same directory is treading on similar ground and even reads one of the same files, tests/data/baseline_parameters.csv (as the master file). However, create_output_file_dictionaries.py doesn't do anything with that info, that I can see, suggesting it is in progress too.

At the moment it simply detects if anything seems to have changed:

(jarenv) ross@barley:/usr/local/jar/OpenABM-Covid19/python$ ./upgrade.py ../../ross/baseline_parameters_sf.csv 
baseline_parameters_sf.csv is not consistent with current parameters.

	Only in the new scheme: {'manual_trace_notifications_per_worker_day', 'test_order_wait_priority', 'manual_trace_delay', 'manual_trace_n_workers', 'test_result_wait_priority', 'manual_traceable_fraction_occupation', 'priority_test_contacts_60_69', 'priority_test_contacts_80', 'priority_test_contacts_10_19', 'priority_test_contacts_70_79', 'priority_test_contacts_50_59', 'test_sensitive_period', 'test_sensitivity', 'manual_trace_on_positive', 'manual_trace_on_hospitalization', 'priority_test_contacts_20_29', 'manual_trace_time_on', 'priority_test_contacts_40_49', 'manual_trace_on', 'manual_traceable_fraction_household', 'priority_test_contacts_30_39', 'priority_test_contacts_0_9', 'manual_trace_exclude_app_users', 'manual_traceable_fraction_random', 'test_specificity', 'test_release_on_negative', 'manual_trace_interviews_per_worker_day'}
	Only in the old scheme: {'quarantine_household_contacts_on_traced_positive', 'quarantine_household_contacts_on_traced_symptoms'}

I suspect the "only in the old scheme" parameters are local creations.

The help indicates what I hope it will eventually do.

$ ./upgrade.py --help
usage: upgrade.py [-h] [--valcheck] [--update] [--override OVERRIDE [OVERRIDE ...]] [--rename RENAME [RENAME ...]]
                  [--newbaseline NEWBASELINE]
                  files [files ...]

Scans old parameter files and checks for compatiblity with the current OpenABM-Covid19 requirements. Optionally,
update those files to match the current requirements.

positional arguments:
  files                 old files with parameters (csv). Shell globbing permitted.

optional arguments:
  -h, --help            show this help message and exit
  --valcheck            check that parameter values, as well as names, match
  --update              Change the files to match new scheme
  --override OVERRIDE [OVERRIDE ...]
                        parameters for which you will specify the value. Python regular expressions allowed
  --rename RENAME [RENAME ...]
                        names of variables in new scheme that you will create by renaming old ones. Regex permitted.
  --newbaseline NEWBASELINE
                        location of file with the new parameter schema

--rename or --override, which may be combined, imply --update. If you request update of foo.csv it will be left as
foo.bak when you are done. Specify file arguments before all options, or use -- to separate files from options. Note
that override and rename take arguments separated by spaces, not commas.