epic icon indicating copy to clipboard operation
epic copied to clipboard

automated warnings about certain parameter changes

Open c-dilks opened this issue 3 years ago • 1 comments

Is your feature request related to a problem? Please describe. It would be useful to be notified if certain subsystem parameters are changed by a pull request. Some changes in one subsystem can have unintended side effects in others, and these side effects can easily slip under the radar.

Describe the solution you'd like A quick way is to diff the constants.out artifact between the current PR and the one from the last push to main. That's probably too much information though, and it would be better to have a much smaller list of specific parameters that we want to watch for changes on.

The idea is to first produce a "lock" file adding certain parameters and their required/preferred values; likely this would be similar to the Menagerie. The parameters in this lock file are the ones we want to keep fixed, or at least be notified when they are changed. For example, the dRICH envelope:

DRICH_rmax2 180.0
DRICH_zmin  195.0
DRICH_zmax  315.0

We should aim to keep this list as minimal as possible.

Then add a CI job that would check if any of the parameters in this lock file differ from the implemented parameter values in the associated pull request. If there are changes, warn the user and fail the job.

Changes to the lock file should be allowed by anyone, with the approval of relevant subsystems (liaisons?). Changes shouldn't be restricted by any automated link to the menagerie, but it would be good to compare the lock file to the menagerie regularly.

Describe alternatives you've considered Hard code such a check in a src/*.cpp file, and printout a warning.

c-dilks avatar Aug 15 '22 22:08 c-dilks

Related to the menagerie, we already write their spreadsheet based on our parametrization, e.g. https://github.com/eic/epic/suites/7821896684/artifacts/330524938 (DetectorParameterTable.csv). A csv-diff with main would indeed be useful. I think that's not so easy with the GitHub API as with the GitLab API.

wdconinc avatar Aug 15 '22 22:08 wdconinc