alphapept icon indicating copy to clipboard operation
alphapept copied to clipboard

Systematic testing of settings and combinations

Open JuliaS92 opened this issue 3 years ago • 1 comments

Is your feature request related to a problem? Please describe. Currently it is rather complicated to trace which settings are used where and which combinations cause which behaviour.

Describe the solution you'd like There should be a workflow, that tests two things:

  1. If you change a setting, it is actually passed on to the executing functions and changes the result. This can potentially be automated by changing each setting to it's min and max value based on the default settings. Both have to be different from the default result and different from one another.
  2. Which workflows are valid and and which ones crash. For the major workflow steps all combinations of selecting or unselecting them should be iterated autoamtically and they should either raise appropriate errors or run through smoothly. All of this could be done on a very minimal dataset (e.g. two short gradient files).

JuliaS92 avatar Jul 21 '21 08:07 JuliaS92

That's an excellent suggestion. We had started with the ParameterScreen at some point: https://github.com/MannLabs/alphapept/blob/master/sandbox/ParameterScreen.ipynb

The idea at that time was to screen through all parameters for some test sets and define sane min max values - and also find out what makes things slow w/o having a performance gain.

straussmaximilian avatar Jul 21 '21 17:07 straussmaximilian