Quantus
Quantus copied to clipboard
Enhancement: out metrics' **kwargs in *args and set default values
To make the parametrisation of metrics in Quantus more explicit (in order to mitigate the risks of user typos and other undefined behaviours e.g., when passing a larger kwargs for different perturb_func calls if large-scale experiments are performed), we want to update the arguments list of all metrics initialisations like def __init__(self, ...) in the library as follows:
- Split kwargs for every Callable used in the metric init like
explain_func,perturb_funcandsimilar_funcas follows:explain_func_kwargs,perturb_kwargsandsimilar_func_kwargs - Make sure to set default values to None where types are immutable in arguments list and reset it after to its actual data type like
perturb_kwargs: Union[None, dict] = None - Remove
*argssince it is useless
AND:
- Update the docstrings to reflect these changes
- Update code in the library where the metrics are called e.g., in the tutorials and pytests
Merging with another issue: https://github.com/understandable-machine-intelligence-lab/Quantus/issues/78.
Quantus is designed to be easy to use: users are allowed to initialise the metrics without explicitly setting any metrics arguments (there are default params for everything) - which means that metrics can be easily looped over etc, making the library very flexible.
But since "the devil is in the details" when it comes to metric parameterisation - we should create a feature that highlights the current parameter setting of each metric - so to identify potential typos etc.
Could be a simple feature that prints the values of metrics' argument at metric initialisation, suggestively:
- create a "disable_parameter_printing" function that checks for deviation of default values
Completed in #124