Quantus icon indicating copy to clipboard operation
Quantus copied to clipboard

New metric: implement Relative Stability

Open annahedstroem opened this issue 3 years ago • 1 comments

Implement Relative Stability (belongs to the Robustness category).

Relative Stability: leverages model information to evaluate the stability of an explanation with respect to the change in the a) input data, b) intermediate representations and c) output logits of the underlying prediction model.

Paper: https://arxiv.org/pdf/2203.06877.pdf

To clarify changes excepted in the PR, the should be reflected in the following folders:

  • [ ] quantus/metrics/faithfulness_metrics.py and quantus/metrics/robustness_metrics.py
  • [ ] tests/metrics/test_faithfulness_metrics.py and quantus/metrics/test_robustness_metrics.py
  • [ ] tests/metrics/test_faithfulness_metrics.py and quantus/metrics/test_robustness_metrics.py and in the tutorial: https://github.com/understandable-machine-intelligence-lab/Quantus/blob/main/tutorials/tutorial_basic_example_all_metrics.ipynb
  • [ ] `tests/helpers/constants.py``
  • [ ] README.md

annahedstroem avatar Apr 19 '22 10:04 annahedstroem

Hi @annahedstroem, I'd like to work on this feature

aaarrti avatar Jul 08 '22 17:07 aaarrti