giskard icon indicating copy to clipboard operation
giskard copied to clipboard

Test contribution/robustness detector

Open ChatBear opened this issue 2 months ago • 4 comments

Description

This PR aim to add a new detector for testing the robustness when a single categorial values. I tried to add a detector and a new tranformation function which shuffle all categorial values and add in issue if the accuracy of the model decreased too much.

I created a new base categorial detector which basically has the same methods than the basedetectortext, only the method "run" change, we only take "category" feature insteand of text.

Furthermore, i tried to check info around the taxonomy, do you have a link or something in order to get more info around that, for now the taxonomy is None on the categorial detector class.

Related Issue

Scan: Add a robustness detector to the scan that perturbs categorial values https://github.com/Giskard-AI/giskard/issues/1847

https://github.com/Giskard-AI/giskard/issues/1847

Type of Change

  • [ ] 📚 Examples / docs / tutorials / dependencies update
  • [ ] 🔧 Bug fix (non-breaking change which fixes an issue)
  • [ ] 🥂 Improvement (non-breaking change which improves an existing feature)
  • [x] 🚀 New feature (non-breaking change which adds functionality)
  • [ ] 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] 🔐 Security fix

Checklist

  • [x] I've read the CODE_OF_CONDUCT.md document.
  • [x] I've read the CONTRIBUTING.md guide.
  • [ ] I've written tests for all new methods and classes that I created.
  • [x] I've written the docstring in Google format for all the methods and classes that I used.
  • [ ] I've updated the pdm.lock running pdm update-lock (only applicable when pyproject.toml has been modified)

ChatBear avatar Apr 21 '24 21:04 ChatBear