Test contribution/robustness detector

[ChatBear]

Description

This PR aim to add a new detector for testing the robustness when a single categorial values. I tried to add a detector and a new tranformation function which shuffle all categorial values and add in issue if the accuracy of the model decreased too much.

I created a new base categorial detector which basically has the same methods than the basedetectortext, only the method "run" change, we only take "category" feature insteand of text.

Furthermore, i tried to check info around the taxonomy, do you have a link or something in order to get more info around that, for now the taxonomy is None on the categorial detector class.

Related Issue

Scan: Add a robustness detector to the scan that perturbs categorial values https://github.com/Giskard-AI/giskard/issues/1847

https://github.com/Giskard-AI/giskard/issues/1847

Type of Change

• [ ] 📚 Examples / docs / tutorials / dependencies update
• [ ] 🔧 Bug fix (non-breaking change which fixes an issue)
• [ ] 🥂 Improvement (non-breaking change which improves an existing feature)
• [x] 🚀 New feature (non-breaking change which adds functionality)
• [ ] 💥 Breaking change (fix or feature that would cause existing functionality to change)
• [ ] 🔐 Security fix

Checklist

• [x] I've read the CODE_OF_CONDUCT.md document.
• [x] I've read the CONTRIBUTING.md guide.
• [ ] I've written tests for all new methods and classes that I created.
• [x] I've written the docstring in Google format for all the methods and classes that I used.
• [ ] I've updated the pdm.lock running pdm update-lock (only applicable when pyproject.toml has been modified)

[ChatBear]

Hello, It is not merge ready yet, i didn't do the testing part because i need to check the doc to do the testing. But i have a few questions, i did create another detector for categories values, which it is basically the same one than base detector text. Only the method run : the category detector only take feature type : "category". So there is a lot boilerplate code, so i am not sure that my solution is good.

Also, i didn't find any info related to the taxonomy, can you provider link for that ? I don't know which taxonomy is appropriate for categorial perubation.

Last thing but not least, if you can check the code if it respect your standard, otherwise i would gladly change it.

Thanks

[mattbit]

Hi @ChatBear !

Also, i didn't find any info related to the taxonomy, can you provider link for that ? I don't know which taxonomy is appropriate for categorial perubation.

Regarding the taxonomy, we use the AVID standard. I think the correct code for this detector would be:

_taxonomy = ["avid-effect:performance:P0201"]

P0201 is correspond to this taxonomy item:

P0201: Resilience/stability Ability for outputs to not be affected by small change in inputs.

You can find the taxonomy documentation here: https://docs.avidml.org/taxonomy/effect-sep-view/performance

[ChatBear]

Thanks i'll add the taxonomy then

[ChatBear]

I took some time off, that's why there is not update of the PR, I'll be able to continue the PR in middle of May !

[alexcombessie]

Hey @ChatBear - Is this PR ready for review?

[ChatBear]

Hi sorry, I didn't do the testing part, I don't have the time to finish it.

Hope someone will finish the issue :(