DiCE icon indicating copy to clipboard operation
DiCE copied to clipboard

Fix #182: Supporting private data for sklearn models

Open amit-sharma opened this issue 2 years ago • 2 comments

[READY FOR MERGE]

This PR adds support for private data interface for sklearn ML models. dice-kdtree is not supported since it requires access to the training data to output the counterfactuals.

amit-sharma avatar Jun 27 '22 09:06 amit-sharma

Unit Test Results

281 tests  +20   276 :heavy_check_mark: +20   11m 26s :stopwatch: +4s     1 suites ±  0       5 :zzz: ±  0      1 files   ±  0       0 :x: ±  0 

Results for commit b3f7fe22. ± Comparison against base commit 16517511.

This pull request removes 20 and adds 40 tests. Note that renamed tests count towards both.
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_features_to_vary[PYT-1-None-2-features_to_vary0-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_features_to_vary[sklearn-1-None-2-features_to_vary0-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_invalid_query_instance[PYT-features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_invalid_query_instance[sklearn-features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_no_cfs[PYT-7-None-3-all-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_no_cfs[sklearn-7-None-3-all-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_permitted_range_categorical[PYT-1-None-2-permitted_range0]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_permitted_range_categorical[sklearn-1-None-2-permitted_range0]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_random_counterfactual_explanations_output[PYT-0-1]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_random_counterfactual_explanations_output[sklearn-0-1]
…
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_features_to_vary[random_binary_classification_exp_object0-1-None-2-features_to_vary0-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_features_to_vary[random_binary_classification_exp_object1-1-None-2-features_to_vary0-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_features_to_vary[random_binary_classification_exp_object2-1-None-2-features_to_vary0-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_features_to_vary[random_binary_classification_exp_object3-1-None-2-features_to_vary0-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_invalid_query_instance[random_binary_classification_exp_object0-features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_invalid_query_instance[random_binary_classification_exp_object1-features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_invalid_query_instance[random_binary_classification_exp_object2-features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_invalid_query_instance[random_binary_classification_exp_object3-features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_no_cfs[random_binary_classification_exp_object0-7-None-3-all-None]
tests.test_dice_interface.test_dice_random.TestDiceRandomBinaryClassificationMethods ‑ test_no_cfs[random_binary_classification_exp_object1-7-None-3-all-None]
…

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jul 11 '22 11:07 github-actions[bot]

Codecov Report

Merging #312 (343883c) into master (2eb8e54) will increase coverage by 0.72%. The diff coverage is 86.85%.

@@            Coverage Diff             @@
##           master     #312      +/-   ##
==========================================
+ Coverage   71.37%   72.09%   +0.72%     
==========================================
  Files          26       27       +1     
  Lines        3462     3602     +140     
==========================================
+ Hits         2471     2597     +126     
- Misses        991     1005      +14     
Flag Coverage Δ
unittests 72.09% <86.85%> (+0.72%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
dice_ml/diverse_counterfactuals.py 85.03% <ø> (+0.68%) :arrow_up:
dice_ml/explainer_interfaces/dice_tensorflow1.py 0.00% <0.00%> (ø)
dice_ml/utils/helpers.py 74.82% <50.00%> (-4.38%) :arrow_down:
dice_ml/explainer_interfaces/dice_genetic.py 95.16% <85.00%> (-0.81%) :arrow_down:
dice_ml/dice.py 80.95% <86.36%> (-1.55%) :arrow_down:
dice_ml/model_interfaces/pytorch_model.py 89.65% <89.65%> (+0.18%) :arrow_up:
dice_ml/data_interfaces/private_data_interface.py 83.14% <92.95%> (+2.85%) :arrow_up:
dice_ml/data_interfaces/public_data_interface.py 87.41% <94.11%> (-1.51%) :arrow_down:
dice_ml/data_interfaces/base_data_interface.py 93.33% <96.42%> (+5.09%) :arrow_up:
dice_ml/explainer_interfaces/explainer_base.py 89.69% <97.14%> (+0.86%) :arrow_up:
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2eb8e54...343883c. Read the comment docs.

codecov-commenter avatar Jul 11 '22 11:07 codecov-commenter