adversarial-robustness-toolbox icon indicating copy to clipboard operation
adversarial-robustness-toolbox copied to clipboard

Geometric Decision Based Attack (`GeoDA`) uncompatible with `ScikitlearnRandomForestClassifier` - Missing Error Handling

Open jetlime opened this issue 5 months ago • 1 comments

Describe the bug Whenever performing an Geometric Decision Based Attack evasion attack on a scikit-learn random forest classifier.

To Reproduce

Steps to reproduce the behavior:

  1. Define and fit a Random Forest Classifier using the sklearn library
model = RandomForestClassifier(verbose=0, n_estimators=1)
model.fit(X_train, y_train)
  1. Define a sklearn classifier object
classifier = SklearnClassifier(model=model)
  1. Generate adversarial samples
attack = GeoDA(classifier)
x_test_true_positives_adv = attack.generate(X_test_true_positives)
  1. See error:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[34], [line 15](notebook-cell:?execution_count=34&line=15)
     [10](notebook-cell:?execution_count=34&line=10) classifier = SklearnClassifier(model=model)
     [11](notebook-cell:?execution_count=34&line=11) X_test_true_positives, y_test_true_positives, prediction_classes, y_test, _, _, _ = scores[
     [12](notebook-cell:?execution_count=34&line=12)     fold
     [13](notebook-cell:?execution_count=34&line=13) ]
---> [15](notebook-cell:?execution_count=34&line=15) attack = GeoDA(classifier)
     [16](notebook-cell:?execution_count=34&line=16) x_test_true_positives_adv = attack.generate(X_test_true_positives)
     [18](notebook-cell:?execution_count=34&line=18) evasion_rate, adversarial_samples_amount = evasion_evaluation(
     [19](notebook-cell:?execution_count=34&line=19)     model, x_test_true_positives_adv, y_test_true_positives
     [20](notebook-cell:?execution_count=34&line=20) )

File ~venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:114, in GeoDA.__init__(self, estimator, batch_size, norm, sub_dim, max_iter, bin_search_tol, lambda_param, sigma, verbose)
    [111](venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:111) if self.estimator.input_shape is None:  # pragma: no cover
    [112](venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:112)     raise ValueError("The `input_shape` of the is required but None.")
    [113](venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:113) self.nb_channels = (
--> [114](venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:114)     self.estimator.input_shape[0] if self.estimator.channels_first else self.estimator.input_shape[2]
    [115](venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:115) )
    [117](venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:117) # Optimal number of iterations
    [118](venv/lib/python3.10/site-packages/art/attacks/evasion/geometric_decision_based_attack.py:118) iteration = round(self.max_iter / 500)

AttributeError: 'ScikitlearnRandomForestClassifier' object has no attribute 'channels_first'

Expected behavior

A clear and concise error explaining that the ScikitlearnRandomForestClassifier is not compatible with the GeoDA attack. For instance:

EstimatorError: GeoDA requires an estimator derived from ..., the provided classifier is an instance of <class 'art.estimators.classification.scikitlearn.ScikitlearnRandomForestClassifier'> and is derived from (<class 'art.estimators.classification.scikitlearn.ScikitlearnClassifier'>,).

System information (please complete the following information):

  • OS: Ubuntu 22
  • Python version: 3.10
  • ART version or commit number: 1.18.1

jetlime avatar Sep 11 '24 23:09 jetlime