adversarial-robustness-toolbox icon indicating copy to clipboard operation
adversarial-robustness-toolbox copied to clipboard

TF import overwrite when using tf2 for tf1 support

Open notbarrie opened this issue 3 years ago • 3 comments

Describe the bug An import in the _get_layers function in the TensorFlowClassifier class overwrites the import when trying to use the tf1 compatibility layer in tf2

To Reproduce Steps to reproduce the behavior:

  1. copy get_started_tensorflow.py script and add a line to disable eager execution
  2. Run the example get_started_tensorflow.py
  3. errors out on classifier instantiation (step 3)

File "scratch.py", line 40, in classifier = TensorFlowClassifier( File "C:\Users\Barrie Tomilson\Documents\github\troj-a-py\troj_env\lib\site-packages\art\estimators\classification\tensorflow.py", line 145, in init self._layer_names = self._get_layers() File "C:\Users\Barrie Tomilson\Documents\github\troj-a-py\troj_env\lib\site-packages\art\estimators\classification\tensorflow.py", line 533, in _get_layers graph = tf.get_default_graph() AttributeError: module 'tensorflow' has no attribute 'get_default_graph'

Expected behavior The _get_layers function should use a try/catch to import the appropriate tf version(?) , as in tf2 the get_default_graph function is stored under tf.compat.v1 but gets overwritten by import tensorflow as tf at the start of the function.

I have a try/except fix that seems to work locally: def _get_layers(self) -> List[str]: """ Return the hidden layers in the model, if applicable.

:return: The hidden layers in the model, input and output layers excluded.
"""
# pylint: disable=E0401
try:
    import tensorflow.compat.v1 as tf  # lgtm [py/repeated-import]
except:
    import tensorflow as tf

Screenshots N/A

System information (please complete the following information):

  • Windows
  • Python version 3.8.5
  • ART 1.7.0
  • TensorFlow V2

Let me know if any additional information is required

notbarrie avatar Sep 08 '21 18:09 notbarrie

Hi @notbarrie Thank you very much for using ART and opening this issue!

We are currently not testing TensorFlowClassifier with TensorFlow 2 - v1 compatibility mode, it is only tested with TensorFlow 1.x. We are testing TensorFlowV2Classifier with TensorFlow 2.

What is your reason to use TensorFlowClassifier in compatibility mode?

I think your proposed solution or something similar should work, but we should then adapt all methods of TensorFlowClassifier and test them accordingly in compatibility mode.

beat-buesser avatar Sep 08 '21 22:09 beat-buesser

The reason is simplicity pretty much, wanted to demo the tf1 functionality of our platform in a public facing colab notebook instance but tensorflow 2.X is installed by default so I thought it would be nice to demo it thru the compat layer strictly for convenience and not worry about removing the current version. Definitely works if tensorflow 2.X is removed though which I can do for now. I was just confused initially since the demo located in get_started_tensorflow.py did the same thing by importing tensorflow.compat.v1 as tf, and broke in that way as well.

notbarrie avatar Sep 09 '21 17:09 notbarrie

Hi @notbarrie I think you are right. I'm not sure what exactly happened with the example script. I think your proposal would work, but could we replace all imports in TensorFlowClassifier with

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

It should work with both versions for TensorFlow.

beat-buesser avatar Sep 13 '21 16:09 beat-buesser