Spectral normalization raises OperatorNotAllowedInGraphError

[lo-zed]

I was trying to use a spectral normalization layer in one of my projects and ran into a bug. When trying to fit the model I got the error:

    File "<venv/path>/lib/python3.10/site-packages/keras/src/engine/training.py", line 1401, in train_function  *
        return step_function(self, iterator)
    File "<venv/path>/lib/python3.10/site-packages/keras/src/engine/training.py", line 1384, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "<venv/path>/lib/python3.10/site-packages/keras/src/engine/training.py", line 1373, in run_step  **
        outputs = model.train_step(data)
    File "<venv/path>/lib/python3.10/site-packages/keras/src/engine/training.py", line 1150, in train_step
        y_pred = self(x, training=True)
    File "<venv/path>/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
        raise e.with_traceback(filtered_tb) from None

    OperatorNotAllowedInGraphError: Exception encountered when calling layer 'spectral_normalization' (type SpectralNormalization).

    Using a symbolic `tf.Tensor` as a Python `bool` is not allowed. You can attempt the following resolutions to the problem: If you are running in Graph mode, use Eager execution mode or decorate this function with @tf.function. If you are using AutoGraph, you can try decorating this function with @tf.function. If that does not work, then you may be using an unsupported feature or your source code may not be visible to AutoGraph. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/limitations.md#access-to-source-code for more information.

    Call arguments received by layer 'spectral_normalization' (type SpectralNormalization):
      • inputs=tf.Tensor(shape=(None, 2), dtype=float32)
      • training=True

So I modified the class in order to make it work. See attached file for minimal example. spectral_norm_bug.zip

[lo-zed]

sorry, here are my packages versions:

• tensorflow 2.15.0.post1
• keras 2.15.0
• numpy 1.26.3

[sachinprasadhs]

Could you please upgrade the Keras package to latest 3.0.4 using pip install -U keras and try again. Let us know the outcome.

[lo-zed]

After installing keras 3.0.4 I got this error (with the same file as uploaded above)

Iterating over a symbolic `tf.Tensor` is not allowed. You can attempt the following resolutions to the problem: If you are running in Graph mode, use Eager execution mode or decorate this function with @tf.function. If you are using AutoGraph, you can try decorating this function with @tf.function. If that does not work, then you may be using an unsupported feature or your source code may not be visible to AutoGraph. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/limitations.md#access-to-source-code for more information.

Arguments received by SpectralNormalization.call():
  • inputs=tf.Tensor(shape=(None, 2), dtype=float32)
  • training=True

My fix seems to work, although there was no warning about any 'if clause' this time.

[sachinprasadhs]

tf.cond is the ideal way to use conditional checks, as per your Gist here https://colab.sandbox.google.com/gist/sachinprasadhs/0f0d5450bffa617f6af2e7247dc53eb9/19140.ipynb.

If your issue is resolved, could you please close the issue. Thanks!

[lo-zed]

Well, it is not really resolved since there is still an error when runnning the code without my fix. Would you rather I'd make a PR? I just couldn't find the time yet.

[SuryanarayanaY]

Hi @lo-zed ,

I have checked and this is an issue with TF backend with Graph mode.Checked with torch backend and it works fine.

Since Kears3 is now multi backend you need to fix the issue at Keras3 code here.

[SuryanarayanaY]

Hi @lo-zed ,

This has been fixed in TF2.16.1 as per attached gist. Please check and confirm.

[lo-zed]

Hi @SuryanarayanaY , I can confirm that the problem is fixed with tf 2.16.1. This issue can be closed. Thank you for your help.

[SuryanarayanaY]

Hi @lo-zed ,

Thanks for confirmation. Closing the issue now as per user confirmation.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No