tf.keras.utils.pack_x_y_sample_weight cannot compute gradient

[VictoriaGriffith]

System information.

• Have I written custom code (as opposed to using a stock example script provided in Keras): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 20.04
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): TF2.9.1
• Python version: 3.9
• Bazel version (if compiling from source):
• GPU model and memory:

Describe the problem. tf.keras.utils.pack_x_y_sample_weight cannot compute gradient

Describe the current behavior. Raises error.

Describe the expected behavior. The backward computation should succeed.

Standalone code to reproduce the issue.

import tensorflow as tf

x = tf.random.uniform([2, 2], dtype=tf.float32)
y = (1, 2,)
sample_weight = None
print(x)
result = tf.keras.utils.pack_x_y_sample_weight(x, y=y, sample_weight=sample_weight, )
print(result)

with tf.GradientTape(persistent=True,) as g:
  g.watch(x)
  res = tf.keras.utils.pack_x_y_sample_weight(x, y=y, sample_weight=sample_weight, ) 
grad = g.gradient(res, x) # AttributeError

Output.

tf.Tensor(
[[0.68789124 0.48447883]
 [0.9309944  0.252187  ]], shape=(2, 2), dtype=float32)
(<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[0.68789124, 0.48447883],
       [0.9309944 , 0.252187  ]], dtype=float32)>, (1, 2))
AttributeError: 'int' object has no attribute '_id'

[sushreebarsa]

@gadagashwini I was able to replicate the issue on colab, please find the gist here. Thank you!

[mattdangerw]

The pack/unpack utilities are very simple. They just move things in and out of tuples. It looks like the error you are hitting is just from trying to attach a gradient to a python integer (y) in your code example, which is not supported for gradient_tape.

The following works.

import tensorflow as tf

x = tf.random.uniform([2, 2], dtype=tf.float32)
y = tf.constant((1, 2,), dtype=tf.float32)

with tf.GradientTape(persistent=True,) as g:
  g.watch(x)
  res = tf.keras.utils.pack_x_y_sample_weight(x, y=y, sample_weight=sample_weight, ) 
grad = g.gradient(res, x)

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No