AssertionError: compute_gradients() on the differentially private optimizer was not called.

[mayankshah1607]

I randomly run into the following error:

AssertionError: compute_gradients() on the differentially private optimizer was not called.

The reason is unknown, as the above error does not seem to appear if I restart the notebook a couple of times or re-run the script.

Additional logs:

WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.optimizer
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.global_step
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-24-c057947b86a8> in <module>()
----> 1 fed_learn('resnet', 10, True)

11 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    966           except Exception as e:  # pylint:disable=broad-except
    967             if hasattr(e, "ag_error_metadata"):
--> 968               raise e.ag_error_metadata.to_exception(e)
    969             else:
    970               raise

AssertionError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:541 train_step  **
        self.trainable_variables)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1814 _minimize
        optimizer.apply_gradients(zip(gradients, trainable_variables))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizers.py:775 apply_gradients
        self.optimizer.apply_gradients(grads, global_step=self.iterations)
    /usr/local/lib/python3.6/dist-packages/tensorflow_privacy/privacy/optimizers/dp_optimizer.py:183 apply_gradients
        'compute_gradients() on the differentially private optimizer was not'

    AssertionError: compute_gradients() on the differentially private optimizer was not called. Which means that the training is not differentially private. It happens for example in Keras training in TensorFlow 2.0+.

[nightldj]

Hi, Do you use a setting in the error message? Using TFPrivacy optimizer in TF2 Keras and model.fit may cause the privacy mechanism to be not effectively applied, probably due to the optimizer usage in model.fit. A workaround now is to write a customized training loop and explicitly call compute_gradients before apply_gradients.

[mayankshah1607]

Hi, @nightldj Thanks for your quick response. This seems a little weird to me because model.fit sometimes works, and sometimes it doesn't. However, I'll try the custom training loop and let you know.

Do we have plans on fixing this issue? It would be very helpful to have DP optimisers to work with keras model.fit

[galenmandrew]

Yes, fixing TFP to work with keras optimizers in TF 2.0 is a high priority feature for us now, although I don't have a specific date I can promise you it will be finished.

[mayankshah1607]

Hi, just wanted to point out that this error doesn't seem to occur once I restarted the notebook and ran it in another environment - I am able to train keras models with TFP. I'm still not sure why this error occasionally occurs. Any suggestions?

[mayankshah1607]

Fixed it. I used this helper function as a wrapper around the TFP optimizer, and used that in model.fit instead, and it worked. :)

[emanuelecuzari]

Fixed it. I used this helper function as a wrapper around the TFP optimizer, and used that in model.fit instead, and it worked. :)

Can you please indicate which function you refer to? I'm having the same issue and I can't find the function you refer to

[dmf49]

Is there any status on this error? While I am using the boiler plate fix, I would prefer to use a function from tensorflow privacy itself.

[XRH-LAB]

i am having the same problems,and i update my tf from 2.3.0 to 2.4.0,but the same mistake is still there.is there anybody can help me solve this?

[SamaaG]

I am running into this issue and was wondering if anyone was able to implement the wrapper function and get it to work. If so could you share a code snippet? Also, do we have a timeline on when this issue is expected to be fixed in TFP?

[ArunTellis]

Hi im getting the same error please do help me with it. Since its my major project i need the solution urgently. pleasse

AssertionError: in user code:

File "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py", line 1401, in train_function  *
    return step_function(self, iterator)
File "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py", line 1384, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py", line 1373, in run_step  **
    outputs = model.train_step(data)
File "<ipython-input-131-69563c87119c>", line 241, in train_step
    d_optimizer.apply_gradients(zip(gradients, self.discriminator.trainable_weights))
File "/usr/local/lib/python3.10/dist-packages/tensorflow_privacy/privacy/optimizers/dp_optimizer.py", line 259, in apply_gradients
    assert self._was_compute_gradients_called, (

AssertionError: compute_gradients() on the differentially private optimizer was not called. Which means that the training is not differentially private. It happens for example in Keras training in TensorFlow 2.0+.