Adversarial debiasing doesn't have Tensorflow reuse persistence

[xmpuspus]

I'm trying to use the adversarial debiasing method but always getting this issue:

Variable debiased_classifier/classifier_mode/W1 already exists, disallowed. Did you mean to set reuse-True...

I suggest using something like

with tf.variable_scope("classifier_model") as scope:
            W1 = tf.get_variable('W1', [features_dim, self.classifier_num_hidden_units],
                                  initializer=tf.contrib.layers.xavier_initializer(seed=self.seed1))
            b1 = tf.Variable(tf.zeros(shape=[self.classifier_num_hidden_units]), name='b1')
            scope.reuse_variables()
            h1 = tf.nn.relu(tf.matmul(features, W1) + b1)
            h1 = tf.nn.dropout(h1, keep_prob=keep_prob, seed=self.seed2)
            W2 = tf.get_variable('W2', [self.classifier_num_hidden_units, 1],
                                 initializer=tf.contrib.layers.xavier_initializer(seed=self.seed3))
            b2 = tf.Variable(tf.zeros(shape=[1]), name='b2')
            pred_logit = tf.matmul(h1, W2) + b2
            pred_label = tf.sigmoid(pred_logit)

within the code but I may have missed something. Would love some eyes on this issue.

[pronics2004]

Could you please provide a code snippet of the issues or some more details about your use case. Are you creating multiple instances of the AdversarialDebiasing in the same session? Did the change you are suggesting fix the issue?

[xmpuspus]

Hi,

Thanks for looking into this, Prasanna. I hope you can help me out if I have missed some steps.

This was the code I used:

privileged_groups, unprivileged_groups = get_attributes(data_train, selected_attr=[priv_category])
sess = tf.Session()

debiased_model = AdversarialDebiasing(privileged_groups = privileged_groups,
                          unprivileged_groups = unprivileged_groups,
                          scope_name='debiased_classifier',
                          num_epochs=10,
                          debias=True, sess=sess)

debiased_model.fit(data_train)

data_pred = debiased_model.predict(data_test)

Running above code once returns fine but running it a second time, say for example just to change parameters or doing a .fit() on a different dataset returns:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-20-69a62ef0fae3> in <module>
      8                           debias=True, sess=sess)
      9 
---> 10 debiased_model.fit(data_orig_sex_train)
     11 
     12 # fair = get_fair_metrics_and_plot(data_orig_sex_test, debiased_model, plot=False, model_aif=True)

~/anaconda3/lib/python3.7/site-packages/aif360/algorithms/transformer.py in wrapper(self, *args, **kwargs)
     25     @wraps(func)
     26     def wrapper(self, *args, **kwargs):
---> 27         new_dataset = func(self, *args, **kwargs)
     28         if isinstance(new_dataset, Dataset):
     29             new_dataset.metadata = new_dataset.metadata.copy()

~/anaconda3/lib/python3.7/site-packages/aif360/algorithms/inprocessing/adversarial_debiasing.py in fit(self, dataset)
    141 
    142             # Obtain classifier predictions and classifier loss
--> 143             self.pred_labels, pred_logits = self._classifier_model(self.features_ph, self.features_dim, self.keep_prob)
    144             pred_labels_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(labels=self.true_labels_ph, logits=pred_logits))
    145 

~/anaconda3/lib/python3.7/site-packages/aif360/algorithms/inprocessing/adversarial_debiasing.py in _classifier_model(self, features, features_dim, keep_prob)
     81         with tf.variable_scope("classifier_model"):
     82             W1 = tf.get_variable('W1', [features_dim, self.classifier_num_hidden_units],
---> 83                                   initializer=tf.contrib.layers.xavier_initializer())
     84             b1 = tf.Variable(tf.zeros(shape=[self.classifier_num_hidden_units]), name='b1')
     85 

~/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py in get_variable(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint, synchronization, aggregation)
   1494       constraint=constraint,
   1495       synchronization=synchronization,
-> 1496       aggregation=aggregation)
   1497 
   1498 

~/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint, synchronization, aggregation)
   1237           constraint=constraint,
   1238           synchronization=synchronization,
-> 1239           aggregation=aggregation)
   1240 
   1241   def _get_partitioned_variable(self,

~/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint, synchronization, aggregation)
    560           constraint=constraint,
    561           synchronization=synchronization,
--> 562           aggregation=aggregation)
    563 
    564   def _get_partitioned_variable(self,

~/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py in _true_getter(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, constraint, synchronization, aggregation)
    512           constraint=constraint,
    513           synchronization=synchronization,
--> 514           aggregation=aggregation)
    515 
    516     synchronization, aggregation, trainable = (

~/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py in _get_single_variable(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource, constraint, synchronization, aggregation)
    862         tb = [x for x in tb if "tensorflow/python" not in x[0]][:5]
    863         raise ValueError("%s Originally defined at:\n\n%s" %
--> 864                          (err_msg, "".join(traceback.format_list(tb))))
    865       found_var = self._vars[name]
    866       if not shape.is_compatible_with(found_var.get_shape()):

ValueError: Variable debiased_classifier/classifier_model/W1 already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

  File "/Users/xmpuspus/anaconda3/lib/python3.7/site-packages/aif360/algorithms/inprocessing/adversarial_debiasing.py", line 83, in _classifier_model
    initializer=tf.contrib.layers.xavier_initializer())
  File "/Users/xmpuspus/anaconda3/lib/python3.7/site-packages/aif360/algorithms/inprocessing/adversarial_debiasing.py", line 143, in fit
    self.pred_labels, pred_logits = self._classifier_model(self.features_ph, self.features_dim, self.keep_prob)
  File "/Users/xmpuspus/anaconda3/lib/python3.7/site-packages/aif360/algorithms/transformer.py", line 27, in wrapper
    new_dataset = func(self, *args, **kwargs)
  File "<ipython-input-19-69a62ef0fae3>", line 10, in <module>
    debiased_model.fit(data_orig_sex_train)
  File "/Users/xmpuspus/anaconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

Is there anything wrong with how I run things? I haven't tried a proper PR and fix on the code but quick inspection tells me my suggestion in the first thread might fix things but I defer to your recommmendations.

[pronics2004]

Yes, this is a valid point. fit() here was designed to be called once on the whole dataset.

Changing the scope_name for each instance of AdversarialDebiasing should fix this.

Your use case is to call fit() on different chunks of data separately? And perhaps update the model with additional datasets?

The fix you suggested should work. Feel free to submit a PR. Thanks!

[ciliney]

Hi Xavier,

I've got the same problem as you. Tried your fix but it didn't work for me - my version of aif360 seems different to yours (v0.4.0) because the initializer parameter settings are different from yours. I tried setting reuse = True and reuse=tf.AUTO_REUSE (see bolded) in the scope but still getting the same error. Have you got any ideas for me please?

Thanks a lot

with tf.variable_scope("classifier_model", reuse = True):

        W1 = tf.get_variable('W1', [features_dim, self.classifier_num_hidden_units],
                              initializer=tf.initializers.glorot_uniform(seed=self.seed1))

        b1 = tf.Variable(tf.zeros(shape=[self.classifier_num_hidden_units]), name='b1')

        h1 = tf.nn.relu(tf.matmul(features, W1) + b1)

        h1 = tf.nn.dropout(h1, keep_prob=keep_prob, seed=self.seed2)

        W2 = tf.get_variable('W2', [self.classifier_num_hidden_units, 1],
                             initializer=tf.initializers.glorot_uniform(seed=self.seed3))
        b2 = tf.Variable(tf.zeros(shape=[1]), name='b2')

        pred_logit = tf.matmul(h1, W2) + b2
        pred_label = tf.sigmoid(pred_logit)