Lab1, Part1: Assertion fails

[Madina-S]

In "### Defining a network Layer ###" while defining OurDenseLayer class, even in solution

AssertionError                            Traceback (most recent call last)
[<ipython-input-3-19850ceade07>](https://z1pxgxqagka-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20230223-060147-RC00_511737765#) in <module>
     36 # test the output!
     37 print(y.numpy())
---> 38 mdl.lab1.test_custom_dense_layer_output(y)

3 frames
[/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py](https://z1pxgxqagka-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20230223-060147-RC00_511737765#) in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf)
    842                                 verbose=verbose, header=header,
    843                                 names=('x', 'y'), precision=precision)
--> 844             raise AssertionError(msg)
    845     except ValueError:
    846         import traceback

AssertionError: 
Arrays are not almost equal to 7 decimals
[FAIL] output is of incorrect value. expected [[0.31251222 0.12787536 0.9313476 ]] but got [[0.2697859  0.45750418 0.66536945]]
Mismatched elements: 3 / 3 (100%)
Max absolute difference: 0.32962883
Max relative difference: 0.72049356
 x: array([[0.3125122, 0.1278754, 0.9313476]], dtype=float32)
 y: array([[0.2697859, 0.4575042, 0.6653695]], dtype=float32)```

[sm1899]

global seed alone fails to ensure reproducibility, probably have to specify an operation-level seed to get the same result

[xubinlikesleeping]

hi, i got the same question, did you get this done!

[danielkon96]

Ran into the same exact issue.

[highel]

It sems that to get deterministic behavior in the latest Tensor Flow you need to use

tf.keras.utils.set_random_seed(1)
tf.config.experimental.enable_op_determinism()

However this will not solve the problem, as you need to recalculate the results in test library anyway

[GKG1312]

Getting same issue here:

---

AssertionError Traceback (most recent call last) in 34 # test the output! 35 print(y.numpy()) ---> 36 mdl.lab1.test_custom_dense_layer_output(y)

3 frames /usr/local/lib/python3.9/dist-packages/numpy/testing/_private/utils.py in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf) 842 verbose=verbose, header=header, 843 names=('x', 'y'), precision=precision) --> 844 raise AssertionError(msg) 845 except ValueError: 846 import traceback

AssertionError: Arrays are not almost equal to 7 decimals [FAIL] output is of incorrect value. expected [[0.14785646 0.23566338 0.8568521 ]] but got [[0.2697859 0.45750418 0.66536945]] Mismatched elements: 3 / 3 (100%) Max absolute difference: 0.2218408 Max relative difference: 0.4848935 x: array([[0.1478565, 0.2356634, 0.8568521]], dtype=float32) y: array([[0.2697859, 0.4575042, 0.6653695]], dtype=float32)

Did anyone get any solution?

[highel]

Yes, the suggested PR by prashantkhurana works for me. Maybe the version of mitdeeplearning should be upped.

I use NVidia CUDA RTX A5000 with x64 ubuntu, tensorflow 2.12, python 3.11

UPD: also tested on Apple M1 Pro tensorflow-metal w tensorflow 2.12

[jilagan]

I'm getting the same error using Google Colab.

[zurichantonpopov]

# Google Colab
class OurDenseLayer(tf.keras.layers.Layer):
  def __init__(self, n_output_nodes):
    super(OurDenseLayer, self).__init__()
    self.n_output_nodes = n_output_nodes

  def build(self, input_shape):
    d = int(input_shape[-1])
    # Define and initialize parameters: a weight matrix W and bias b
    # Note that parameter initialization is random!
    self.W = self.add_weight("weight", shape=[d, self.n_output_nodes]) # note the dimensionality
    self.b = self.add_weight("bias", shape=[1, self.n_output_nodes]) # note the dimensionality

  def call(self, x):
    '''TODO: define the operation for z (hint: use tf.matmul)'''
    z = tf.matmul(x, self.W) + self.b

    '''TODO: define the operation for out (hint: use tf.sigmoid)'''
    y = tf.sigmoid(z)
    return y

# Since layer parameters are initialized randomly, we will set a random seed for reproducibility
tf.random.set_seed(1)
layer = OurDenseLayer(3)
layer.build((1,2))
x_input = tf.constant([[1,2.]], shape=(1,2))
y = layer.call(x_input)

# test the output!
print(y.numpy())
mdl.lab1.test_custom_dense_layer_output(y)

AssertionError: 
Arrays are not almost equal to 7 decimals
[FAIL] output is of incorrect value. expected [[0.2086701  0.68627274 0.14512508]] but got [[0.2697859  0.45750418 0.66536945]]
Mismatched elements: 3 / 3 (100%)
Max absolute difference: 0.52024436
Max relative difference: 0.78188795
 x: array([[0.2086701, 0.6862727, 0.1451251]], dtype=float32)
 y: array([[0.2697859, 0.4575042, 0.6653695]], dtype=float32)

same here

[zaineb]

Getting the same assertion error:

Arrays are not almost equal to 7 decimals [FAIL] output is of incorrect value. expected [[0.9217561 0.39742658 0.14384568]] but got [[0.2697859 0.45750418 0.66536945]] Mismatched elements: 3 / 3 (100%)

[rlamba89]

Getting same assertion error

[adlternative]

It looks like #118 has solved the issue. When will we merge it?

[oSamDavis]

Same error getting on Google colab. Any fix yet?

[zaineb]

Still getting this error on colab even after replacing tf.random.set_seed(1) with tf.keras.utils.set_random_seed(1) tf.config.experimental.enable_op_determinism()

[highel]

Still getting this error on colab even after replacing tf.random.set_seed(1) with tf.keras.utils.set_random_seed(1) tf.config.experimental.enable_op_determinism()

I think is's because mitdeeplearning library file mitdeeplearning/lab1.py need to be patched also with the new values but in colab you use remote version, this should work after version update. See the PR above. I tested in both on x64 and Apple M1 so likely it will work.

[mkx000]

same error

/usr/local/lib/python3.10/dist-packages/ipykernel/ipkernel.py:283: DeprecationWarning: should_run_async will not call transform_cell automatically in the future. Please pass the result to transformed_cell argument and any exception that happen during thetransform in preprocessing_exc_tuple in IPython 7.17 and above. and should_run_async(code) /usr/lib/python3.10/random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version return self.randrange(a, b+1) [[0.27064407 0.1826951 0.50374055]]

AssertionError Traceback (most recent call last) in <cell line: 40>() 38 # test the output! 39 print(y.numpy()) ---> 40 mdl.lab1.test_custom_dense_layer_output(y)

1 frames [... skipping hidden 2 frame]

/usr/local/lib/python3.10/dist-packages/numpy/testing/_private/utils.py in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf) 842 verbose=verbose, header=header, 843 names=('x', 'y'), precision=precision) --> 844 raise AssertionError(msg) 845 except ValueError: 846 import traceback

AssertionError: Arrays are not almost equal to 7 decimals [FAIL] output is of incorrect value. expected [[0.27064407 0.1826951 0.50374055]] but got [[0.2697859 0.45750418 0.66536945]] Mismatched elements: 3 / 3 (100%) Max absolute difference: 0.27480906 Max relative difference: 0.60067004 x: array([[0.2706441, 0.1826951, 0.5037405]], dtype=float32) y: array([[0.2697859, 0.4575042, 0.6653695]], dtype=float32)