global seed alone fails to ensure reproducibility, probably have to specify an operation-level seed to get the same result

hi, i got the same question, did you get this done!

Ran into the same exact issue.

It sems that to get deterministic behavior in the latest Tensor Flow you need to use

tf.keras.utils.set_random_seed(1)
tf.config.experimental.enable_op_determinism()

However this will not solve the problem, as you need to recalculate the results in test library anyway

Getting same issue here:

---

AssertionError Traceback (most recent call last)
in
34 # test the output!
35 print(y.numpy())
---> 36 mdl.lab1.test_custom_dense_layer_output(y)

3 frames
/usr/local/lib/python3.9/dist-packages/numpy/testing/_private/utils.py in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf)
842 verbose=verbose, header=header,
843 names=('x', 'y'), precision=precision)
--> 844 raise AssertionError(msg)
845 except ValueError:
846 import traceback

AssertionError:
Arrays are not almost equal to 7 decimals
[FAIL] output is of incorrect value. expected [[0.14785646 0.23566338 0.8568521 ]] but got [[0.2697859 0.45750418 0.66536945]]
Mismatched elements: 3 / 3 (100%)
Max absolute difference: 0.2218408
Max relative difference: 0.4848935
x: array([[0.1478565, 0.2356634, 0.8568521]], dtype=float32)
y: array([[0.2697859, 0.4575042, 0.6653695]], dtype=float32)

Did anyone get any solution?

Yes, the suggested PR by prashantkhurana works for me. Maybe the version of mitdeeplearning should be upped.

I use NVidia CUDA RTX A5000 with x64 ubuntu, tensorflow 2.12, python 3.11

UPD: also tested on Apple M1 Pro tensorflow-metal w tensorflow 2.12

I'm getting the same error using Google Colab.

# Google Colab
class OurDenseLayer(tf.keras.layers.Layer):
  def __init__(self, n_output_nodes):
    super(OurDenseLayer, self).__init__()
    self.n_output_nodes = n_output_nodes

  def build(self, input_shape):
    d = int(input_shape[-1])
    # Define and initialize parameters: a weight matrix W and bias b
    # Note that parameter initialization is random!
    self.W = self.add_weight("weight", shape=[d, self.n_output_nodes]) # note the dimensionality
    self.b = self.add_weight("bias", shape=[1, self.n_output_nodes]) # note the dimensionality

  def call(self, x):
    '''TODO: define the operation for z (hint: use tf.matmul)'''
    z = tf.matmul(x, self.W) + self.b

    '''TODO: define the operation for out (hint: use tf.sigmoid)'''
    y = tf.sigmoid(z)
    return y

# Since layer parameters are initialized randomly, we will set a random seed for reproducibility
tf.random.set_seed(1)
layer = OurDenseLayer(3)
layer.build((1,2))
x_input = tf.constant([[1,2.]], shape=(1,2))
y = layer.call(x_input)

# test the output!
print(y.numpy())
mdl.lab1.test_custom_dense_layer_output(y)

AssertionError: 
Arrays are not almost equal to 7 decimals
[FAIL] output is of incorrect value. expected [[0.2086701  0.68627274 0.14512508]] but got [[0.2697859  0.45750418 0.66536945]]
Mismatched elements: 3 / 3 (100%)
Max absolute difference: 0.52024436
Max relative difference: 0.78188795
 x: array([[0.2086701, 0.6862727, 0.1451251]], dtype=float32)
 y: array([[0.2697859, 0.4575042, 0.6653695]], dtype=float32)

same here

Getting the same assertion error:

Arrays are not almost equal to 7 decimals
[FAIL] output is of incorrect value. expected [[0.9217561 0.39742658 0.14384568]] but got [[0.2697859 0.45750418 0.66536945]]
Mismatched elements: 3 / 3 (100%)

Getting same assertion error

It looks like #118 has solved the issue. When will we merge it?

Same error getting on Google colab. Any fix yet?

Still getting this error on colab even after replacing tf.random.set_seed(1) with
tf.keras.utils.set_random_seed(1)
tf.config.experimental.enable_op_determinism()

Still getting this error on colab even after replacing tf.random.set_seed(1) with tf.keras.utils.set_random_seed(1) tf.config.experimental.enable_op_determinism()

I think is's because mitdeeplearning library file mitdeeplearning/lab1.py need to be patched also with the new values but in colab you use remote version, this should work after version update. See the PR above. I tested in both on x64 and Apple M1 so likely it will work.

same error

/usr/local/lib/python3.10/dist-packages/ipykernel/ipkernel.py:283: DeprecationWarning: should_run_async will not call transform_cell automatically in the future. Please pass the result to transformed_cell argument and any exception that happen during thetransform in preprocessing_exc_tuple in IPython 7.17 and above.
and should_run_async(code)
/usr/lib/python3.10/random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version
return self.randrange(a, b+1)
[[0.27064407 0.1826951 0.50374055]]

AssertionError Traceback (most recent call last)
in <cell line: 40>()
38 # test the output!
39 print(y.numpy())
---> 40 mdl.lab1.test_custom_dense_layer_output(y)

1 frames
[... skipping hidden 2 frame]

/usr/local/lib/python3.10/dist-packages/numpy/testing/_private/utils.py in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf)
842 verbose=verbose, header=header,
843 names=('x', 'y'), precision=precision)
--> 844 raise AssertionError(msg)
845 except ValueError:
846 import traceback

AssertionError:
Arrays are not almost equal to 7 decimals
[FAIL] output is of incorrect value. expected [[0.27064407 0.1826951 0.50374055]] but got [[0.2697859 0.45750418 0.66536945]]
Mismatched elements: 3 / 3 (100%)
Max absolute difference: 0.27480906
Max relative difference: 0.60067004
x: array([[0.2706441, 0.1826951, 0.5037405]], dtype=float32)
y: array([[0.2697859, 0.4575042, 0.6653695]], dtype=float32)