Duncan Riach
Duncan Riach
Sorry for the delay in responding, Phil; I was on vacation. I have not run this code or got into debugging it. Just from looking at it, I can see...
Hi @yanniskar, thank you for isolating this issue, and for providing a reproducer. Please will you open an issue against the stock TensorFlow repo for this. Please reference this current...
Also @yanniskar, for my records of customers who value this work, please can I know a bit more about you? Please could you give me your name and/or your affiliation?
Thank you, and my pleasure. BTW, from TensorFlow version **2.7 onwards**, you no longer need to manually serialize (`num_parallel_calls=1`) your `dataset.map` preprocessing. @reedwm added a step that ensures that that...
That's a very thorough report, @ornellamarciano; thank you. Having scanned your code, what stands out to me is your use of `tf.compat.v1.keras.layers.Embedding`. We have isolated a source of nondeterminism to...
Right, I would expect you to see a much smaller amount of noise accumulating when using float64, which seems to be what you witnessed.
I don't have an ETA for you, but I can tell you that we're actively working on it and that's it's top priority. I'm hoping for a release in the...
Hi @ornellamarciano, Thanks for checking-in. From stock TensorFlow version 2.5 onwards, the use of TensorFlow's segment reduction ops running on a GPU, with the expectation of reproducibility (i.e. when TF_DETERMINISTIC_OPS...
You might also want to try applying the unreleased patch from the cloned repo. See the discussion in [issue 19](https://github.com/NVIDIA/framework-determinism/issues/19).
Hi @SageAgastya, I assume that you're attempting to apply the `patch` function from the PyPI distribution. We have paused development in the middle of a major update and will not...