Jared

Support `hvd.allreduce(..., op=hvd.Adasum)` in TF 2.X

3

**Environment:** 1. Framework: TensorFlow 2. Framework version: 2.1 (also occurs in 2.0) 3. Horovod version: 0.19.0 4. MPI version: 3.1.4 5. CUDA version: 10.2 6. NCCL version: 2.5.6 7. Python...

tf.keras saves step at end of batch

2

Running the following script with tensorflow==1.15.0: ``` import tensorflow.compat.v2 as tf import smdebug.tensorflow as smd from tempfile import TemporaryDirectory mnist = tf.keras.datasets.mnist (x_train, y_train), (x_test, y_test) = mnist.load_data() x_train, x_test...

The function signature for tf.nn.dropout in TF1 is: ```python tf.nn.dropout( x, keep_prob=None, noise_shape=None, seed=None, name=None, rate=None ) ``` while TF2 has: ```python tf.nn.dropout( x, rate, noise_shape=None, seed=None, name=None ) ```...

Gradient checkpointing: Wrap `tf.keras.Model` or `tf.keras.layers.Layer` in `tf.recompute_gradients()`

4

I'm implementing gradient checkpointing with my Tensorflow 2.1 project, following the doc here: https://www.tensorflow.org/api_docs/python/tf/recompute_grad When I try ```code model = tf.recompute_grad(model) ``` it fails, and likewise with ```code model.layer =...