Tomasz Bartczak
Tomasz Bartczak
I ran all the unit tests in keras/engine, however I couldn't find a good way and an example for the test to write for this functionality. I would appreciate some...
I've added the test now - it fails if one removes the `model.distribute_reduction_method = "sum"` configuration
Hi @rchao, thanks for the review. > * With MirroredStrategy, the output from the `strategy.run(run_step)` is no longer a distributed value (and therefore it takes [this](https://github.com/keras-team/keras/blob/b88f655b5fb81d157e1489cf4f3eff577a81203f/keras/engine/training.py#L3684-L3685) path) [...] Investigating `strategy.run(run_step)`...
> * do the returned replica values from `strategy.run()` have the same values, or they differ? > > * if you increase the number of replicas, do the loss and...
@gbaned yes, I've merged master and resolved conflicts.
@haifeng-jin code reformatted
@gbaned it's rebased now on top of master
@rchao yes, that's right indeed, it is just needed when a custom `train_step` is used. I've added a relevant note in the docstring in https://github.com/keras-team/keras/pull/16664/commits/5be6742aedc47c23555b642b599f50323b85c61e At the same time it...
@lucasdavid I understand the idea of the default being an `auto` value. At the same time the suggested implementation would be something different than the issue I am trying to...
@lucasdavid Thanks for the explanation. I get it now. Current version in this branch didn't change the default, and so for TPU it stays 'first'. However I've updated the docstrings...