Lucas David

@rchao @kretes may I suggest `auto` to be the default value, and the following decision flow: ```py class Reduction: AUTO = 'auto' FIRST = 'first' SUM = 'sum' CONCAT =...

@kretes from what I gathered from @rchao's [comment](https://github.com/keras-team/keras/pull/16664#issuecomment-1156837535), > The code path prevents the MirroredStrategy/PSS case from getting affected regardless of 'first' or 'sum', but it does affect TPUStrategy in...

Unless I'm missing something, `sum` can be used for `MirroredStrategy` and **will not** imply in behavioral changes: 1. `MirroredStrategy` + default training loop: `sum` works, because `first([l1]) == sum([l1]) ==...

haha thanks for the enthusiasm. I wasn't expecting you to pull the commit, it was more just to show that it works and that the tests pass. In any case,...

@kretes I wrote tests for `MultiWorkerMirroredStrategy` in this [commit](https://github.com/lucasdavid/keras/commit/47bc791b0eff4aa47089c262ca29bc1992bf0353). They are being skipped in my laptop though. I think I don't have the appropriate hardware for it.

It's my understanding we could still do some work on this (update the docs as requested by @rchao and maybe add some tests for the multi worker case), and that...

Okay, I can try to do it.

In the sparse case, we don't have to multiply `(target * -np.log(output)`, as target = 1 for label i and 0 for the remaining one. It's more efficient to simply...

p is the predictions or output. y is the labels or target. Starting from your own implementation: ```py def categorical_crossentropy(target, output, from_logits=False): ... return np.sum(target * -np.log(output), axis=-1, keepdims=False) ```...

@qlzh727 current behavior is squeezing both tensors (`[batch]` and `[batch, 1]`) into the shape `[batch]`, which is incorrectly reduced within mse. This problem is aggravated if the user has a...