How to add DDP with val loader?

[cswwp]

How to add DDP with val evaluation? Is it same with train? @yangkky

[yangkky]

You'd have to find a way to compile the validation statistics onto one process.

On Fri, Oct 23, 2020 at 6:07 AM cswwp [email protected] wrote:

How to add DDP with val evaluation? Is it same with train? @yangkky https://github.com/yangkky

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/yangkky/distributed_tutorial/issues/6, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEMNWF465QKJEIFUHTKMDTSMFIVVANCNFSM4S4LGKEQ .

[rancheng]

This code is pretty clear: https://github.com/rwightman/pytorch-image-models/blob/master/train.py#L730-L732

            reduced_loss = reduce_tensor(loss.data, args.world_size)
            acc1 = reduce_tensor(acc1, args.world_size)
            acc5 = reduce_tensor(acc5, args.world_size)