Grishchenko Ivan

Hi! Can you please explain are weights of the model same on all GPUs after first batch or not? I mean, that we make copy of model on each GPU...