BurakHasircioglu

Understanding the relation between parallelism and microbatches

1

Hi all, In DP-SGD, given a constant number of minibatches, let us say 256, I was wondering why increasing the number of microbatches has a bad effect on the performance....