Convergence and group size

[emduc]

Hi!

I'm apologize for the question that might come a few years late, but I have been digging into your paper, and there is one graph that I cannot explain: the number of round to convergence according to the group size.

I do not understand the intuition behind this result. If anything, I expect that when not grouping we converge faster. My second intuition is that since grouping is a simple aggregation, it should be equivalent. What is the explanation behind the fact that the model requires less rounds when aggregating the gradients and item representations? How exactly were those results computed?

I would truly appreciate if you could help me out :)

Thanks and have a great day!

[yjw1029]

In federated learning, the size of the group is analogous to the batch size in traditional deep learning. This is because the more users sampled in one round, the more data the gradient will be based on for that round. In general, large batch sizes lead to more stable gradients and fewer convergence steps, but also leads to weaker generalization.

You can refer to this blog for further analysis on the impact of batch size, especially the reason why larger batches result in fewer updates.