PaddleFL icon indicating copy to clipboard operation
PaddleFL copied to clipboard

Published 20 hours ago • verified publisher icon PaddlePaddle

DPSGD question

Open EveNovikova opened this issue 3 years ago • 3 comments

Could you clarify implementation details for DPSGD:

  1. where clipping operation is implemented - client (trainer) or aggregator (server)?
  2. where noise addition is implemented - client (trainer) or aggregator (server)?
  3. what is the purpose for batch_size for the server?

EveNovikova avatar Mar 03 '21 16:03 EveNovikova

Could you clarify implementation details for DPSGD:

  1. where clipping operation is implemented - client (trainer) or aggregator (server)?
  2. where noise addition is implemented - client (trainer) or aggregator (server)?
  3. what is the purpose for batch_size for the server?
  1. where clipping operation is implemented - client (trainer)
  2. where noise addition is implemented - client (trainer)
  3. what is the purpose for batch_size for the server? batch_size is only used for training on client, is not used for aggregation on server.

jhjiangcs avatar Mar 08 '21 09:03 jhjiangcs

Thank you for the reply. Could then explain or give any clue why DPSGD training is more time consuming. Network traffic also increases significantly in comparison with FedAvg and SecAggr strategies. Clipping and adding noise does not seem too much computationally expensive operations task (if it is implemented as in arXiv:1607.00133v2), and if no extra aggregation operation is introduced traffic should not also increase a lot.

EveNovikova avatar Mar 09 '21 12:03 EveNovikova

Thank you for the reply. Could then explain or give any clue why DPSGD training is more time consuming. Network traffic also increases significantly in comparison with FedAvg and SecAggr strategies. Clipping and adding noise does not seem too much computationally expensive operations task (if it is implemented as in arXiv:1607.00133v2), and if no extra aggregation operation is introduced traffic should not also increase a lot.

Can you provide more details, such as log or screenshot?

jhjiangcs avatar Apr 02 '21 08:04 jhjiangcs