privacy
privacy copied to clipboard
tensorflow
Calculating the privacy budget in the absence of additive noise
Hello. The "rdp_accountant" code assumes an additive Sampled Gaussian Mechanism (SGM). However, I am interested in calculating the privacy budget (epsilon and delta) when no noise is added. In this case, the privacy is offered by other adjustments on the queries (gradients). Any idea on how to calculate the privacy measures in this case? Many thanks in advance.
Hello,
Right now, we only support that subsampled Gaussian mechanism. Can you be a bit more specific about what other "adjustments on the queries" you have in mind?
Thanks, peter
Hi Peter, Thank you so much for the reply. I really appreciate it. Adjustments like clipping gradient. I am interested in calculating the privacy budget caused by just clipping gradient, for example.
The best, Mehdi
Unfortunately, clipping the gradients alone does not provide differential privacy. You need to add noise, in addition to clipping, to be able to obtain differential privacy.
On Mon, Nov 2, 2020 at 8:46 AM mhdiamian [email protected] wrote:
Hi Peter, Thank you so much for the reply. I really appreciate it. Adjustments like clipping gradient. I am interested in calculating the privacy budget caused by just clipping gradient, for example.
The best, Mehdi
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tensorflow/privacy/issues/136#issuecomment-720589368, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7LCU2PWVILEU5VVBDN2ODSN3O4TANCNFSM4THVFA3Q .
Thanks again Peter. So, if the clip contributes to no privacy, why should we keep it? I mean, we could avoid the clip (if it offers no privacy) and reach a better accuracy (as the clip compromises the utility).
Adding noise alone also does not provide DP. You need to clip the gradients to bound the sensitivity of the query you are computing. The clipped gradients are then aggregated and noised. The combination of clipping and noise addition provides DP. Without the clipping step, we cannot assert that the sum query we are computing has a bounded sensitivity (meaning that changing the gradient of one item has a bounded impact on the sum of the gradients). Without noise, we get bounded sensitivity but we cannot provide DP (randomization is necessary). Hope this helps!
On Mon, Nov 2, 2020 at 11:26 AM mhdiamian [email protected] wrote:
Thanks again Peter. So, if the clip contributes to no privacy, why should we keep it? I mean, we could avoid the clip (if it offers no privacy) and reach a better accuracy (as the clip compromises the utility).
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tensorflow/privacy/issues/136#issuecomment-720676810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7LCU4MZEWD4I7JOPX65DDSN4BXHANCNFSM4THVFA3Q .
Does the current algorithm (the "rdp_accountant" code) of calculating delta and epsilon support other forms of clipping rather than that is proposed by Abadi? I mean, keeping noise addition (as it is), and clipping but in a modified manner.
Many thanks in advance.
Mehdi
We currently do l2 clipping (i.e. we make sure that each gradient has a bounded l2 norm). As mentioned before, this translates into a bound on the l2 sensitivity of the sum/average query performed in the update step of DP-SGD. Without a bound on the l2 sensitivity, we cannot prove that adding Gaussian offers DP or Renyi-DP. Therefore, doing other forms of clipping won't help if you intend to use the Gaussian mechanism.
On Tue, Nov 3, 2020 at 10:05 AM mhdiamian [email protected] wrote:
Does the current algorithm (the "rdp_accountant" code) of calculating delta and epsilon support other forms of clipping rather than that is proposed by Abadi? I mean, keeping noise addition (as it is), and clipping but in a modified manner.
Many thanks in advance.
Mehdi
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tensorflow/privacy/issues/136#issuecomment-721290735, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7LCUZKXH5NN5NGPAAYYHLSOBA7PANCNFSM4THVFA3Q .