opacus
opacus copied to clipboard
pytorch
Training PyTorch models with differential privacy
## 🚀 Feature Is it possible for the privacy engine to allow for batchnorm layers when we freeze their running stats (i.e., all batchnorm layers in .eval() mode)? ## Motivation...
Summary: Other examples in Opacus follow this logic for arguments related to batch size: - batch_size and batch_size_test CIFAR-10 still uses the `sample_rate * len(train_data)`. This diff updates that, to...
Implementation of convolution backward with a convolution. Original implementation due to @zou3519 (https://gist.github.com/zou3519/080f3a296f190ea1730d97396d5267d6). The original code has been extended to handle the general case (i.e., groups, dilation and stride). There...
**TL;DR** We can use Functorch to compute per sample gradients automatically. This PR proposes to use Functorch to compute per sample gradients. The "meat" is in grad_sample/functorch.py. It basically does...
We want to warn users of an unexpected behaviour with `set_to_none` flag. Normally, both `nn.Module` and `Optimizer` let clients choose whether they want to remove the `.grad` attribute altogether or...
## 🐛 Bug Bug described here: https://discuss.pytorch.org/t/virtual-batch-sizing-with-multi-gpu-training/158614.
## 🚀 Feature Augmentation multiplicity as explained in Section 3.1 of https://arxiv.org/abs/2204.13650. Before clipping gradients, we may average gradients over different augmentations. ## Motivation Paper shows improved results compared to...
Differential Revision: D37811390
`PrivacyEngine.make_private_with_epsilon` does not behave as explained by the documentation. In particular, `poisson_sampling` can not passed as an argument to this method. I think that the error is due to calling...
## 🐛 Bug Hi, thanks so much for quick response to previous issues! I've recently been receiving a small bug for Expanded Weights where my model parameters can't handle some...
