Support for retraining the official BigGAN generator pretrained models

[gwern]

The official BigGAN generator pretrained models were released 2 or 3 weeks ago & can be used on Colab for free evaluations, and made a big splash as people started finding bizarre examples and making things like GANbreeder. Indeed, it's led to a bit of a backlash about it being boring or all-samey: "Helena Sarin: Why Bigger Isn’t Always Better With GANs And AI Art". It's hard to do better, though, since the 512-core TPUv3 slices are not publicly available, and a BigGAN might require 1+ GPU-years (especially with the large minibatches that might be necessary for the highest quality, see issue #1). Another issue is the lack of source code or discriminator pretrained model release, the latter of which is deliberate ("This work was conducted to advance the state of the art in generative adversarial networks for image generation. We are releasing the pre-trained generator to allow our work to be verified, which is standard practice in academia. It does not include the discriminator to minimize the potential for exploitation."), and the former of which seems increasingly unlikely (ajmooch has been asked multiple times and made no promises, suggesting Google/DM isn't going to allow it).

What would be ideal would be the ability to retrain the BigGAN G models. (The lack of discriminator models is a problem but can probably be fixed by setting the G learning rate to 0 and running for a few epochs on ImageNet before re-enabling G. Discriminators seem to learn faster than generators, anyway, so catching up to the G should be relatively quick.) This would let people drop in new datasets for finetuning and get fun new BigGANs which exploit the enormous high-quality prior information embedded in the official generators. It would be a big deal for artists & hobbyists (maybe not so much researchers), especially given that this is an easy-to-use codebase and has support for dropping in a folder-of-folder of images.

I don't know how easy this would be to implement (loading a Tensorflow-made model in PyTorch), but on the bright side, it offers a test of how compatible/accurate a reimplementation this codebase is - if it can't load and retrain the official generators, something's different.

[sxhxliang]

I tried to convert the TensorFlow models to PyTroch, but failed，because I seldom use TensorFlow before. I will use PyTroch to reproduce the results. (if Google open the source code)

[gwern]

As I said, it is almost certain that the original BigGAN code will never be open sourced, and if it ever is, it may take months or years. This repo is as good as it gets right now.