GenerativeZoo

A collection of generative algorithms and techniques implemented in Python.

This work is part of the Xecs TASTI project, nr. 2022005.

Prerequisites

You will need:

• python (see pyproject.toml for full version)
• Git
• Make
• a .secrets file with the required secrets and credentials
• load environment variables from .env
• NVIDIA Drivers(mandatory) and CUDA >= 12.1 (mandatory if Docker is not used)
• Weights & Biases account

Installation

Clone this repository (requires git ssh keys)

git clone --recursive [email protected]:caetas/GenerativeZoo.git
cd generativezoo

Using Docker

Create the image using the provided Dockerfile and then run the container:

docker build --tag generativezoo .
docker create --gpus all --shm-size=1g -i --name generativezoo_container generativezoo
docker start generativezoo_container

To access the shell, please run:

docker exec -it generativezoo_container /bin/bash

Note: Edit the Dockerfile if you want to include data or model checkpoints in your image.

Normal Installation

or if environment already exists

conda env create -f environment.yml
conda activate python3.10

On Linux

And then setup all virtualenv using make file recipe

(python3.10) $ make setup-all

You might be required to run the following command once to setup the automatic activation of the conda environment and the virtualenv:

direnv allow

Feel free to edit the .envrc file if you prefer to activate the environments manually.

On Windows

You can setup the virtualenv by running the following commands:

python -m venv .venv-dev
.venv-dev/Scripts/Activate.ps1
python -m pip install --upgrade pip setuptools
python -m pip install -r requirements/requirements.txt

To run the code please remember to always activate both environments:

conda activate python3.10
.venv-dev/Scripts/Activate.ps1

Models

Implemented Models

The listed models are already implemented and fully integrated in the model zoo. Models with 🚀 support Multi-GPU and Mixed Precision training using Accelerate. Models with 📦 support a latent space implementation with the flag --latent.

VAEs

• Vanilla VAE Paper | Code¹ | Script | Documentation
• Conditional VAE Paper | Code¹ | Script | Documentation
• Hierarchical VAE Paper | Code² | Script | Documentation

GANs

• Adversarial VAE Paper | Code¹ | Script | Documentation
• DC-GAN Paper | Code³ | Script | Documentation
• Conditional GAN Paper | Code³ | Script | Documentation
• CycleGAN Paper | Code⁴ | Script | Documentation
• Prescribed GAN Paper | Code⁵ | Script | Documentation
• Wasserstein GAN with Gradient Penalty Paper | Code⁶ | Script | Documentation

DDPMs

• DDPM^🚀📦 Paper | Code^7,8,9,21 | Script | Documentation
• Conditional DDPM^🚀📦 Paper | Code^10,21 | Script | Documentation
• Diffusion AE Paper | Code¹¹ | Script | Documentation

Score Matching

• SGM^🚀📦 Paper | Code^12,21 | Script | Documentation
• NCSNv2 Paper | Code¹³ | Script | Documentation

Autoregressive

• VQ-VAE + Transformer Paper | Code¹¹ | Script | Documentation
• VQ-GAN + Transformer Paper | Code¹¹ | Script | Documentation
• PixelCNN Paper | Code¹⁴ | Script | Documentation

Normalizing Flows

• Vanilla Flow Paper | Code¹⁴ | Script | Documentation
• RealNVP Paper | Code¹⁵ | Script | Documentation
• Glow Paper | Code¹⁶ | Script | Documentation
• Flow++ Paper | Code¹⁷ | Script | Documentation

Flow Matching

• Flow Matching^🚀📦 Paper | Code^18,21 | Script | Documentation
• Conditional Flow Matching^🚀📦 Paper | Code^18,21 | Script | Documentation
• Rectified Flows^🚀📦 Paper | Code¹⁹ | Script | Documentation

Stable Diffusion

• Stable Diffusion + LoRA^🚀 Paper | Code²⁰ | Script | Documentation
• ControlNet^🚀 Paper | Code²⁰ | Script | Documentation
• InstructPix2Pix^🚀 Paper | Code²⁰ | Script | Documentation

Multi-GPU support and Mixed Precision

Accelerate should be configured for your hardware setup using:

accelerate config

The models that support these features should be launched with:

accelerate launch --multi_gpu --mixed_precision=fp16 --num_processes=2 {script_name.py} {--arg1} {--arg2} ...

Datasets

The following datasets are ready to be used to train and sample from the provided models. They are automatically downloaded when you try to use them for the first time.

Grayscale

• MNIST Source
• FashionMNIST Source
• ChestMNIST++ Source
• OctMNIST++ Source
• PneumoniaMNIST++ Source
• TissueMNIST++ Source

RGB

• CIFAR-10 Source
• CIFAR-100 Source
• SVHN Source
• Places365 Source
• DTD Source
• TinyImageNet Source MANUAL DOWNLOAD REQUIRED Link
• Horse2Zebra Source MANUAL DOWNLOAD REQUIRED Link
• ImageNet-1k Source

Tracking

The code examples are setup to use Weights & Biases as a tool to track your training runs. Please refer to the full documentation if required or follow the following steps:

1. Create an account in Weights & Biases

2. If you have installed the requirements you can skip this step. If not, activate the conda environment and the virtualenv and run:

   pip install wandb
   

3. Run the following command and insert you API key when prompted:

   wandb login
   

If you want to fully disable Weights & Biases during training, use the flag --no_wandb.

If you want to turn off syncing with the server but want to retain the local copy, run:

wandb offline 

Repository Information

Documentation

Full documentation is available here: docs/.

Dev

See the Developer guidelines for more information.

Contributing

Contributions of any kind are welcome. Please read CONTRIBUTING.md for details and the process for submitting pull requests to us.

Please read MODELRULES.md for details on how you should build your models for this repository.

Security

Thank you for improving the security of the project, please see the Security Policy for more information.

License

This project is licensed under the terms of the CC-BY-4.0 license. See LICENSE for more details.

References

All the repositories used to generate this code are mentioned in each of the corresponding files and referenced in Implemented Models:

1. PyTorch-VAE
2. nvae
3. conditional-GAN
4. PyTorch-GAN
5. PresGANs
6. wgan-gp
7. minDiffusion
8. DenoisingDiffusionProbabilisticModels
9. ddim
10. Conditional_Diffusion_MNIST
11. Generative Models
12. SGM Tutorial
13. score_sde_pytorch
14. uvadlc_notebooks
15. real-nvp
16. Glow-PyTorch
17. flowplusplus
18. conditional-flow-matching
19. minRF
20. diffusers
21. guided-diffusion

Citation

If you publish work that uses GenerativeZoo, please cite GenerativeZoo as follows:

@misc{GenerativeZoo,
author = {Francisco Caetano},
title = {A collection of generative algorithms and techniques implemented in Python.},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/caetas/GenerativeZoo}},
year = {2024},
}