Tensorflow 2 Reimplementation (Image Generation)

Tensorflow 2 reimplementation of image generation model.

Results

GAN - mnist

CGAN - mnist

All random latent	Same latent per class

DCGAN - LSUN

DCGAN - CIFAR-10

DCGAN - mnist

LSGAN - CIFAR-10

WGAN-GP - LSUN

Neural Style Transfer

5000 Step From Content

Fast Style Transfer

Differences from the official implementation

• In Residual Block, follow the steps below.
  input -> (normalization -> activation -> conv) -> (normalization -> activation -> conv) -> add
• Use FIR Filter in Transposed Convolution
• Use Reflection Padding
• Use Leaky ReLU

AdaIN

Model List

GAN

• [x] GAN: Paper
• [x] CGAN: Paper
• [x] DCGAN: Paper
• [x] Conditional-DCGAN(cDCGAN)
• [x] LSGAN: Paper
• [x] WGAN: Paper
• [x] WGAN-GP: Paper
• [ ] SAGAN
• [ ] ProGAN(PGGAN)
• [ ] BigGAN
• [ ] StyleGAN

Style Transfer & Image to Image Translation

• [x] Neural Style Transfer: Paper
• [x] Fast Style Transfer: Paper
• [x] AdaIN: Paper
• [ ] CycleGAN
• [ ] StarGAN
• [ ] UNIT

TF2 Custom Layer

• [x] Resample Layers (Downsample, Upsample)
• [x] Padding Layers
• [x] Noise Layers
• [x] Linear Block (support noise, weight scaling)
• [x] Convolution Layers (support int pad, noise, weight scaling, fir filter)
• [x] Convolution Blocks (support normalization, activation, etc)
• [x] Residual Blocks (support shortcut, Resample, Transpose, etc)
• [x] Subpixel Convolution
• [x] ICNR Initializer
• [x] Decomposed Transposed Convolution
• [x] FIR Filter Layer (Need to set learning rate low or resolution high to use filter, according to experiment.)
• [x] Haar Transform Layers
• [x] Denormalization Layers (AdaIN, SPADE, ...)

Utilities

Docker

Build

~$ docker pull tensorflow/tensorflow:nightly-gpu
~$ docker build -f Dockerfile \
  -t tf/image-generation:nightly \
  --build-arg user_name=$USER \
  --build-arg user_uid=$UID \
  .

Docker Compose

interactive container
~$ docker-compose up -d
~$ docker exec -it tf_nightly /bin/bash
~$ cd TF2-Image-Generation
~$ {something to do}

otherwise, run command below after modifying docker-compose.yaml
~$ docker-compose up

Dataset Extractor

• [x] mnist: HomePage
• [x] CIFAR-10: HomePage
• [ ] CelebA: HomePage
• LSUN: HomePage
• [x] WikiArt: HomePage
  This dataset has broken files.
  Should run datasets/wikiart.py to remove the broken files after unzipping.

GIF Maker

Make gif file from image files.

usage: make_gif.py [-h] -i INPUT [-o OUTPUT] [-f FPS] [-r RESOLUTION]
                   [-fc FRAMES_CONSECUTIVE | -fsr FRAMES_SPACE_RATE | -fi FRAMES_INTERVAL]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        Input images directory
  -o OUTPUT, --output OUTPUT
                        Output file name
  -f FPS, --fps FPS     Frames per Second
  -r RESOLUTION, --resolution RESOLUTION
                        Output file resolution
  -fc FRAMES_CONSECUTIVE, --frames_consecutive FRAMES_CONSECUTIVE
                        Total consecutive frames of gif counting from scratch
  -fsr FRAMES_SPACE_RATE, --frames_space_rate FRAMES_SPACE_RATE
                        Rate of total frames from start to end (if 0.5, use half of frames)
  -fi FRAMES_INTERVAL, --frames_interval FRAMES_INTERVAL
                        Interval index between adjacent frames (if 10, images=[0, 10, 20, ...])

GIF maker uses following options.
If you run it for the first time, you need to run imageio.plugins.freeimage.download() first.
(or automatically download in the runtime)

• library: imageio
• plugin: FreeImage
• format: GIF-FI
• quantizer: nq (neuqant) - Dekker A. H., Kohonen neural networks for optimal color quantization

Patch Selector

Interactive patch selector.
Select exact patches with numpy indexing from images separated by (ROW x COL) sections.

usage: select_patch.py [-h] -i INPUT -r ROW -c COL [-o OUTPUT] [-n N_TARGET] [-as AUTO_SQUARE]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        Input images directory
  -r ROW, --row ROW     Number of rows in input images
  -c COL, --col COL     Number of columns in input images
  -o OUTPUT, --output OUTPUT
                        Output directory name (default=./output/patches)
  -n N_TARGET, --n_target N_TARGET
                        Target number of patches in output
  -as AUTO_SQUARE, --auto_square AUTO_SQUARE
                        Flag. Make Selected Patches to almost square

Tensorflow Log Extractor

Extract scalars(to csv) and images(to png) from tensorflow log file.
The extractor is 20~30 times slower than downloading from the Tensorboard GUI.
Don't use this extractor if saved the images at training time.
If need only csv file of scalars log, just download in Tensorboard GUI.

usage: extract_tf_log.py [-h] [-l LOG_DIR] [-o OUTPUT] [-ei EXTRACT_IMAGE]

optional arguments:
  -h, --help            show this help message and exit
  -l LOG_DIR, --log_dir LOG_DIR
                        Event log files directory, Select exact log in runtime (default=./**/checkpoints)
  -o OUTPUT, --output OUTPUT
                        Output directory (default=./log_output)
  -ei EXTRACT_IMAGE, --extract_image EXTRACT_IMAGE
                        Extract Image Flag (default=True)

Requirements

• tensorflow 2.x
• pip install -r requirements.txt