Semantic Image Synthesis with SPADE (GauGAN) - Tensorflow

Simple Tensorflow implementation of "Semantic Image Synthesis with Spatially-Adaptive Normalization" (CVPR 2019 Oral)

Project page | Paper | Pytorch code

Requirements

• scipy == 1.2.0
  • The latest version is not available. imsave is deprecated.
• tqdm
• numpy
• pillow
• opencv-python
• tensorflow-gpu
• keras

Preparation

• YOUR DATASET
  • Image
  • Segmentation map
    • Don't worry. I do one-hot encoding of segmentation map automatically (whether color or gray)
• CelebAMask-HQ
  • Download from here

Pretrained model

• Download checkpoint
  • CelebAMask-HQ, hinge loss
  • It is a better performance than the results in the READEME
  • segmap_label.txt for CelebAMask-HQ
    • {(0, 0, 0): 0, (0, 0, 255): 1, (255, 0, 0): 2, (150, 30, 150): 3, (255, 65, 255): 4, (150, 80, 0): 5, (170, 120, 65): 6, (125, 125, 125): 7, (255, 255, 0): 8, (0, 255, 255): 9, (255, 150, 0): 10, (255, 225, 120): 11, (255, 125, 125): 12, (200, 100, 100): 13, (0, 255, 0): 14, (0, 150, 80): 15, (215, 175, 125): 16, (220, 180, 210): 17, (125, 125, 255): 18}

Usage

├── dataset
   └── YOUR_DATASET_NAME
       ├── image
           ├── 000001.jpg 
           ├── 000002.png
           └── ...
       ├── segmap
           ├── 000001.jpg
           ├── 000002.png
           └── ...
       ├── segmap_test
           ├── a.jpg 
           ├── b.png
           └── ...
       ├── segmap_label.txt (Automatically created) 
       
├── guide.jpg (example for guided image translation task)

Train

> python main.py --dataset spade_celebA --img_ch 3 --segmap_ch 3 --phase train 

Random test

> python main.py --dataset spade_celebA --segmap_ch 3 --phase random

Guide test

> python main.py --dataset spade_celebA --img_ch 3 --segmap_ch 3 --phase guide --guide_img ./guide_img.png

Our Results

Loss grpah

CityScape

CelebA-HQ (Style Manipulation)

CelebA-HQ (Random Manipulation)

How about the Least-Square loss ?

CelebA-HQ (Style Manipulation)

CelebA-HQ (Random Manipulation)

Architecture

Generator	Image Encoder	Discriminator	All-in-one

SPADE architecture

SPADE	SPADE Residual Block

Author

Junho Kim