ArcaneAnimeGAN

AnimeGAN2 trained on Arcane
trying to follow bryandlee animegan2 training methodology

Result

Training Workflow

• Get video data
• Split video into frames
• Align frame image using face-alignment
• Filter blurry image using opencv Laplacian
• Zip image dataset for fit into styleGAN
• Finetune FFHQ pretrained styleGAN using created zip dataset
• Blend findtuned styleGAN weight and pretrained styleGAN weight
• Create data pair using blended styleGAN model and pretrained model
• Train animeGAN using paired data

Change log

• 0.2
  • use anime-face-detector
  • add color correction on data preprocessing
  • add aug_transforms() on batch transform
  • use l1_loss(vgg(g(x)), vgg(y)) and mse_loss(g(x), y) instead of vgg feature loss and gram matrix loss
• 0.1
  • first release

To Do

• Use animegan vgg19[0,255] instead of vgg19_bn[0,1]
• Add canny edge method to gaussigan blur
• Background segmentation

Required environment to run

!conda install pytorch torchvision cudatoolkit=11.1 -c pytorch -c nvidia -y
!sudo apt install ffmpeg
!pip install face-alignment
!pip install --upgrade psutil
!pip install kornia
!pip install fastai==2.5.3
!pip install opencv-python
!git clone https://github.com/NVlabs/stylegan3.git

!pip install openmim
!mim install mmcv-full mmdet mmpose -y
!pip install anime-face-detector --no-dependencies

Acknowledgement and References

• AnimeGAN
• AnimeGANv2
• animegan2-pytorch
• animegan2-pytorch-Face-Portrait-v1
• pytorch-animeGAN
• AnimeGANv2_pytorch
• AnimeGAN_in_Pytorch
• AnimeGAN-torch
• style_transfer_implementation
• Anime-Sketch-Coloring
• CartoonGAN-Tensorflow
• cartoon-gan
• pytorch-implementation-of-perceptual-losses
• Artistic-Style-Transfer
• animegan2-pytorch-arcane
• DeepStudio
• ArcaneGAN
• stylegan3_blending
• toonify
• BlendGAN
• Cartoon-StyleGAN
• FFHQ-Alignment
• FFHQ-dataset
• face-alignment
• stylegan3
• FreezeD
• Image-Blur-Detection
• Classifying_image_blur_nonblur
• arcane
• JoJoGAN
• anime-face-detector
• color correction
• White-box-Cartoonization