[ECCV'22] Language-Driven Artistic Style Transfer

A PyTorch implementation of LDAST

Paper | Project | Slide | Video

Overview

LDAST is an implementation of
"Language-Driven Artistic Style Transfer"
Tsu-Jui Fu, Xin Eric Wang, and William Yang Wang
in European Conference on Computer Vision (ECCV) 2022

Language visual artist (LVA) extracts content structures from <I>C</I> and visual patterns from <I>X</I> to perform LDAST. LVA adopts the patch-wise style discriminator <I>D</I> to connect extracted visual semantics to patches of paired style image (<I>P_S</I>). Contrastive reasoning (CR) allows comparing contrastive pairs <I>C₁-X₁</I>, <I>C₂-X₁</I>, and <I>C₂-X₂</I> of content image and style instruction.

Requirements

This code is implemented under Python 3.8, PyTorch 1.7, and Torchvision 0.8.

• tqdm
• CLIP

Usage

Dataset

The dataset includes content images and visual attribute instructions (DTD).
Please visit WikiArt and here for emotional effect instructions (ArtEmis).

Train

Put sanet.pt in ./_ckpt and dtd.pkl in ./_data.

python train_lva.py
python train_ctr.py

Inference & GUI

Put clva_dtd.pt in ./_ckpt.

python inference.py
python gui.py

Citation

@inproceedings{fu2022ldast, 
  author = {Tsu-Jui Fu and Xin Eric Wang and William Yang Wang}, 
  title = {{Language-Driven Artistic Style Transfer}}, 
  booktitle = {European Conference on Computer Vision (ECCV)}, 
  year = {2022} 
}

Acknowledgement

This code is based on SANet