Deep_Generative_Models icon indicating copy to clipboard operation
Deep_Generative_Models copied to clipboard

Published 20 hours ago • verified publisher icon PeterouZh
→

Metadata

A collection of papers I am interested in.

GAN-Inversion

A collection of papers I am interested in.

Awesome

  • https://ait.ethz.ch/index.php
  • https://liuyebin.com/student.html
  • https://virtualhumans.mpi-inf.mpg.de/
  • https://ps.is.mpg.de/publications
  • https://www.mpi-inf.mpg.de/departments/visual-computing-and-artificial-intelligence/publications
  • https://ait.ethz.ch/people/hilliges/
  • https://vlg.inf.ethz.ch/publications.html

Renderer

  • https://github.com/eth-ait/aitviewer
  • https://github.com/mitsuba-renderer/mitsuba3
  • https://github.com/angeloskath/simple-3dviz
  • https://github.com/BachiLi/redner

Pybind

  • https://github.com/pybind/cmake_example

Video

  • https://github.com/mli/autocut

Project

Face

3D

Tools

GUI

  • https://github.com/gradio-app/gradio

StyleGAN

  • https://github.com/justinpinkney/awesome-pretrained-stylegan2
  • https://github.com/justinpinkney/awesome-pretrained-stylegan3
  • generative-evaluation-prdc

Style transfer

Art

  • https://github.com/fogleman/primitive

Anime

  • https://github.com/TachibanaYoshino/AnimeGAN
  • https://github.com/TachibanaYoshino/AnimeGANv2

TOC

  • To be read
  • Disentanglement
  • Inversion
  • Encoder
  • Survey
  • GANs
  • Style transfer
  • Metric
  • Spectrum
  • Weakly Supervised Object Localization
  • NeRF
  • 3D

arXiv

Title Venue Code Year
Perceptual Gradient Networks arXiv:2105.01957 [cs] 2021
InfinityGAN: Towards Infinite-Resolution Image Synthesis arXiv:2104.03963 [cs] 2021
Aliasing Is Your Ally: End-to-End Super-Resolution from Raw Image Bursts arXiv:2104.06191 [cs, eess] 2021
StylePeople: A Generative Model of Fullbody Human Avatars arXiv:2104.08363 [cs] 2021
Cross-Domain and Disentangled Face Manipulation with 3D Guidance arXiv:2104.11228 [cs] 2021
On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation arXiv:2104.11222 [cs] 2021
FDA: Fourier Domain Adaptation for Semantic Segmentation arXiv:2004.05498 [cs] github 2020
StyleMapGAN: Exploiting Spatial Dimensions of Latent in GAN for Real-Time Image Editing CVPR 2021
Learning a Deep Reinforcement Learning Policy Over the Latent Space of a Pre-Trained GAN for Semantic Age Manipulation arXiv:2011.00954 [cs] 2021
GANalyze: Toward Visual Definitions of Cognitive Image Properties arXiv:1906.10112 [cs] 2019
On the “Steerability” of Generative Adversarial Networks arXiv:1907.07171 [cs] 2020
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation arXiv:2104.11116 [cs, eess] 2021
Unsupervised Image-to-Image Translation via Pre-Trained StyleGAN2 Network arXiv:2010.05713 [cs] github 2020
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort arXiv:2104.06490 [cs] 2021
Anycost GANs for Interactive Image Synthesis and Editing CVPR 2021
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization arXiv:2104.05833 [cs] 2021
Positional Encoding as Spatial Inductive Bias in GANs arXiv:2012.05217 [cs] 2020
An Empirical Study of the Effects of Sample-Mixing Methods for Efficient Training of Generative Adversarial Networks arXiv:2104.03535 [cs.CV] 2021
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks CVPRW github 2020
Image Demoireing with Learnable Bandpass Filters arXiv:2004.00406 [cs] 2020
Unveiling the Potential of Structure Preserving for Weakly Supervised Object Localization arXiv:2103.04523 [cs] 2021
LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions arXiv:2104.00820 [cs] 2021
Generating Images with Sparse Representations arXiv:2103.03841 [cs, stat] 2021
PiCIE: Unsupervised Semantic Segmentation Using Invariance and Equivariance in Clustering CVPR 2021
Dual Contrastive Loss and Attention for GANs arXiv:2103.16748 [cs.CV] 2021
Unsupervised Disentanglement of Linear-Encoded Facial Semantics CVPR 2021
Emergence of Object Segmentation in Perturbed Generative Models arXiv:1905.12663 [cs] github 2019
Unsupervised Discovery of DisentangledManifolds in GANs arXiv:2011.11842 [cs] github 2020
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery arXiv:2103.17249 [cs] github 2021
Few-Shot Semantic Image Synthesis Using StyleGAN Prior arXiv:2103.14877 [cs] 2021

Disentanglement

Title Venue Code Year
GANSpace: Discovering Interpretable GAN Controls arXiv:2004.02546 [cs] GANSpace 2020
Interpreting the Latent Space of GANs for Semantic Face Editing CVPR InterFaceGAN 2020
Closed-Form Factorization of Latent Semantics in GANs arXiv:2007.06600 [cs] sefa 2020
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation arXiv:2011.12799 [cs] StyleSpace 2020
Unsupervised Image Transformation Learning via Generative Adversarial Networks arXiv:2103.07751 [cs] github 2021
Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains arXiv:2010.05334 [cs] toonify 2020
WarpedGANSpace: Finding Non-Linear RBF Paths in GAN Latent Space arXiv:2109.13357 [cs] 2021
[Discovering Interpretable Latent Space Directions of GANs beyond Binary Attributes] CVPR 2021

Semantic hierarchy

Title Venue Code Year
Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis arXiv:1911.09267 [cs] 2020

Inversion

Optimization

Title Venue Code Year
Image2StyleGAN++: How to Edit the Embedded Images? arXiv:1911.11544 [cs] 2020
Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? ICCV 2019
Inverting The Generator Of A Generative Adversarial Network arXiv:1611.05644 [cs] 2016
Feature-Based Metrics for Exploring the Latent Space of Generative Models ICLRW 2018
Understanding Deep Image Representations by Inverting Them CVPR 2015
Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion arXiv:1912.08795 [cs, stat] DeepInversion 2020
IMAGINE: Image Synthesis by Image-Guided Model Inversion arXiv:2104.05895 [cs] 2021
Image Processing Using Multi-Code GAN Prior CVPR mGANprior 2020
Generative Visual Manipulation on the Natural Image Manifold ECCV 2018
GAN Dissection: Visualizing and Understanding Generative Adversarial Networks arXiv:1811.10597 [cs] 2018
GAN-Based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems arXiv:1902.09698 [cs, eess, stat] 2019
Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models CVPR 2020
Rewriting a Deep Generative Model arXiv:2007.15646 [cs] 2020
Transforming and Projecting Images into Class-Conditional Generative Networks arXiv:2005.01703 [cs] 2020
StyleGAN2 Distillation for Feed-Forward Image Manipulation arXiv:2003.03581 [cs.CV] 2020
On the “Steerability” of Generative Adversarial Networks arXiv:1907.07171 [cs] 2020
Unsupervised Discovery of DisentangledManifolds in GANs arXiv:2011.11842 [cs] 2020
PIE: Portrait Image Embedding for Semantic Control arXiv:2009.09485 [cs] 2020
GANSpace: Discovering Interpretable GAN Controls NeurIPS 2020
When and How Can Deep Generative Models Be Inverted? arXiv:2006.15555 [cs, stat] 2020
Style Intervention: How to Achieve Spatial Disentanglement with Style-Based Generators? arXiv:2011.09699 [cs] 2020
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation arXiv:2011.12799 [cs] 2020
Navigating the GAN Parameter Space for Semantic Image Editing arXiv:2011.13786 [cs] 2021
Mask-Guided Discovery of Semantic Manifolds in Generative Models arXiv:2105.07273 [cs] masked-gan-manifold 2021
StyleFlow: Attribute-Conditioned Exploration of StyleGAN-Generated Images Using Conditional Continuous Normalizing Flows arXiv:2008.02401 [cs] StyleFlow 2020
Disentangled Face Attribute Editing via Instance-Aware Latent Space Search arXiv:2105.12660 [cs] 2021
Barbershop: GAN-Based Image Compositing Using Segmentation Masks arXiv:2106.01505 [cs] 2021
Unsupervised Discovery of Interpretable Directions in the GAN Latent Space arXiv:2002.03754 [cs, stat] GANLatentDiscovery 2020
Pivotal Tuning for Latent-Based Editing of Real Images arXiv:2106.05744 [cs] PTI 2021
Editing in Style: Uncovering the Local Semantics of GANs CVPR 2020
Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval arXiv:2107.06256 [cs] RetrieveInStyle 2021
StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation arXiv:2107.04331 [cs] 2021
A Simple Baseline for StyleGAN Inversion arXiv:2104.07661 [cs] 2021
From Continuity to Editability: Inverting GANs with Consecutive Images arXiv:2107.13812 [cs] 2021
AgileGAN: Stylizing Portraits by Inversion-Consistent Transfer Learning ACM Transactions on Graphics (Proc. SIGGRAPH) 2021
Talk-to-Edit: Fine-Grained Facial Editing via Dialog ICCV Talk-to-Edit 2021
Improved StyleGAN Embedding: Where Are the Good Latents? arXiv:2012.09036 [cs] II2S 2021
EditGAN: High-Precision Semantic Image Editing editGAN_release 2021
Grasping the Arrow of Time from the Singularity: Decoding Micromotion in Low-Dimensional Latent Spaces from StyleGAN arXiv:2204.12696 [cs] 2022
Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing CVPR sam_inversion arXiv. 2022
Real Image Inversion via Segments arXiv:2110.06269 Chunkmogrify 2021

Encoder

Title Venue Code Year
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution arXiv:2012.00739 [cs] GLEAN 2020
Swapping Autoencoder for Deep Image Manipulation arXiv:2007.00653 [cs] github 2020
In-Domain GAN Inversion for Real Image Editing ECCV 2020
ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement arXiv:2104.02699 [cs] ReStyle 2021
Interpreting the Latent Space of GANs for Semantic Face Editing CVPR 2020
Face Identity Disentanglement via Latent Space Mapping arXiv:2005.07728 [cs] 2020
Collaborative Learning for Faster StyleGAN Embedding arXiv:2007.01758 [cs] 2020
Unsupervised Discovery of DisentangledManifolds in GANs arXiv:2011.11842 [cs] 2020
Generative Hierarchical Features from Synthesizing Images arXiv:2007.10379 [cs] 2020
One Shot Face Swapping on Megapixels arXiv:2105.04932 [cs] 2021
GAN Prior Embedded Network for Blind Face Restoration in the Wild 2021
Adversarial Latent Autoencoders CVPR ALAE 2020
Encoding in Style: A StyleGAN Encoder for Image-to-Image Translation arXiv:2008.00951 [cs] psp 2021
Designing an Encoder for StyleGAN Image Manipulation arXiv:2102.02766 [cs] encoder4editing 2021
A Latent Transformer for Disentangled and Identity-Preserving Face Editing arXiv:2106.11895 [cs] 2021
ShapeEditer: A StyleGAN Encoder for Face Swapping arXiv:2106.13984 [cs] 2021
Force-in-Domain GAN Inversion arXiv:2107.06050 [cs, eess] 2021
StyleFusion: A Generative Model for Disentangling Spatial Segments arXiv:2107.07437 [cs] 2021
Perceptually Validated Precise Local Editing for Facial Action Units with StyleGAN arXiv:2107.12143 [cs] 2021
StyleGAN2 Distillation for Feed-Forward Image Manipulation arXiv:2003.03581 [cs.CV] 2020
GAN Inversion for Out-of-Range Images with Geometric Transformations ICCV 2021
:heart: DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing arXiv:2109.10737 [cs] DyStyle 2021
High-Fidelity GAN Inversion for Image Attribute Editing arXiv:2109.06590 [cs] 2021
:heart: Few-Shot Knowledge Transfer for Fine-Grained Cartoon Face Generation arXiv:2007.13332 [cs] 2020
:heart: HyperInverter: Improving StyleGAN Inversion via Hypernetwork CVPR HyperInverter arXiv. 2022
[High-Fidelity GAN Inversion with Padding Space] ECCV padinv 2022

Hybrid optimization

Title Venue Code Year
Generative Visual Manipulation on the Natural Image Manifold ECCV 2018
Semantic Photo Manipulation with a Generative Image Prior ACM Transactions on Graphics 2019
Seeing What a GAN Cannot Generate arXiv:1910.11626 [cs, eess] 2019
In-Domain GAN Inversion for Real Image Editing ECCV 2020

Without optimization

Title Venue Code Year
Closed-Form Factorization of Latent Semantics in GANs arXiv:2007.06600 [cs] 2020
GAN “Steerability” without Optimization arXiv:2012.05328 [cs] 2021
Low-Rank Subspaces in GANs arXiv:2106.04488 [cs] 2021
LARGE: Latent-Based Regression through GAN Semantics arXiv:2107.11186 [cs] 2021
Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation ICCV 2021
Controllable and Compositional Generation with Latent-Space Energy-Based Models NeurIPS LACE 2021
Do Generative Models Know Disentanglement? Contrastive Learning Is All You Need arXiv:2102.10543 [cs] DisCo 2021

DGP

Title Venue Code Year
:heavy_check_mark: Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation ECCV DGP 2020
:heavy_check_mark: PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models CVPR PULSE 2020
:heavy_check_mark: GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution arXiv:2012.00739 [cs] 2020
Unsupervised Portrait Shadow Removal via Generative Priors arXiv:2108.03466 [cs] 2021
Towards Real-World Blind Face Restoration with Generative Facial Prior CVPR GFPGAN 2021
Towards Vivid and Diverse Image Colorization with Generative Color Prior ICCV 2021
Self-Validation: Early Stopping for Single-Instance Deep Generative Priors arXiv:2110.12271 [cs.CV] 2021
One-Shot Generative Domain Adaptation arXiv:2111.09876 [cs] 2021
:heart: Time-Travel Rephotography ACM Transactions on Graphics code 2021

Cls

Title Venue Code Year
Contrastive Model Inversion for Data-Free Knowledge Distillation arXiv:2105.08584 [cs] 2021
Generative Models as a Data Source for Multiview Representation Learning arXiv:2106.05258 [cs] 2021
Inverting and Understanding Object Detectors arXiv:2106.13933 [cs] 2021
Deep Neural Networks Are Surprisingly Reversible: A Baseline for Zero-Shot Inversion arXiv:2107.06304 [cs] 2021
Ensembling with Deep Generative Views arXiv:2104.14551 [cs] 2021

Change pose implicitly

Title Venue Code Year
On the “Steerability” of Generative Adversarial Networks arXiv:1907.07171 [cs] 2020
Interpreting the Latent Space of GANs for Semantic Face Editing CVPR 2020
GANSpace: Discovering Interpretable GAN Controls arXiv:2004.02546 [cs] GANSpace 2020
Closed-Form Factorization of Latent Semantics in GANs arXiv:2007.06600 [cs] sefa 2020
StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGAN arXiv:2111.01619 [cs] 2021
Using Latent Space Regression to Analyze and Leverage Compositionality in GANs ICLR 2021

Survey

Title Venue Code Year
GAN Inversion: A Survey arXiv:2101.05278 [cs] 2021

GANs

NeurIPS 2021

Title Venue Code Year
Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training NeurIPS 2021

Theory

Title Venue Code Year
:white_check_mark: Towards a Better Global Loss Landscape of GANs NeurIPS 2020
On the Benefit of Width for Neural Networks: Disappearance of Bad Basins arXiv:1812.11039 [cs, math, stat] 2021

Regs

Title Venue Code Year
The Hessian Penalty: A Weak Prior for Unsupervised Disentanglement ECCV 2020

Detection

Title Venue Code Year
Self-Supervised Object Detection via Generative Image Synthesis arXiv:2110.09848 [cs] 2021

StyleGANs

Title Venue Code Year
A Style-Based Generator Architecture for Generative Adversarial Networks CVPR 2019
Analyzing and Improving the Image Quality of StyleGAN arXiv:1912.04958 [cs, eess, stat] 2019
Training Generative Adversarial Networks with Limited Data arXiv:2006.06676 [cs, stat] 2020
Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data NeurIPS 2021
Alias-Free Generative Adversarial Networks arXiv:2106.12423 [cs, stat] alias-free-gan, rep2 2021
Transforming the Latent Space of StyleGAN for Real Face Editing arXiv:2105.14230 [cs] TransStyleGAN 2021
MobileStyleGAN: A Lightweight Convolutional Neural Network for High-Fidelity Image Synthesis arXiv:2104.04767 [cs, eess] MobileStyleGAN 2021
Few-Shot Image Generation via Cross-Domain Correspondence CVPR few-shot-gan-adaptation 2021
EigenGAN: Layer-Wise Eigen-Learning for GANs arXiv:2104.12476 [cs, stat] EigenGAN 2021
:heart: Toward Spatially Unbiased Generative Models ICCV toward_spatial_unbiased 2021
Interpreting Generative Adversarial Networks for Interactive Image Generation arXiv:2108.04896 [cs] 2021
Explaining in Style: Training a GAN to Explain a Classifier in StyleSpace ICCV explaining-in-style 2021
:white_check_mark: Projected GANs Converge Faster NeurIPS projected_gan 2021
:white_check_mark: Towards Faster and Stabilized GAN Training for High-Fidelity Few-Shot Image Synthesis ICLR2021 github 2021
:heart: Ensembling Off-the-Shelf Models for GAN Training arXiv:2112.09130 [cs] vision-aided-gan 2021
:heart: StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets arXiv:2202.00273 [cs] 2022
When, Why, and Which Pretrained GANs Are Useful? ICLR 2022
A U-Net Based Discriminator for Generative Adversarial Networks CVPR 2020

Transformer

Title Venue Code Year
Compositional Transformers for Scene Generation NeurIPS 2021
:heart: GAN-Supervised Dense Visual Alignment arXiv:2112.05143 [cs] gangealing 2021
Improved Transformer for High-Resolution GANs arXiv:2106.07631 [cs] 2021
MaskGIT: Masked Generative Image Transformer arXiv:2202.04200 [cs] 2022
StyleSwin: Transformer-Based GAN for High-Resolution Image Generation CVPR 2022

SinGAN

Title Venue Code Year
ExSinGAN: Learning an Explainable Generative Model from a Single Image arXiv:2105.07350 [cs] 2021

Video

Title Venue Code Year
:heart: Diverse Generation from a Single Video Made Possible arXiv:2109.08591 [cs] 2021

GANs

Title Venue Code Year
:white_check_mark: Differentiable Augmentation for Data-Efficient GAN Training arXiv:2006.10738 [cs] 2020
Sampling Generative Networks arXiv:1609.04468 [cs, stat] 2016
Combining Transformer Generators with Convolutional Discriminators arXiv:2105.10189 [cs] 2021
Improving Generation and Evaluation of Visual Stories via Semantic Consistency arXiv:2105.10026 [cs] 2021
TediGAN: Text-Guided Diverse Face Image Generation and Manipulation CVPR 2021
Data-Efficient Instance Generation from Instance Discrimination arXiv:2106.04566 [cs] 2021
Styleformer: Transformer Based Generative Adversarial Networks with Style Vector arXiv:2106.07023 [cs, eess] 2021
FBC-GAN: Diverse and Flexible Image Synthesis via Foreground-Background Composition arXiv:2107.03166 [cs] 2021
ViTGAN: Training GANs with Vision Transformers arXiv:2107.04589 [cs, eess] 2021
Learning Efficient GANs for Image Translation via Differentiable Masks and Co-Attention Distillation arXiv:2011.08382 [cs] 2021
CGANs with Auxiliary Discriminative Classifier arXiv:2107.10060 [cs] 2021
A Good Image Generator Is What You Need for High-Resolution Video Synthesis ICLR 2021
Dual Projection Generative Adversarial Networks for Conditional Image Generation ICCV 2021
Your GAN Is Secretly an Energy-Based Model and You Should Use Discriminator Driven Latent Sampling arXiv:2003.06060 [cs, stat] CGAN-DDLS 2021
Manifold-Preserved GANs arXiv:2109.08955 [cs] 2021
Latent Reweighting, an Almost Free Improvement for GANs arXiv:2110.09803 [cs] 2021
STRANSGAN: AN EMPIRICAL STUDY ON TRANS- FORMER IN GANS arXiv:2110.13107 [cs.CV] 2021
Self-Supervised GANs with Label Augmentation arXiv:2106.08601 [cs] 2021
Regularizing Generative Adversarial Networks under Limited Data CVPR github 2021

cGANs

Title Venue Code Year
Unbiased Auxiliary Classifier GANs with MINE arXiv:2006.07567 [cs] 2020
Twin Auxiliary Classifiers GAN arXiv:1907.02690 [cs, stat] 2019

Finetune

Title Venue Code Year
FreezeG github
:white_check_mark: Freeze the Discriminator: A Simple Baseline for Fine-Tuning GANs arXiv:2002.10964 [cs, stat] FreezeD 2020
Fine-Tuning StyleGAN2 For Cartoon Face Generation arXiv:2106.12445 [cs, eess] Cartoon-StyleGAN 2021
Transferring GANs: Generating Images from Limited Data ECCV 2018
Image Generation From Small Datasets via Batch Statistics Adaptation ICCV 2019
MineGAN: Effective Knowledge Transfer From GANs to Target Domains With Few Images CVPR 2020

Compression

Title Venue Code Year
GAN Compression: Efficient Architectures for Interactive Conditional GANs CVPR 2020
Online Multi-Granularity Distillation for GAN Compression ICCV 2021
Revisiting Discriminator in GAN Compression: A Generator-Discriminator Cooperative Compression Scheme arXiv:2110.14439 [cs] GCC 2021

Detection fake

Title Venue Code Year
Robust Attentive Deep Neural Network for Exposing GAN-Generated Faces arXiv:2109.02167 [cs] 2021

Segmentation

Title Venue Code Year
Labels4Free: Unsupervised Segmentation Using StyleGAN arXiv:2103.14968 [cs] 2021
BigDatasetGAN: Synthesizing ImageNet with Pixel-Wise Annotations ArXiv:2201.04684 [Cs] arXiv. 2022

Datasets

Title Venue Code Year
Gradient-Based Learning Applied to Document Recognition Proceedings of the IEEE [mnist] 1998
Learning Multiple Layers of Features from Tiny Images [cifar] 2009
ImageNet: A Large-Scale Hierarchical Image Database CVPR [ImageNet] 2009
Learning Hybrid Image Templates (HIT) by Information Projection TPAMI AnimalFace 2012
A Style-Based Generator Architecture for Generative Adversarial Networks CVPR FFHQ 2019
StarGAN v2: Diverse Image Synthesis for Multiple Domains CVPR AFHQ 2020
Automated Flower Classification over a Large Number of Classes 102Flowers 2008
XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings ICML CartoonSet 2018
Anime Faces Sourced from Safebooru Resized to 256x256 Kaggle AnimeFace
Facial Expressions of Manga (Japanese Comic) Character Faces Kaggle MangaExpressions
Open-Source Cartoon Dataset Kaggle photo2cartoon
Simpsons Faces: A Lot of Images of Your Favourite Characters Kaggle SimpsonsFaces
Bitmoji Faces Kaggle BitmojiFaces
BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generation AAHQ 2021
:heart: Fake It Till You Make It: Face Analysis in the Wild Using Synthetic Data Alone ICCV FaceSynthetics 2021
Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models CVPR chair 2014
A Large-Scale Car Dataset for Fine-Grained Categorization and Verification CVPR [CompCars] arXiv. 2015
The ArtBench Dataset: Benchmarking Generative Models with Artworks 2022
DwNet: Dense Warp-Based Network for Pose-Guided Human Video Generation BMVC Fashion 2019
MoCoGAN: Decomposing Motion and Content for Video Generation CVPR [Tai-Chi] 2018
Text2Human: Text-Driven Controllable Human Image Generation ACM Transactions on Graphics (TOG) DeepFashion-MultiModal 2022

alias (ref)

Title Venue Code Year
Alias-Free Generative Adversarial Networks arXiv:2106.12423 [cs, stat] 2021
On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation arXiv:2104.11222 [cs] 2021

Texture

  • https://github.com/carson-katri/dream-textures

Tiles

Title Venue Code Year
TileGAN: Synthesis of Large-Scale Non-Homogeneous Textures ACM Transactions on Graphics 2019
InsetGAN for Full-Body Image Generation arXiv:2203.07293 [cs] 2022
Collaging Class-Specific GANs for Semantic Image Synthesis ICCV 2021

GAN application

Title Venue Code Year
SC-FEGAN: Face Editing Generative Adversarial Network with User’s Sketch and Color arXiv:1902.06838 [cs] 2019
Semantic Text-to-Face GAN -ST^2FG arXiv:2107.10756 [cs] 2021
CRD-CGAN: Category-Consistent and Relativistic Constraints for Diverse Text-to-Image Generation arXiv:2107.13516 [cs] 2021

Image-to-Image Translation

Title Venue Code Year
Image-to-Image Translation with Conditional Adversarial Networks CVPR pix2pix 2017
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs CVPR pix2pix-HD 2018
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks ICCV CycleGAN 2017
StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation CVPR 2018
StarGAN v2: Diverse Image Synthesis for Multiple Domains CVPR 2020
Multimodal Unsupervised Image-to-Image Translation arXiv:1804.04732 [cs, stat] MUNIT 2018
High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network arXiv:2105.09188 [cs] 2021
MixerGAN: An MLP-Based Architecture for Unpaired Image-to-Image Translation arXiv:2105.14110 [cs] 2021
GANs N’ Roses: Stable, Controllable, Diverse Image to Image Translation (Works for Videos Too!) arXiv:2106.06561 [cs] 2021
:heart: Sketch Your Own GAN ICCV 2021
Contrastive Learning for Unpaired Image-to-Image Translation ECCV contrastive-unpaired-translation 2020
The Animation Transformer: Visual Correspondence via Segment Matching arXiv:2109.02614 [cs] 2021
Image Synthesis via Semantic Composition ICCV 2021
You Only Need Adversarial Supervision for Semantic Image Synthesis arXiv:2012.04781 [cs, eess] 2020

Style transfer

  • https://github.com/nrupatunga/L0-Smoothing
Title Venue Code Year
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization ICCV 2017
Texture Synthesis Using Convolutional Neural Networks NeurIPS 2015
A Neural Algorithm of Artistic Style arXiv:1508.06576 [cs, q-bio] 2015
Image Style Transfer Using Convolutional Neural Networks CVPR 2016
Perceptual Losses for Real-Time Style Transfer and Super-Resolution ECCV 2016
Texture Networks: Feed-Forward Synthesis of Textures and Stylized Images ICML 2016
Attention-Based Stylisation for Exemplar Image Colourisation arXiv:2105.01705 [cs, eess] 2021
StyleBank: An Explicit Representation for Neural Image Style Transfer Stylebank 2017
Rethinking and Improving the Robustness of Image Style Transfer arXiv:2104.05623 [cs, eess] 2021
Paint Transformer: Feed Forward Neural Painting with Stroke Prediction ICCV 2021
:heart: AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer ICCV 2021
ZiGAN: Fine-Grained Chinese Calligraphy Font Generation via a Few-Shot Style Transfer Approach arXiv:2108.03596 [cs] 2021
Domain-Aware Universal Style Transfer ICCV 2021
Aesthetics and Neural Network Image Representations arXiv:2109.08103 [cs, eess, q-bio] 2021
:heart: Collaborative Distillation for Ultra-Resolution Universal Style Transfer CVPR collaborative-distillation 2020
Adaptive Convolutions for Structure-Aware Style Transfer CVPR ada-conv-pytorch 2021
CCPL: Contrastive Coherence Preserving Loss for Versatile Style Transfer ECCV CCPL arXiv. 2022

Metric & perceptual loss

Title Venue Code Year
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric arXiv:1801.03924 [cs] lpips-pytorch 2018
Generating Images with Perceptual Similarity Metrics Based on Deep Networks NeurIPS Perceptual Similarity 2016
Generic Perceptual Loss for Modeling Structured Output Dependencies CVPR [random] 2021
Inverting Adversarially Robust Networks for Image Synthesis arXiv:2106.06927 [cs] 2021
Demystifying MMD GANs ICLR Kernel Inception Distance (KID) 2018
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium NeurIPS Fréchet Inception Distance (FID) 2017
Improved Techniques for Training GANs NeurIPS Inception Score (IS) 2016
High-Fidelity Performance Metrics for Generative Models in PyTorch torch-fidelity 2020
Reliable Fidelity and Diversity Metrics for Generative Models ICML generative-evaluation-prdc 2020
The Contextual Loss for Image Transformation with Non-Aligned Data ECCV contextualLoss arXiv. 2018
Maintaining Natural Image Statistics with the Contextual Loss ArXiv:1803.04626 [Cs] 2018

Spectrum

Title Venue Code Year
Reproducibility of "FDA: Fourier Domain Adaptation ForSemantic Segmentation arXiv:2104.14749 [cs] 2021
A Closer Look at Fourier Spectrum Discrepancies for CNN-Generated Images Detection CVPR 2021

Weakly Supervised Object Localization

Title Venue Code Year
TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization arXiv:2103.14862 [cs] 2021
Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models arXiv:2105.08127 [cs] 2021
Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP arXiv:2107.12518 [cs] 2021

Implicit Neural Representations

Title Venue Code Year
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation arXiv:1901.05103 [cs] 2019
Occupancy Networks: Learning 3D Reconstruction in Function Space arXiv:1812.03828 [cs] 2019
:heart: Neural Image Representations for Multi-Image Fusion and Layer Separation arXiv:2108.01199 [cs] 2021
Learning Continuous Image Representation with Local Implicit Image Function CVPR 2021

Energy

Title Venue Code Year
How to Train Your Energy-Based Models ArXiv:2101.03288 arXiv. 2021
Your Classifier Is Secretly an Energy Based Model and You Should Treat It Like One ICLR JEM arXiv. 2020
Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models NeurIPS Generative-Visual-Prompt 2022

Flow

Title Venue Code Year
Variational Inference with Normalizing Flows ICML 2015
Density Estimation Using Real NVP ICLR arXiv. 2017

ChatGPT

  • https://github.com/golfzert/chatgpt-chinese-prompt-hack
  • https://github.com/rawandahmad698/PyChatGPT

Diffusion

  • https://github.com/heejkoo/Awesome-Diffusion-Models
  • https://github.com/huggingface/diffusers
  • https://github.com/Jack000/glid-3-xl
  • https://github.com/SirWaffle/AIrtist-k-diffusion-wrap
  • https://github.com/altryne/awesome-ai-art-image-synthesis
  • https://github.com/YangLing0818/Diffusion-Models-Papers-Survey-Taxonomy
  • https://github.com/Jack000/glid-3-xl-stable
  • https://github.com/Stability-AI/stablediffusion

Generation

Title Venue Code Year
Understanding Diffusion Models: A Unified Perspective arXiv:2208.11970 2022
1 Deep Unsupervised Learning Using Nonequilibrium Thermodynamics ArXiv:1503.03585 [Cond-Mat, q-Bio, Stat] arXiv. 2015
2 Generative Modeling by Estimating Gradients of the Data Distribution NeurIPS 2019
3 Denoising Diffusion Probabilistic Models arXiv:2006.11239 [cs, stat] diffusion, denoising-diffusion-pytorch 2020
Denoising Diffusion Implicit Models ICLR [DDIM] arXiv. 2021
Improved Denoising Diffusion Probabilistic Models ArXiv:2102.09672 [Cs.LG] improved-diffusion 2021
Score-Based Generative Modeling through Stochastic Differential Equations ICLR 2021
35 steps Elucidating the Design Space of Diffusion-Based Generative Models ArXiv:2206.00364 [Cs, Stat] k-diffusion arXiv. 2022
10 steps DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps ArXiv:2206.00927 [Cs, Stat] dpm-solver arXiv. 2022
:heart: SDEdit: Image Synthesis and Editing with Stochastic Differential Equations arXiv:2108.01073 [cs] SDEdit 2021
D2C: Diffusion-Denoising Models for Few-Shot Conditional Generation arXiv:2106.06819 [cs] 2021
Label-Efficient Semantic Segmentation with Diffusion Models ddpm-segmentation 2021
Analog Bits: Generating Discrete Data Using Diffusion Models with Self-Conditioning bit-diffusion 2022
Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise arXiv:2208.09392 Cold-Diffusion-Models 2022
Diffusion-GAN: Training GANs with Diffusion arXiv:2206.02262 Diffusion-GAN 2022
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs ICLR denoising-diffusion-gan 2022
Score-Based Generative Modeling in Latent Space NeurIPS LSGM arXiv. 2021
Compositional Visual Generation with Composable Diffusion Models ECCV Composable-Diffusion arXiv. 2022
Accelerating Score-Based Generative Models with Preconditioned Diffusion Sampling ECCV PDS 2022
Diffusion Autoencoders: Toward a Meaningful and Decodable Representation CVPR diffae 2022
Cascaded Diffusion Models for High Fidelity Image Generation arXiv:2106.15282 2021

Inversion

Title Venue Code Year
ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models ICCV ilvr_adm arXiv. 2021
Diffusion Models Beat GANs on Image Synthesis arXiv:2105.05233 [cs, stat] guided-diffusion 2021
An Image Is Worth One Word: Personalizing Text-to-Image Generation Using Textual Inversion arXiv:2208.01618 textual_inversion 2022
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation arXiv:2208.12242 dreambooth, Dreambooth-Stable-Diffusion 2022
DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation CVPR DiffusionCLIP 2022

Text-to-image

  • https://github.com/GeeveGeorge/Stable-Craiyon
Title Venue Code Year
Cross-Modal Contrastive Learning for Text-to-Image Generation CVPR 2021
Zero-Shot Text-to-Image Generation ICML arXiv. 2021
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance ArXiv:2204.08583 2022
Learning Transferable Visual Models From Natural Language Supervision ICML CLIP open_clip 2021
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models arXiv:2112.10741 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents DALLE2-pytorch 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding imagen-pytorch, Imagen-pytorch 2022
[Scaling Autoregressive Models for Content-Rich Text-to-Image Generation] parti
CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers arXiv:2204.14217 2022
High-Resolution Image Synthesis with Latent Diffusion Models CVPR stable-diffusion, latent-diffusion, stable-diffusion arXiv. 2022
Prompt-to-Prompt Image Editing with Cross Attention Control arXiv:2208.01626 CrossAttentionControl 2022
SINE: SINgle Image Editing with Text-to-Image Diffusion Models arXiv:2212.04489 SINE 2022

Image_to_image

Title Venue Code Year
Palette: Image-to-Image Diffusion Models arXiv:2111.05826 Palette-Image-to-Image-Diffusion-Models 2022
Image Super-Resolution via Iterative Refinement arXiv:2104.07636 Image-Super-Resolution-via-Iterative-Refinement 2021

3D

  • https://github.com/neverix/pixel-dreamfusion
Title Venue Code Year
RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation arXiv:2211.09869 2022
Magic3D: High-Resolution Text-to-3D Content Creation arXiv:2211.10440 2022

Detection

Title Venue Code Year
DiffusionInst: Diffusion Model for Instance Segmentation arXiv:2212.02773 DiffusionInst 2022

3D & NeRF

  • https://www.meshlab.net/
Title Venue Code Year
Efficient Ray Tracing of Volume Data ACM Transactions on Graphics 1990
Surface Light Fields for 3D Photography SIGGRAPH 2000
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections arXiv:2008.02268 [cs] nerfw 2021
Modulated Periodic Activations for Generalizable Local Functional Representations arXiv:2104.03960 [cs] 2021
Neural Volume Rendering: NeRF And Beyond arXiv:2101.05204 [cs] awesome-NeRF 2021
Editing Conditional Radiance Fields arXiv:2105.06466 [cs] editnerf 2021
Recursive-NeRF: An Efficient and Dynamically Growing NeRF arXiv:2105.09103 [cs] 2021
MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo arXiv:2103.15595 [cs] mvsnerf 2021
Depth-Supervised NeRF: Fewer Views and Faster Training for Free arXiv:2107.02791 [cs] 2021
Rethinking Positional Encoding arXiv:2107.02561 [cs] 2021
Nerfies: Deformable Neural Radiance Fields arXiv:2011.12948 nerfies 2020
Self-Calibrating Neural Radiance Fields ICCV 2021
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering arXiv:2106.02634 [cs] 2021

Sine

Title Venue Code Year Cite
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains arXiv:2006.10739 [cs] 2020
:white_check_mark: Implicit Neural Representations with Periodic Activation Functions NeurIPS 2020
:white_check_mark: Modulated Periodic Activations for Generalizable Local Functional Representations arXiv:2104.03960 [cs] 2021
Learned Initializations for Optimizing Coordinate-Based Neural Representations arXiv:2012.02189 [cs] nerf-meta 2021
Seeing Implicit Neural Representations as Fourier Series arXiv:2109.00249 [cs] 2021

INR

Title Venue Code Year Cite
Adversarial Generation of Continuous Images arXiv:2011.12026 [cs] inr-gan 2020
Image Generators with Conditionally-Independent Pixel Synthesis arXiv:2011.13775 [cs] CIPS 2020
A Structured Dictionary Perspective on Implicit Neural Representations arXiv:2112.01917 [cs] 2021

3D & NeRF GANs

  • https://mrtornado24.github.io/Next3D/
Title Venue Code Year Cite
:heavy_check_mark: HoloGAN: Unsupervised Learning of 3D Representations from Natural Images ICCV 2019
BlockGAN: Learning 3D Object-Aware Scene Representations from Unlabelled Images NeurIPS 2020
:heavy_check_mark: GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis arXiv:2007.02442 [cs] 2021
:heavy_check_mark: Pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis arXiv:2012.00926 [cs] pi-GAN 2021 19
:heavy_check_mark: GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields CVPR giraffe 2021
:heavy_check_mark: GIRAFFE HD: A High-Resolution 3D-Aware Generative Model CVPR 2022
:heart: StyleNeRF: A Style-Based 3D-Aware Generator for High-Resolution Image Synthesis arXiv:2110.08985 [cs, stat] 2021
CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields arXiv:2103.17269 [cs] 2021
:heavy_check_mark: GNeRF: GAN-Based Neural Radiance Field without Posed Camera arXiv:2103.15606 [cs] gnerf 2021
:heart: Unconstrained Scene Generation with Locally Conditioned Radiance Fields ICCV ml-gsn 2021
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering ICCV 2021
:heavy_check_mark: A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis NeurIPS 2021
:heavy_check_mark: Generative Occupancy Fields for 3D Surface-Aware Image Synthesis NeurIPS 2021
:heavy_check_mark: Efficient Geometry-Aware 3D Generative Adversarial Networks arXiv:2112.07945 [cs] eg3d 2021
:heavy_check_mark: 3D-Aware Image Synthesis via Learning Structural and Textural Representations arXiv:2112.10759 [cs] VolumeGAN 2021
:heavy_check_mark: GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation arXiv:2112.08867 [cs] GRAM 2021
CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs CVPR 2022
:heavy_check_mark: Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images CVPR 2022
:heavy_check_mark: Multi-View Consistent Generative Adversarial Networks for 3D-Aware Image Synthesis CVPR MVCGAN 2022
:heavy_check_mark: FENeRF: Face Editing in Neural Radiance Fields CVPR FENeRF 2022
:heavy_check_mark: IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-Aware Portrait Synthesis arXiv:2205.15517 2022
:heavy_check_mark: EpiGRAF: Rethinking Training of 3D GANs ArXiv:2206.10535 [Cs] epigraf arXiv. 2022
https://github.com/rethinking-3d-gans/code
:heart: Generative Multiplane Images: Making a 2D GAN 3D-Aware ECCV ml-gmpi arXiv. 2022
GAUDI: A Neural Architect for Immersive 3D Scene Generation ArXiv:2207.13751 [Cs] ml-gaudi arXiv. 2022
Deep Deformable 3D Caricatures with Learned Shape Control SIGGRAPH DeepDeformable3DCaricatures Vancouver BC Canada: ACM. 2022
Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis ECCV SURF-GAN arXiv. 2022
Pix2NeRF: Unsupervised Conditional $\pi$-GAN for Single Image to Neural Radiance Fields Translation arXiv:2202.13162 [cs] 2022
[Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation] TT-GNeRF

Diffusion

Title Venue Code Year Cite
DiffuStereo: High Quality Human Reconstruction via Diffusion-Based Stereo Using Sparse Cameras ECCV DiffuStereo arXiv. 2022
DiffRF: Rendering-Guided 3D Radiance Field Diffusion arXiv:2212.01206 DiffRF 2022

NeRF large scene

Title Venue Code Year Cite
Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs CVPR 2022
Block-NeRF: Scalable Large Scene Neural View Synthesis CVPR BlockNeRFPytorch arXiv. 2022
IBRNet: Learning Multi-View Image-Based Rendering arXiv:2102.13090 [cs] IBRNet 2021

NeRF

  • https://github.com/kakaobrain/NeRF-Factory/ :heart:
  • https://github.com/openxrlab/xrnerf
  • https://github.com/ActiveVisionLab/nerfmm
  • https://github.com/ventusff/improved-nerfmm
  • https://github.com/Kai-46/nerfplusplus
  • https://github.com/kwea123/nerf_pl
  • https://github.com/NVlabs/instant-ngp
  • https://github.com/sxyu/nerfvis
  • https://github.com/frozoul/4K-NeRF
Title Venue Code Year Cite
Ray Tracing Volume Densities SIGGRAPH 1984
:white_check_mark: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis ECCV nerf-pytorch 2020
:white_check_mark: NeRF--: Neural Radiance Fields Without Known Camera Parameters arXiv:2102.07064 [cs] nerfmm, improved-nerfmm 2021
:white_check_mark: NeRF++: Analyzing and Improving Neural Radiance Fields arXiv:2010.07492 [cs] nerfplusplus 2020
:white_check_mark: FastNeRF: High-Fidelity Neural Rendering at 200FPS arXiv:2103.10380 [cs] 2021
:white_check_mark: KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs ICCV 2021
Plenoxels: Radiance Fields without Neural Networks arXiv:2112.05131 [cs] svox2 2021
Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs arXiv:2112.10703 [cs] mega-nerf 2021
:heart: Neural Sparse Voxel Fields arXiv:2007.11571 [cs] NSVF 2021
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields ICCV 2021
:heart: Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields arXiv:2111.12077 [cs.CV] 2021
:heart: Neural Actor: Neural Free-View Synthesis of Human Actors with Pose Control arXiv:2106.02019 [cs] 2022
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding instant-ngp
:heart: Point-NeRF: Point-Based Neural Radiance Fields arXiv:2201.08845 [cs] pointnerf 2022
MoFaNeRF: Morphable Facial Neural Radiance Field arXiv:2112.02308 [cs] 2021
Object-Centric Neural Scene Rendering 2020
Semantic View Synthesis 2020
NeRS: Neural Reflectance Surfaces for Sparse-View 3D Reconstruction in the Wild 2021
MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis arXiv:2103.14910 [cs] 2021
:white_check_mark: CodeNeRF: Disentangled Neural Radiance Fields for Object Categories ICCV code-nerf 2021
NeRF-SR: High-Quality Neural Radiance Fields Using Super-Sampling arXiv:2112.01759 [cs] 2021
:heart: TensoRF: Tensorial Radiance Fields arXiv:2203.09517 [cs] TensoRF 2022
Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fields arXiv:2203.10821 [cs] 2022
CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields arXiv:2112.05139 [cs] 2022
BARF: Bundle-Adjusting Neural Radiance Fields arXiv:2104.06405 [cs] 2021
Unified Implicit Neural Stylization arXiv:2204.01943 [cs] 2022
SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image arXiv:2204.00928 [cs] 2022
NeRF-Editing: Geometry Editing of Neural Radiance Fields CVPR NeRF-Editing 2022
PixelNeRF: Neural Radiance Fields from One or Few Images CVPR pixel-nerf 2021
Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields CVPR refnerf 2022

3D inversion

Title Venue Code Year Cite
Unsupervised 3D Shape Completion through GAN Inversion CVPR 2021
3D GAN Inversion for Controllable Portrait Image Animation ArXiv:2203.13441 [Cs] arXiv. 2022
Pix2NeRF: Unsupervised Conditional $\pi$-GAN for Single Image to Neural Radiance Fields Translation ArXiv:2202.13162 [Cs] arXiv. 2022
[Monocular 3D Object Reconstruction with GAN Inversion] ECCV 2022
[INeRF: Inverting Neural Radiance Fields for Pose Estimation] IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) inerf 2021
Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion arXiv:2211.11674 nerf-from-image 2022

Dynamic

Title Venue Code Year Cite
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes arXiv:2011.13084 [cs] Neural-Scene-Flow-Fields 2021
D-NeRF: Neural Radiance Fields for Dynamic Scenes arXiv:2011.13961 [cs] D-NeRF 2020
Dynamic View Synthesis from Dynamic Monocular Video arXiv:2105.06468 [cs] DynamicNeRF 2021
:heart: HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields arXiv:2106.13228 [cs] hypernerf 2021
Neural Radiance Flow for 4D View Synthesis and Video Processing 2020
:heart: Animatable Neural Implicit Surfaces for Creating Avatars from Videos arXiv:2203.08133 [cs] 2022

Voice

  • https://github.com/CorentinJ/Real-Time-Voice-Cloning

Hand

  • https://github.com/reyuwei/NIMBLE_model

Hair

  • https://github.com/clach/Realtime-Vulkan-Hair

Loose garment

  • https://cape.is.tue.mpg.de/dataset.html
Title Venue Code Year Cite
:heavy_check_mark: Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks SIGGRAPH VirtualBones 2022
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style CVPR TailorNet_dataset arXiv. 2020
Learning Implicit Templates for Point-Based Clothed Human Modeling ECCV 2022
3D Clothed Human Reconstruction in the Wild ECCV ClothWild_RELEASE 2022
:heart: TightCap: 3D Human Shape Capture with Clothing Tightness Field ACM Transactions on Graphics TightCap 2021
ARCH: Animatable Reconstruction of Clothed Humans CVPR ARCH 2020

Rigging

Title Venue Code Year Cite
:heart: [Learning Skeletal Articulations with Neural Blend Shapes] ACM Transactions on Graphics neural-blend-shapes 2021

Anime Body

Title Venue Code Year
Collaborative Neural Rendering Using Anime Character Sheets ArXiv:2207.05378 [Cs] CoNR arXiv. 2022

Body

  • https://github.com/3DFaceBody/awesome-3dbody-papers
  • https://github.com/openMVG/awesome_3DReconstruction_list
  • https://github.com/ytrock/THuman2.0-Dataset
  • https://github.com/Danial-Kord/DigiHuman
  • https://github.com/zhaofuq/Instant-NSR
Title Venue Code Year
SMPL: A Skinned Multi-Person Linear Model ACM Trans. Graphics (Proc. SIGGRAPH Asia) 2015
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image CVPR [SMPL-X] 2019
AMASS: Archive of Motion Capture as Surface Shapes ICCV AMASS 2019
:heavy_check_mark: SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes ICCV 2021
:heavy_check_mark: Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies ICCV animatable_nerf 2021
Neural Actor: Neural Free-View Synthesis of Human Actors with Pose Control SIGGRAPH Asia 2021
:heavy_check_mark: Animatable Neural Radiance Fields from Monocular RGB Videos ArXiv:2106.13629 [Cs] Anim-NeRF arXiv. 2021
VIBE: Video Inference for Human Body Pose and Shape Estimation CVPR VIBE arXiv. 2020
:heavy_check_mark: A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose NeurIPS A-NeRF arXiv. 2021
HumanNeRF: Free-Viewpoint Rendering of Moving People from Monocular Video CVPR humannerf 2022
:heart: The Power of Points for Modeling Humans in Clothing ICCV POP 2021
:heart: Neural Point-Based Shape Modeling of Humans in Challenging Clothing International Conference on 3D Vision (3DV) SkiRT 2022
StylePeople: A Generative Model of Fullbody Human Avatars arXiv:2104.08363 [cs] 2021
NPMs: Neural Parametric Models for 3D Deformable Shapes arXiv:2104.00702 [cs] 2021
:heart: ICON: Implicit Clothed Humans Obtained from Normals arXiv:2112.09127 [cs] ICON 2022
:heart: GDNA: Towards Generative Detailed Neural Avatars CVPR 2022
SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks CVPR 2021
NeuralAnnot: Neural Annotator for 3D Human Mesh Training Sets arXiv:2011.11232 [cs] 2022
:heart: PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop ICCV 2021
:heart: Structured Local Radiance Fields for Human Avatar Modeling CVPR arXiv. 2022
:heavy_check_mark: SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video CVPR SelfRecon arXiv. 2022
[ARAH: Animatable Volume Rendering of Articulated Human SDFs] ECCV arah 2022
Neural Actor: Neural Free-View Synthesis of Human Actors with Pose Control SIGGRAPH Asia Neural_Actor_Main_Code arXiv. 2021
:heart: Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis arXiv:2204.11798 [cs] gnr 2022
:heart: NeuMan: Neural Human Radiance Field from a Single Video ECCV ml-neuman 2022
Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis CVPR surface-aligned-nerf arXiv. 2022
LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling ECCV LoRD 2022
TAVA: Template-Free Animatable Volumetric Actors ECCV tava 2022
Fast-SNARF: A Fast Deformer for Articulated Neural Fields fast-snarf 2022
InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds arXiv:2212.10550 InstantAvatar 2022

Body Generation

  • https://github.com/justimyhxu/awesome-3D-generation
Title Venue Code Year
DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations CVPR DeepFashion 2016
Text2Human: Text-Driven Controllable Human Image Generation ACM Transactions on Graphics (TOG) Text2Human 2022
StyleGAN-Human: A Data-Centric Odyssey of Human Generation arXiv:2204.11823 [cs] 2022
:heavy_check_mark: 3D-Aware Semantic-Guided Generative Model for Human Synthesis arXiv:2112.01422 [cs] 2021
:heart: InsetGAN for Full-Body Image Generation arXiv:2203.07293 [cs] 2022
Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis ICCV impersonator 2019
:heavy_check_mark: SMPLpix: Neural Avatars from 3D Human Models Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision smplpix arXiv. 2021
Neural Articulated Radiance Field ICCV 2021
Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations ECCV ENARF-GAN 2022
Generative Neural Articulated Radiance Fields ArXiv:2206.14314 [Cs] gnarf arXiv. 2022
AvatarGen: A 3D Generative Model for Animatable Human Avatars ArXiv:2208.00561 [Cs] AvatarGen arXiv. 2022
EVA3D: Compositional 3D Human Generation from 2D Image Collections arXiv:2210.04888 2022

Body from video

Title Venue Code Year
SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video arXiv:2201.12792 [cs] 2022

3DMM Face

  • https://github.com/tencent-ailab/hifi3dface
  • https://github.com/ascust/3DMM-Fitting-Pytorch
Title Venue Code Year
Neural Head Reenactment with Latent Pose Descriptors CVPR latent-pose-reenactment 2020
Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry arXiv:2110.09772 [cs] 2021
REALY: Rethinking the Evaluation of 3D Face Reconstruction ECCV REALY 2022

3D FACE Avatars

  • https://github.com/TimoBolkart/BFM_to_FLAME
  • https://github.com/HavenFeng/photometric_optimization
  • https://github.com/soubhiksanyal/FLAME_PyTorch
  • https://github.com/Azmarie/Face-Morphing
Title Venue Code Year
A Morphable Model for the Synthesis of 3D Faces Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques SIGGRAPH ’99, USA: ACM Press/Addison-Wesley Publishing Co. 1999
Learning a Model of Facial Shape and Expression from 4D Scans ACM Transactions on Graphics [FLAME] 2017
:heart: FLAME-in-NeRF : Neural Control of Radiance Fields for Free View Face Animation arXiv:2108.04913 [cs] 2021
Learning a Model of Facial Shape and Expression from 4D Scans ACM Transactions on Graphics 2017
:heart: EMOCA: Emotion Driven Monocular Face Capture and Animation CVPR emoca 2022
FaceVerse: A Fine-Grained and Detail-Controllable 3D Face Morphable Model from a Hybrid Dataset CVPR 2022
I M Avatar: Implicit Morphable Head Avatars from Videos CVPR IMavatar 2022
:heavy_check_mark: Neural Head Avatars from Monocular RGB Videos arXiv:2112.01554 [cs] neural-head-avatars 2022
PVA: Pixel-Aligned Volumetric Avatars arXiv:2101.02697 [cs] 2021
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis arXiv:2103.11078 [cs] 2021
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation arXiv:2201.07786 [cs, eess] 2022
HeadGAN: One-Shot Neural Head Synthesis and Editing arXiv:2012.08261 [cs] 2021
KeypointNeRF: Generalizing Image-Based Volumetric Avatars Using Relative Spatial Encoding of Keypoints arXiv:2205.04992 [cs] 2022
Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set ArXiv:1903.08527 [Cs] Deep3DFaceRecon_pytorch arXiv. 2020

Stylization

Title Venue Code Year
Unified Implicit Neural Stylization ECCV arXiv. 2022
ARF: Artistic Radiance Fields ECCV ARF-svox2 arXiv. 2022
UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene arXiv:2208.07059 UPST-NeRF 2022

Face Style

Title Venue Code Year
Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer arXiv:2203.13248 [cs] DualStyleGAN 2022
Stitch It in Time: GAN-Based Facial Editing of Real Videos arXiv. STIT 2022
Fix the Noise: Disentangling Source Feature for Transfer Learning of StyleGAN ArXiv:2204.14079 [Cs] FixNoise arXiv. 2022
AnimeCeleb: Large-Scale Animation CelebHeads Dataset for Head Reenactment ECCV AnimeCeleb arXiv. 2022
DCT-Net: Domain-Calibrated Translation for Portrait Stylization ACM Transactions on Graphics DCT-Net 2022
VToonify: Controllable High-Resolution Portrait Video Style Transfer ACM Transactions on Graphics (TOG) VToonify n.d.
BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generation NeurIPS BlendGAN 2021
Unpaired Cartoon Image Synthesis via Gated Cycle Mapping CVPR 2022

Face Animation

Title Venue Code Year
Thin-Plate Spline Motion Model for Image Animation CVPR 2022
Depth-Aware Generative Adversarial Network for Talking Head Video Generation CVPR DaGAN arXiv. 2022

Renderer & Regularization

  • https://github.com/ventusff/neurecon
Title Venue Code Year
Implicit Geometric Regularization for Learning Shapes ICML [Eikonal] 2020
Neural 3D Scene Reconstruction with the Manhattan-World Assumption CVPR manhattan_sdf 2022
:heart: Differentiable Signed Distance Function Rendering Transactions on Graphics (Proceedings of SIGGRAPH) sdf 2022
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-View Reconstruction NeuS 2021
SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data ECCV snes 2022
:heart: Volume Rendering of Neural Implicit Surfaces arXiv:2106.12052 [cs] volsdf 2021
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance NeurIPS idr 2020
:heavy_check_mark: Multi-View Mesh Reconstruction With Neural Deferred Shading CVPR neural-deferred-shading 2022
:heart: IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images CVPR IRON arXiv. 2022
:heavy_check_mark: UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction ICCV unisurf 2021
MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction ArXiv:2206.00665 2022
Direct Voxel Grid Optimization: Super-Fast Convergence for Radiance Fields Reconstruction CVPR DirectVoxGO arXiv. 2022
Improved Direct Voxel Grid Optimization for Radiance Fields Reconstruction ArXiv:2206.05085 [Cs] arXiv. 2022
Improved Surface Reconstruction Using High-Frequency Details ArXiv:2206.07850 [Cs] arXiv. 2022
InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering CVPR InfoNeRF arXiv. 2022
Improving Neural Implicit Surfaces Geometry with Patch Warping CVPR NeuralWarp arXiv. 2022
SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views ECCV SparseNeuS arXiv. 2022
[NeuMesh: Learning Disentangled Neural Mesh-Based Implicit Field for Geometry and Texture Editing] ECCV NeuMesh 2022
Neural Density-Distance Fields ECCV neddf arXiv. 2022
Neural 3D Reconstruction in the Wild SIGGRAPH NeuralRecon-W 2022
:heart: KeypointNeRF: Generalizing Image-Based Volumetric Avatars Using Relative Spatial Encoding of Keypoints arXiv:2205.04992 [cs] KeypointNeRF 2022
GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction International Conference on 3D Vision (3DV) go-surf 2022

Material and lighting

Title Venue Code Year
NeILF: Neural Incident Light Field for Physically-Based Material Estimation ECCV neilf arXiv. 2022
[NeRF for Outdoor Scene Relighting] ECCV NeRF-OSR 2022

Motion

  • https://github.com/xianfei/SysMocap
  • https://github.com/zju3dv/EasyMocap
  • https://github.com/EricGuo5513/HumanML3D
Title Venue Code Year
GANimator: Neural Motion Synthesis from a Single Sequence ACM Transactions on Graphics (TOG) ganimator 2022
[Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects] CVPR watch-it-move 2022
Learn to Dance with AIST++: Music Conditioned 3D Dance Generation ICCV 2021
Talking Head(?) Anime from a Single Image 3: Now the Body Too talking-head-anime 2022
PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time ACM Transactions on Graphics 2020
The Wanderings of Odysseus in 3D Scenes CVPR GAMMA arXiv. 2022
Adversarial Parametric Pose Prior CVPR adv_param_pose_prior arXiv. 2022
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars SIGGRAPH AvatarCLIP 2022
[SOMA: Solving Optical Marker-Based MoCap Automatically] ICCV soma 2021
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model arXiv:2208.15001 MotionDiffuse 2022
TEACH: Temporal Action Composition for 3D Humans International Conference on 3D Vision (3DV) teach arXiv. 2022
TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts ECCV TM2T 2022

Shape generation

Title Venue Code Year
Learning Implicit Fields for Generative Shape Modeling arXiv:1812.02822 [cs] 2019

SMPL estimation

  • https://github.com/open-mmlab/mmhuman3d
Title Venue Code Year
End-to-End Recovery of Human Shape and Pose CVPR [hmr] arXiv. 2018
VIBE: Video Inference for Human Body Pose and Shape Estimation CVPR VIBE arXiv. 2020
TransPose: Real-Time 3D Human Translation and Pose Estimation with Six Inertial Sensors ACM Transactions on Graphics TransPose 2021
Monocular Expressive Body Regression through Body-Driven Attention European Conference on Computer Vision (ECCV) expose 2020
Human Mesh Recovery from Multiple Shots CVPR multishot arXiv. 2022
:heart: Learned Vertex Descent: A New Direction for 3D Human Model Fitting ECCV LVD arXiv. 2022
DeciWatch: A Simple Baseline for 10x Efficient 2D and 3D Pose Estimation ECCV DeciWatch arXiv. 2022
PARE: Part Attention Regressor for 3D Human Body Estimation ICCV PARE arXiv. 2021
Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers ECCV FastMETRO 2022

Segmentation

  • https://github.com/facebookresearch/MaskFormer
Title Venue Code Year
Real-Time High-Resolution Background Matting arXiv:2012.07810 BackgroundMattingV2 2020
Robust High-Resolution Video Matting with Temporal Guidance ArXiv:2108.11515 [Cs] RobustVideoMatting arXiv. 2021

Datasets

  • https://github.com/karfly/human36m-camera-parameters
  • https://github.com/deepimagination/TalkingHead-1KH
Title Venue Code Year
Structured Local Radiance Fields for Human Avatar Modeling CVPR THUman4.0-Dataset 2022
Multiface: A Dataset for Neural Face Rendering ArXiv:2207.11243 [Cs.CV] multiface 2022
ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations CVPR ImFace 2022

FLAME estimation

Title Venue Code Year
Towards Metrical Reconstruction of Human Faces ECCV MICA arXiv. 2022

Dog estimation

Title Venue Code Year
[BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information] CVPR barc_release 2022

Panoptic

Title Venue Code Year
Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation ArXiv:2203.15224 [Cs] PanopticNeRF arXiv. 2022

SDF

  • https://github.com/facebookresearch/pifuhd
  • https://github.com/pmneila/PyMCubes
Title Venue Code Year
:white_check_mark: DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation arXiv:1901.05103 [cs] DeepSDF 2019
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling NeurIPS 2016
Occupancy Networks: Learning 3D Reconstruction in Function Space arXiv:1812.03828 [cs] 2019
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization arXiv:1905.05172 [cs] 2019
Deep Meta Functionals for Shape Representation arXiv:1908.06277 [cs] 2019

3D

Title Venue Code Year
Escaping Plato’s Cave: 3D Shape From Adversarial Rendering ICCV 2019
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images arXiv:2004.00121 [cs] 2020
Exemplar-Based 3D Portrait Stylization arXiv:2104.14559 [cs] github 2021
:heart: Landmark Detection and 3D Face Reconstruction for Caricature Using a Nonlinear Parametric Model arXiv:2004.09190 [cs] CaricatureFace 2021
SofGAN: A Portrait Image Generator with Dynamic Styling arXiv:2007.03780 [cs] sofgan 2021
:heart: FreeStyleGAN: Free-View Editable Portrait Rendering with the Camera Manifold arXiv:2109.09378 [cs] 2021
PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering ICCV PIRender 2021

Point Cloud

Title Venue Code Year
Point-Based Modeling of Human Clothing ICCV 2021
ADOP: Approximate Differentiable One-Pixel Point Rendering arXiv:2110.06635 [cs] 2021

Stylization

Title Venue Code Year
Learning to Stylize Novel Views arXiv:2105.13509 [cs] stylescene 2021

Datasets

  • https://github.com/ofirkris/Faces-datasets
Title Venue Code Year
Common Objects in 3D: Large-Scale Learning and Evaluation of Real-Life 3D Category Reconstruction ICCV 2021
A 3D Face Model for Pose and Illumination Invariant Face Recognition IEEE International Conference on Advanced Video and Signal Based Surveillance BFM 2009
SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild arXiv:1712.01261 [cs] 2018

3D-aware image synthesis (ref)

Title Venue Code Year
Visual Object Networks: Image Generation with Disentangled 3D Representation arXiv:1812.02725 [cs, stat] 2018
Escaping Plato’s Cave: 3D Shape From Adversarial Rendering ICCV 2019
HoloGAN: Unsupervised Learning of 3D Representations from Natural Images ICCV 2019

Face

Tools

  • https://github.com/wuhuikai/FaceSwap
  • https://github.com/hysts/anime-face-detector
  • https://github.com/qq775193759/3D-CariGAN
  • https://github.com/yeemachine/kalidokit
  • https://github.com/sicxu/Deep3DFaceRecon_pytorch
  • https://github.com/happy-jihye/face-vid2vid-demo

Edit

Title Venue Code Year
FaceEraser: Removing Facial Parts for Augmented Reality arXiv:2109.10760 [cs] 2021
DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing arXiv:2109.10737 [cs] 2021
:heart: StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators arXiv:2108.00946 [cs] 2021
Beholder-GAN: Generation and Beautification of Facial Images with Conditioning on Their Beauty Level arXiv:1902.02593 [cs] 2019
Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks arXiv:2110.08398 [cs] 2021
Fine-Grained Control of Artistic Styles in Image Generation arXiv:2110.10278 [cs] 2021

Anime Face

  • https://github.com/Sxela/ArcaneGAN
  • https://github.com/mchong6/GANsNRoses
  • https://github.com/FilipAndersson245/cartoon-gan
  • https://github.com/venture-anime/cartoongan-pytorch
Title Venue Code Year
AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation arXiv:2102.12593 [cs] 2021
[AnimeGAN: A Novel Lightweight GAN for Photo Animation] AnimeGANv2 2020
:heart: Learning to Cartoonize Using White-Box Cartoon Representations CVPR White-box-Cartoonization 2020
Generative Adversarial Networks for Photo to Hayao Miyazaki Style Cartoons arXiv:2005.07702 [cs, eess] 2020

3DMM

  • https://github.com/lattas/AvatarMe
Title Venue Code Year
A Morphable Model for the Synthesis of 3D Faces Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques [3DMM] SIGGRAPH ’99, USA: ACM Press/Addison-Wesley Publishing Co. 1999

Face

Title Venue Code Year
SketchHairSalon: Deep Sketch-Based Hair Image Synthesis arXiv:2109.07874 [cs] 2021

Face Alignment

Title Venue Code Year
Face Alignment Across Large Poses: A 3D Solution IEEE Transactions on Pattern Analysis and Machine Intelligence 2019

Face Recognition

Title Venue Code Year
High-Fidelity Pose and Expression Normalization for Face Recognition in the Wild 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015

Face swapping

  • https://github.com/mindslab-ai/hififace

3D

Title Venue Code Year
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild arXiv:1911.11130 [cs] unsup3d 2020
Do 2D GANs Know 3D Shape? Unsupervised 3D Shape Reconstruction from 2D Image GANs arXiv:2011.00844 [cs] GAN2Shape 2021
A Geometric Analysis of Deep Generative Image Models and Its Applications ICLR 2021
Lifting 2D StyleGAN for 3D-Aware Face Generation CVPR LiftedGAN 2021
Image GANs Meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering arXiv:2010.09125 [cs] 2021
Neural 3D Mesh Renderer CVPR 2018
Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction arXiv:2105.07474 [cs] 2021
Inverting Generative Adversarial Renderer for Face Reconstruction CVPR StyleRenderer 2021
Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection arXiv:2106.07852 [cs] 2021
Subdivision-Based Mesh Convolution Networks arXiv:2106.02285 [cs] 2021
Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection CVPR 2021
To Fit or Not to Fit: Model-Based Face Reconstruction and Occlusion Segmentation from Weak Supervision arXiv:2106.09614 [cs] 2021
Unsupervised Learning of Depth and Depth-of-Field Effect from Natural Images with Aperture Rendering Generative Adversarial Networks arXiv:2106.13041 [cs, eess, stat] 2021
DOVE: Learning Deformable 3D Objects by Watching Videos arXiv:2107.10844 [cs] 2021
De-Rendering the World’s Revolutionary Artefacts CVPR 2021
Learning Generative Models of Textured 3D Meshes from Real-World Images ICCV 2021
Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images ICCV 2021

DA

Title Venue Code Year
Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment arXiv:2106.02845 [cs] 2021
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation arXiv:2101.10979 [cs] 2021

Data

  • https://github.com/koaning/doubtlab
Title Venue Code Year
:white_check_mark: Semi-Supervised Active Learning with Temporal Output Discrepancy ICCV 2021
:heart: Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results NeurIPS 2017
When Deep Learners Change Their Mind: Learning Dynamics for Active Learning arXiv:2107.14707 [cs] 2021
On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models arXiv:2108.00114 [cs] 2021
StyleAugment: Learning Texture De-Biased Representations by Style Augmentation without Pre-Defined Textures arXiv:2108.10549 [cs] 2021
Multi-Task Self-Training for Learning General Representations ICCV 2021
OOWL500: Overcoming Dataset Collection Bias in the Wild arXiv:2108.10992 [cs] 2021
Ghost Loss to Question the Reliability of Training Data IEEE Access 2020
Revisiting 3D ResNets for Video Recognition arXiv:2109.01696 [cs, eess] 2021
:heart: Revisiting ResNets: Improved Training and Scaling Strategies arXiv:2103.07579 [cs] 2021
Learning Fast Sample Re-Weighting Without Reward Data ICCV 2021
How Important Is Importance Sampling for Deep Budgeted Training? arXiv:2110.14283 [cs] 2021

CNN & BN

Light architecture

  • https://github.com/yoshitomo-matsubara/torchdistill
  • https://github.com/milesial/Pytorch-UNet
Title Venue Code Year
Network Augmentation for Tiny Deep Learning arXiv:2110.08890 [cs] 2021
Non-Deep Networks arXiv:2110.07641 [cs] 2021
When to Prune? A Policy towards Early Structural Pruning arXiv:2110.12007 [cs] 2021
:heart: ConformalLayers: A Non-Linear Sequential Neural Network with Associative Layers arXiv:2110.12108 [cs] 2021
CHIP: CHannel Independence-Based Pruning for Compact Neural Networks arXiv:2110.13981 [cs] 2021
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training arXiv:2102.02887 [cs] 2021

Antialiased CNNs

Title Venue Code Year
Making Convolutional Networks Shift-Invariant Again arXiv:1904.11486 [cs] 2019
Group Equivariant Convolutional Networks ICML arXiv. 2016
Harmonic Networks: Deep Translation and Rotation Equivariance CVPR arXiv. 2017
Learning Steerable Filters for Rotation Equivariant CNNs CVPR arXiv. 2018

Architecture

Title Venue Code Year
Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning arXiv:2106.05956 [cs] 2021
R-Drop: Regularized Dropout for Neural Networks arXiv:2106.14448 [cs] 2021
Switchable Whitening for Deep Representation Learning ICCV 2019
Positional Normalization arXiv:1907.04312 [cs] 2019
On Feature Normalization and Data Augmentation arXiv:2002.11102 [cs, stat] 2021
Channel Equilibrium Networks for Learning Deep Representation arXiv:2003.00214 [cs] 2020
Representative Batch Normalization with Feature Calibration CVPR 2021
EPSANet: An Efficient Pyramid Squeeze Attention Block on Convolutional Neural Network arXiv:2105.14447 [cs] 2021
Bias Loss for Mobile Neural Networks arXiv:2107.11170 [cs] 2021
Compositional Models: Multi-Task Learning and Knowledge Transfer with Modular Networks arXiv:2107.10963 [cs] 2021
Log-Polar Space Convolution for Convolutional Neural Networks arXiv:2107.11943 [cs] 2021
Decoupled Dynamic Filter Networks arXiv:2104.14107 [cs] 2021
Spectral Leakage and Rethinking the Kernel Size in CNNs arXiv:2101.10143 [cs] 2021
Learning with Noisy Labels via Sparse Regularization ICCV 2021
:heart: Impact of Aliasing on Generalization in Deep Convolutional Networks ICCV 2021
Orthogonal Over-Parameterized Training CVPR 2021
Multiplying Matrices Without Multiplying ICML 2021
AASeg: Attention Aware Network for Real Time Semantic Segmentation arXiv:2108.04349 [cs, eess] 2021
MicroNet: Improving Image Recognition with Extremely Low FLOPs ICCV 2021
Contextual Convolutional Neural Networks arXiv:2108.07387 [cs] 2021
Torch.Manual_seed(3407) Is All You Need: On the Influence of Random Seeds in Deep Learning Architectures for Computer Vision arXiv:2109.08203 [cs] 2021
KATANA: Simple Post-Training Robustness Using Test Time Augmentations arXiv:2109.08191 [cs] 2021
Global Pooling, More than Meets the Eye: Position Information Is Encoded Channel-Wise in CNNs ICCV 2021
:white_check_mark: A ConvNet for the 2020s arXiv:2201.03545 [cs] ConvNeXt 2022

Compression

Title Venue Code Year
AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance arXiv:2109.06397 [cs] 2021

Detection

Title Venue Code Year
Anchor DETR: Query Design for Transformer-Based Detector arXiv:2109.07107 [cs] 2021
:heart: Detecting Twenty-Thousand Classes Using Image-Level Supervision arXiv:2201.02605 [cs] 2022

Segmentation

  • https://github.com/xuebinqin/U-2-Net#usage-for-portrait-generation
Title Venue Code Year
Robust High-Resolution Video Matting with Temporal Guidance arXiv:2108.11515 [cs.CV] 2021

MLP

Title Venue Code Year
ResMLP: Feedforward Networks for Image Classification with Data-Efficient Training arXiv:2105.03404 [cs] 2021
ConvMLP: Hierarchical Convolutional MLPs for Vision arXiv:2109.04454 [cs] 2021
A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP arXiv:2108.13002 [cs.CV] 2021
Sparse-MLP: A Fully-MLP Architecture with Conditional Computation arXiv:2109.02008 [cs] 2021
MLP-Mixer: An All-MLP Architecture for Vision 2021
CycleMLP: A MLP-like Architecture for Dense Prediction ICLR 2022

Transformer

  • https://github.com/xxxnell/how-do-vits-work
  • https://github.com/hamidkazemi22/vit-visualization
Title Venue Code Year
Training Data-Efficient Image Transformers & Distillation through Attention arXiv:2012.12877 [cs] deit 2020
Intriguing Properties of Vision Transformers arXiv:2105.10497 [cs] 2021
CogView: Mastering Text-to-Image Generation via Transformers arXiv:2105.13290 [cs] 2021
An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale arXiv:2010.11929 [cs] 2021
Scaling Vision Transformers arXiv:2106.04560 [cs] 2021
IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers arXiv:2106.12620 [cs] 2021
Rethinking and Improving Relative Position Encoding for Vision Transformer ICCV 2021
Go Wider Instead of Deeper arXiv:2107.11817 [cs] 2021
A Unified Efficient Pyramid Transformer for Semantic Segmentation arXiv:2107.14209 [cs] 2021
:heart: Conditional DETR for Fast Training Convergence ICCV 2021
:heart: Sketch Your Own GAN ICCV 2021
CrossFormer: A Versatile Vision Transformer Based on Cross-Scale Attention arXiv:2108.00154 [cs] 2021
Uformer: A General U-Shaped Transformer for Image Restoration arXiv:2106.03106 [cs] 2021
ConvNets vs. Transformers: Whose Visual Representations Are More Transferable? arXiv:2108.05305 [cs] 2021
Mobile-Former: Bridging MobileNet and Transformer arXiv:2108.05895 [cs] 2021
SOTR: Segmenting Objects with Transformers ICCV 2021
Video Transformer Network arXiv:2102.00719 [cs] 2021
Do Vision Transformers See Like Convolutional Neural Networks? arXiv:2108.08810 [cs, stat] 2021
UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-Wise Perspective with Transformer arXiv:2109.04335 [cs, eess] 2021
$\infty$-Former: Infinite Memory Transformer arXiv:2109.00301 [cs] 2021
PnP-DETR: Towards Efficient Visual Analysis with Transformers ICCV 2021
MobileViT: Light-Weight, General-Purpose, and Mobile-Friendly Vision Transformer arXiv:2110.02178 [cs] 2021
MetaFormer Is Actually What You Need for Vision arXiv:2111.11418 [cs] 2021
Restormer: Efficient Transformer for High-Resolution Image Restoration arXiv:2111.09881 [cs] Restormer 2021
:white_check_mark: An Empirical Study of Training Self-Supervised Vision Transformers arXiv:2104.02057 [cs] 2021
:white_check_mark: When Vision Transformers Outperform ResNets without Pre-Training or Strong Data Augmentations arXiv:2106.01548 [cs.CV] 2021
Visual Attention Network arXiv:2202.09741 [cs] 2022

ssl

  • https://github.com/ucasligang/awesome-MIM
Title Venue Code Year
Emerging Properties in Self-Supervised Vision Transformers arXiv:2104.14294 [cs] dino 2021
What Is Considered Complete for Visual Recognition? arXiv:2105.13978 [cs] 2021
On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals arXiv:2107.14762 [cs] 2021
:heart: Improving Contrastive Learning by Visualizing Feature Transformation ICCV 2021
Scale Efficiently: Insights from Pre-Training and Fine-Tuning Transformers arXiv:2109.10686 [cs] 2021
FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling arXiv:2110.08263 [cs] 2021
BEiT: BERT Pre-Training of Image Transformers arXiv:2106.08254 [cs] 2021
:heart: Parametric Contrastive Learning ICCV 2021
:heart: ImageNet-21K Pretraining for the Masses NeurIPS ImageNet21K 2021
:heart: ML-Decoder: Scalable and Versatile Classification Head arXiv:2111.12933 [cs] ML_Decoder 2021
Asymmetric Loss For Multi-Label Classification ICCV ASL 2021
Grounded Language-Image Pre-Training arXiv:2112.03857 [cs] 2021

Finetune

Title Venue Code Year
:heart: How Transferable Are Features in Deep Neural Networks? arXiv:1411.1792 [cs] 2014

Positional Encoding

Title Venue Code Year
Positional Encoding as Spatial Inductive Bias in GANs arXiv:2012.05217 [cs] 2020
Mind the Pad -- CNNs Can Develop Blind Spots arXiv:2010.02178 [cs, stat] 2020
:heart: How Much Position Information Do Convolutional Neural Networks Encode? ICLR 2020
On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location CVPR 2020
Rethinking and Improving Relative Position Encoding for Vision Transformer ICCV 2021
A Structured Dictionary Perspective on Implicit Neural Representations arXiv:2112.01917 [cs] 2021

NAS

NAS cls

Title Venue Code Year
Neural Architecture Search with Reinforcement Learning ICLR 2017
Learning Transferable Architectures for Scalable Image Recognition CVPR 2018
Progressive Neural Architecture Search ECCV 2018
Efficient Neural Architecture Search via Parameter Sharing ICML 2018
MnasNet: Platform-Aware Neural Architecture Search for Mobile CVPR 2019
DARTS: Differentiable Architecture Search ICLR 2019

NAS GAN

Title Venue Code Year
AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks IEEE Transactions on Pattern Analysis and Machine Intelligence 2021
GAN Compression: Efficient Architectures for Interactive Conditional GANs CVPR 2020
Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search ECCV 2020
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks ICML 2020
A Multi-Objective Architecture Search for Generative Adversarial Networks 2020
AutoGAN: Neural Architecture Search for Generative Adversarial Networks ICCV 2019

Low-level

Super-resolution

  • https://github.com/nihui/realsr-ncnn-vulkan

Frame Interpolation

Title Venue Code Year
FILM: Frame Interpolation for Large Motion arXiv:2202.04901 [cs] 2022

Denoising

Title Venue Code Year
Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering IEEE Transactions on Image Processing 2007
Towards Flexible Blind JPEG Artifacts Removal arXiv:2109.14573 [cs, eess] FBCNN 2021

Scholar

← Metadata

28
Stars
0
Forks
Watchers

Owner

verified publisher icon PeterouZh

Metadata

A collection of papers I am interested in.

Back