Generative Deep Art

A curated list of Generative Deep Art projects, tools, artworks, and models

Inbox

• [2208.01618] An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
• yuval-alaluf/stylegan3-editing: Official Implementation of "Third Time's the Charm? Image and Video Editing with StyleGAN3" https://arxiv.org/abs/2201.13433
• autonomousvision/projected_gan: [NeurIPS'21] Projected GANs Converge Faster
• apple/ml-no-token-left-behind: PyTorch Implementation of No Token Left Behind: Explainability-Aided Image Classification and Generation
• hlky/stable-diffusion-webui: Stable Diffusion web UI
• DreamStudio: Frontend for Stable Diffusion API by Stability.ai
• amrzv/awesome-colab-notebooks: Collection of google colaboratory notebooks for fast and easy experiments
• LatentVision_rynmurdock.ipynb - Colaboratory
• Latent_Diffusion_with_LAION_400M.ipynb - Colaboratory
• Top 41 AI Art Generators: Make AI Art, Paintings & More (2021 GUIDE) — AIArtists.org
• 10 Best AI Art Generators (2022) - Unite.AI
• baowenbo/DAIN: Depth-Aware Video Frame Interpolation (CVPR 2019)
• Dain-App 1.0 [Nvidia Only] by GRisk: Depth-Aware Video Frame Interpolation (CVPR 2019)
• Lucid Lyrics - AI Assisted Art: AI-Assisted Lyrical Interpretations by Walter Arnold
• TencentARC/GFPGAN: GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration
• Machine learning for arts
• ProsePainter: an interactive tool to "paint with words." It incorporates guidable text-to-image generation into a traditional digital painting interface
• ProsePainter: Image + Sketching Interface + CLIP! - YouTube
• AI Image to sound [Melobytes.com]
• http://runwayml.com/
• https://hotpot.ai/
• Get started with making AI art in 2022 – deeplearn.art
• (3) promptism: Promptism is an art movement whose creators apply machine learning models trained with contrastive language-image pre-training (CLIP) techniques for image generation
• The Promptist Manifesto – deeplearn.art
• Post Reddit - MachineLearning: List of sites/programs/projects that use OpenAI's CLIP neural network for steering image/video creation to match a text description
• Text-to-Image Summary – Part 1 | Softology's Blog
• disco-diffusion/Local_Disco_Diffusion_v4_1.ipynb at main · Midgraph/disco-diffusion
• Audio to keyframe string: this tool is used to generate strings for the keyframes of AI animation notebooks, such as this VQGAN+CLIP Animations notebook, using the volume of audio tracks.
• pixray/pixray: Pixray is an image generation system
• pixray/pixray_notebooks: pixray demo notebooks
• https://pixray.gob.io/text2image/
• sberbank-ai/ru-dalle: Generate images from texts. In Russian.
• [🔥🔥] Hitchhiker's Guide To The Latent Space: Community Notebook Document: great collection of interesting notebooks
• [🔥🔥] Pollinations.AI: a frontend hosting a set of curated notebooks that allow creating and experimenting with generative art. Their objective is to facilitate the creation and translation of multiple forms of human expressions.
• [🔥 ] hive/notebooks at main · pollinations/hive: collection of notebooks used by Pollinations.AI
• ouhenio/StyleGAN3-CLIP-notebooks: A collection of Jupyter notebooks to play with NVIDIA's StyleGAN3 and OpenAI's CLIP for a text-based guided image generation.
• k-l-lambda/stylegan-web: A web porting for NVlabs' StyleGAN.
• [🔥🚀 ] S2ML Image Generator: evolution of the first VQGAN+CLIP Google Colab notebook by Katherine Crownson maintained by Justin Bennington
• [🔥 ] Create Variations on Images With Looking Glass 1.1 (ru-DALLE) - YouTube | Artificial Images
• [🔥🚀 ] Looking Glass 1.1 (ru-DALLE): Making ruDALL-E fine tuning quick and painless. Copyright (C) 2021 Bearsharktopus Studios
• [🔍 ] NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion (ML Research Paper Explained) - YouTube | Yannic Kilcher
• [🔥 ] yuval-alaluf/hyperstyle: Official Implementation for "HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing" https://arxiv.org/abs/2111.15666
• [🔥 ] Vadim Epstein’s Aphantasia library: CLIP + FFT/DWT/RGB = text to image/video
• mikaelalafriz/lucid-sonic-dreams: syncs GAN-generated visuals to music
• aakashjhawar/AvatarGAN: Generate Cartoon Images using Generative Adversarial Network
• Greg Surma - Portfolio
• crowsonkb (Katherine Crowson): who wrote the tutorial of VQGAN+CLIP
• [🔥 ] dvschultz/stylegan3: Forked by Schultz from Stylegan3
• Alias-Free Generative Adversarial Networks (StyleGAN3): NVidia official StyleGAN3 release page
• [🔥 ] dvschultz/ml-art-colabs: A list of Machine Learning Art Colabs
• [🚀 ] StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators
• Let's Talk Energy Usage of Generative Machine Learning
• StyleGAN: Use machine learning to generate and customize realistic images | by Jamshed Khan | Heartbeat
• Audio-reactive Latent Interpolations with StyleGAN
• DALL·E: Creating Images from Text
• Alien Dreams: An Emerging Art Scene - ML@B Blog
• DeepStory: A tale of co-creation between man & machine
• SofGAN: A GAN Face Generator That Offers Greater Control - Unite.AI
• Appreciating the Poetic Misunderstandings of A.I. Art | The New Yorker
• DALL-E mini: DALL·E mini is an AI model that generates images from any prompt you give!
• DALL-E mini GitHub
• DALL-E mini Project Report
• CLIPIT PixelDraw - Colaboratory
• CLIP Guided Diffusion HQ 512x512.ipynb - Colaboratory
• Smooth Transitioning Between Position / Rotation / Zoom and Text Inputs by Keyframing Parameters: A Proof of Concept [15,000 Frames] : deepdream
• justinpinkney/awesome-pretrained-stylegan2: A collection of pre-trained StyleGAN 2 models to download
• Image Super-Resolution
• neural-dream Alternatives and Similar Photos & Graphics Apps | AlternativeTo
• Real GANs in AI | Hacker Noon: deep dive into what the generative models are, the recent developments in the field, and the usage of GANs in business
• CoG 21: Adversarial Reinforcement Learning for Procedural Content Generation
• GANshare | Towards Data Science: Creating and Curating Art with AI
• GitHub Repositories of Hugging Face
• AgaMiko/pixel_character_generator: Generating retro pixel game characters with Generative Adversarial Networks. Dataset "TinyHero" included.

Big techs, big models

• Imagen: Google's Text-to-Image Diffusion Models
• Make-A-Scene: Meta's creative control for AI image generation
• DALL·E 2: OpenAI system, evolution of the DALL-E launched in Jan 2021, that can create realistic images and art from a description in natural language.
• Stable Diffusion: Stability.Ai's text-to-image model that is a breakthrough in speed and quality meaning that it can run on consumer GPUs
• LAION: Large-scale Artificial Intelligence Open Network

Text-to-image

Prompt Engineering | Prompt Design | Prompt Craft

• DALL·Ery GALL·Ery Resources: DALL·E 2 and AI art prompt resources & tools to inspire beautiful images
• [2204.13988] A Taxonomy of Prompt Modifiers for Text-To-Image Generation
• List of Aesthetics | Aesthetics Wiki | Fandom
• Artist Directory (Volcano Comparison) | AI Art Creation Wiki | Fandom
• The DALL·E 2 Prompt Book – DALL·Ery GALL·Ery
• DALL·Ery GALL·Ery: A guide to OpenAI's DALL·E – prompts, projects, examples, and tips
• (2) MASSIVE 💥 DALL-E 2 ANIME ⚡︎ KEYWORDS + MODIFIERS LIST ★ : haaaaven: image prompt modifier collection by haaaaven
• DrawBench: a list of prompts the Google Imagen is organizing as a benchmark
• CLIP Prompt Engineering for Generative Art - matthewmcateer.me: list of styles tested with Quick CLIP Guided Diffusion
• Adobe should make a boring app for prompt engineers (Interconnected)
• [2206.00169] Discovering the Hidden Vocabulary of DALLE-2

Text-to-image Google Colab Notebooks

• VQGAN + CLIP + Gumbel
• OpenAI DVAE+CLIP
• Text2Image VQGAN
• Improved Multi Perceptor VQGAN + CLIP
• Latent Majesty Diffusion v1.3
• CLIP Guided Decision Transformer
• CLIP Guided Diffusion
• GLIDE
• PixelDirect
• CLIP Guided Diffusion Secondary Model Method
• Zoetrope
• Quick CLIP Guided Diffusion
• MSE Regulized VQGAN+CLIP
• CLIPRGB ImStack
• CLIP Guided Diffusion v4

GLIDE

• (3) "GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models", Nichol et al 2021 (OpenAI's DALL-E successor: 5b-parameter diffusion models + noise-aware CLIP) : MediaSynthesis
• (3) OpenAI paper: "GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models". Code and a smaller filtered model (300 million parameters vs 3.5 billion parameters in the paper) are available. : bigsleep

VQGAN+CLIP

• VQ-GAN | Paper Explanation - YouTube: explanation video about VQGAN with didactic visualizations
• VQGAN+CLIP — How does it work?. The synthetic imagery (“GAN Art”) scene… | by Alexa Steinbrück | Aug, 2021 | Medium
• Introduction to VQGAN+CLIP - 🟧Sourceful
• How to use VQGAN+CLIP to generate images from a text prompt —tutorial for beginners | NightCafe Creator
• VQGAN + CLIP Keyword Modifier Comparison - NightCafe Creator
• ~200 CLIP+VQGAN keywords tested on 4 subjects : bigsleep
• AI Generated Art Scene Explodes as Hackers Create Groundbreaking New Tools CLIP+VQ-GAN | VICE
• VQGAN+CLIP (with pooling): Generate images from text prompts with VQGAN and CLIP (z+quantize method)
• ~200 CLIP+VQGAN keywords on 4 subjects, by @kingdomakrillic  - Imgur
• Experimentando o VQGAN+CLIP: gerador de imagens a partir de textos - YouTube Filipe Calegario

Text-to-video

THUDM/CogVideo: Text-to-video generation

How did they do it?

• Creep - YouTube by Glenn Marshall Neural Art: how did they translated the images using VQGAN+CLIP? How did they seamlessly wander on the latent space?

To organize

• https://cocreator.ai/
• Pyttipanna: visual interface for Pytti by @_staus. Pytti is created by @sportsracer48

Inbox: Text and NLP

• bigscience/bloom · Hugging Face: getting started with BLOOM
• BLOOM: open-source 176-billion-parameter model aims to democratize large-language models
• Top 10 GPT-3 Powered Applications to Know in 2022
• Google AI Blog: Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance
• Riku.AI | The vault for your A.I. creations
• First look - Riku.ai - inference platform Mar/2022 - J1, GPT-3, Fairseq-13B, GPT-NeoX-20B, Cohere-XL - YouTube
• Rytr: Rytr is an AI writing assistant that helps creating content
• Charisma: Charisma is a platform for creating interactive stories with believable virtual characters
• Authors A.I.: AI-powered text analysis
• DeepMind says its new language model can beat others 25 times its size | MIT Technology Review
• Integrated AI: How to talk to AI for free using nine platforms (Megatron, GPT-3, GPT-J, Wudao, J1..) - YouTube by Dr Alan D. Thompson. The following references came from this video description
• InferKit
• CopyHat
• Emerson on iOS
• Jurassic-1
• EleutherAI | GPT-J-6B
• AIx
• Wudao
• Emerson on Telegram

OpenAI GPT-3

• GPT-3 playground
• Fine-tuning GPT-3: how to customize a model for OpenAI's GPT-3

Online Tools and Applications

• Toonify yourself | Justin Pinkney
• https://deepart.io/
• Artbreeder: web-based tool to generate images by breeding existing images
• Ostagram.ru: image style transfer plataform
• cleanup.pictures: remove objects, people, text and defects from any picture for free
• remove.bg: remove background from images
• Quick, Draw!: Can a neural network learn to recognize doodling? A game to help NL by adding users drawing
• LOVO AI: AI Voiceover & Text to Speech Platform with human-like voices
• AIVA: The AI composing emotional soundtrack music

Papers and references for reading and understanding

• Casual GAN Papers Reading Club: Community knowledge base for Casual GAN Papers
• Casual GAN Papers: Easy to read summaries of popular AI papers
• The Illustrated VQGAN: illustrated explanation on how VQGAN works
• CLIP: Connecting Text and Images: OpenAI's explanation on how CLIP works
• VQGAN+CLIP — How does it work?. The synthetic imagery (“GAN Art”) scene… | by Alexa Steinbrück | Medium
• The Methods Corpus | Papers With Code
• https://ieeexplore.ieee.org/abstract/document/9043519: A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks
• Utilizando redes adversárias generativas (GANs) como agente de apoio à inspiração para artistas: Trabalho de Graduação de Cláudio Carvalho no Centro de Informática - UFPE
• GAN Lab: Play with Generative Adversarial Networks in Your Browser!

Papers To organize

• [PDF] Music2Video: Automatic Generation of Music Video with fusion of audio and text | Semantic Scholar
• [PDF] Active Divergence with Generative Deep Learning - A Survey and Taxonomy | Semantic Scholar
• [PDF] Automating Generative Deep Learning for Artistic Purposes: Challenges and Opportunities | Semantic Scholar

Interesting Instagram Accounts

• Deep Music Visualizer
• Lucid Sonic Dreams (@lucidsonicdreams)

Interesting Youtube Channels

• Artificial Images: Demos and explanations to make art using machine learning
• Glenn Marshall Neural Art
• How to Generate Art - Intro to Deep Learning #8

Interesting GitHub Repositories

• https://github.com/dvschultz: GitHub de Derrick Schultz com vários notebooks que podem ser rodados no Google Colab
• dvschultz/ml-art-colabs

Artworks

• Confluence: a generative art project by Devi Parikh on BrainDrops.
• Learning to See – Memo Akten | Mehmet Selim Akten | The Mega Super Awesome Visuals Company
• Neural Zoo | Sofia Crespo
• KRЯRL DЯAWINGS: Runway ML -- 3rd "Model" (based on long poses)
• Frea Buckler ~ Artist: obras usadas para criar essa rede (19) derrick has started yet another project on Twitter: "Just sent @buntworthy a demo StyleGAN model I trained / Twitter
• (Non-)Human
• Authentic Digital Art - Unknown Departure | SuperRare
• A Selection of Machine Learning Art Inspiration
• Top 25 AI Artists of 2021 (Photos, Profiles & History of AI Art)- AIArtists.org: AIArtists.org showcases leading artists using Artificial Intelligence, tools to make AI Art, and a timeline of AI Art History.
• Helena Sarin – Artist Profile (Photos, Videos, Exhibitions) — AIArtists.org
• Images Generated By AI Machines (@images_ai) / Twitter
• https://www.instagram.com/refikanadol/

Galleries

• AICAN
• Ganvas Studio - Neural Network Paintings
• Syn Feather Sweater / STRELITZIA – HATRA E STORE

Selected examples of This X Does Not Exist

• This vessel does not exist.: A gallery of both authentic and fake vessels generated by machine learning software (StyleGAN)
• https://thispersondoesnotexist.com/ - gerador de rostos humanos
• https://thisxdoesnotexist.com/ - site com vários exemplos de outros modelos

Pre-trained Models

• justinpinkney/awesome-pretrained-stylegan2: A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution
• A collection of pre-trained StyleGAN 2 models to download - Deep Learning | ReposHub
• Modelo pretreinado com obras de arte moderna

Neural Net Models

• Yuheng-Li/MixNMatch
• lucidrains/lightweight-gan: 512x512 flowers after 12 hours of training, 1 gpu 256x256 flowers after 12 hours of training, 1 gpu Implementation of 'lightweight' GAN proposed in ICLR 2021
• VQGAN+CLIP: This is a brief tutorial on how to operate VQGAN+CLIP by Katherine Crowson
• rinongal/StyleGAN-nada: Zero-Shot non-adversarial domain adaptation of pre-trained generators

Latent Space Explorarion

• Latent Space Exploration with StyleGAN2

Other Related Awesome Lists

• margaretmz/awesome-ai-art-design: An awesome list: AI for art and design.
• toxtli/awesome-machine-learning-jupyter-notebooks-for-colab: A curated list of Machine Learning and Deep Learning tutorials in Jupyter Notebook format ready to run in Google Colaboratory

Notes, ideas for projects

• Video clip: music + lyrics => image + video
• Book: text => illustration
• Interfaces for editing parameters that can be exported as string. This string is the input for a notebook or model. "Copy string", "Paste string", "Export file", "Import file".

Improving Google Colab experience

• 7 ways to load external data into Google Colab | by B. Chen | Towards Data Science
• 10 tricks for a better Google Colab experience | by Cyprien NIELLY | Towards Data Science
• Quickly share ML WebApps from Google Colab using ngrok for Free | by AbdulMajedRaja RS | Towards Data Science
• Jupyter Widgets for Interactivity in Google Colab: notebook with examples of using Jupyter Widgets in Colab, allowing interactive inputs
• Jupyter Widgets official documentation

Troubleshooting

Ref

Inbox for Adjacent or Related Pero No Mucho

• The Art of PNG Glitch
• HashLips/hashlips_art_engine: tool used to create multiple different instances of artworks based on provided layers

Philosophical Discussion

• Deep Else: A Critical Framework for AI Art

Contribute

Contributions welcome! Read the contribution guidelines first.

License

To the extent possible under law, Filipe Calegario has waived all copyright and related or neighboring rights to this work.