How to use Aesthetic Gradients: Stable Diffusion Tutorial

A new paper "Personalizing Text-to-Image Generation via Aesthetic Gradients" was published which allows for the training of a special "aesthetic embedding" which allows the user to specify more clearly what they want to any existing stable diffusion model. In this tutorial we walk through how to train an aesthetic embedding, and use it to generate images.

Discord: https://discord.gg/s8rVscu2pM

00:00 - Summary
01:07 - Paper Explanation
06:51 - Webui Installation
10:25 - Aesthetic Gradients Installation
11:45 - Using Pretrained Embeddings
21:50 - Training New Embedding
29:40 - Comparing Embeddings
34:39 - Experiment Outcomes

-------- Requirements --------

python 3.10
Local Nvidia GPU
CUDA 11.3+

-------- Links --------

Aesthetic Gradients Paper: https://arxiv.org/abs/2209.12330

GIthub Desktop: https://desktop.github.com/
Webui: https://github.com/AUTOMATIC1111/stable-diffusion-webui
Huggingface Stable Diffusion 1.4 Model: https://huggingface.co/CompVis/stable-diffusion-v-1-4-original
Aesthetic Gradients Extension: https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients
Premade Aesthetic Embeddings: https://github.com/vicgalle/stable-diffusion-aesthetic-gradients/tree/main/aesthetic_embeddings
CLIP search of LAION: https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fknn5.laion.ai&index=laion5B&useMclip=false
Code for downloading LAION images: https://github.com/Lewington-pitsos/clip-download
Script to download LAION: https://github.com/Lewington-pitsos/clip-download


Webui thread on embeddings: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/2585
Another webui thread on embeddings: https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/3350
Github Repo for original implementaion: https://github.com/vicgalle/stable-diffusion-aesthetic-gradients
Useful Blog Post: https://metaphysic.ai/custom-styles-in-stable-diffusion-without-retraining-or-high-computing-resources/
Github Repo containing the portrait embeddings: https://github.com/Lewington-pitsos/clip-download

-------- Misc --------

ffmpeg commands to extract images from videos:

ffmpeg resize and crop video: ffmpeg -i ghib.mp4 -c:a copy -filter:v "scale=960:512,crop=iw-448:ih-0" smol-ghib.mp4
ffmpeg extract images: ffmpeg -i smol-ghib.mp4 -r 0.3 -f image2 image-%4d.jpeg

-------- Music --------

Music from https://freetousemusic.com

‘Late Morning’ by ‘LuKremBo’:https://www.youtube.com/watch?v=oqMrdjO6pbA
‘Daily’ by ‘LuKremBo’:https://www.youtube.com/watch?v=Tchb1Q4V-nc
‘Marshmallow’ by ‘LuKremBo’:https://www.youtube.com/watch?v=y7KYdqVND4o
‘Travel’ by ‘LuKremBo’:https://www.youtube.com/watch?v=iixgZWLipXI
‘Sunset’ by ‘LuKremBo’:https://www.youtube.com/watch?v=gv7hcXCnjOw
‘Biscuit’ by ‘LuKremBo’:https://www.youtube.com/watch?v=EtZ2m2Zm3vY
‘Sunflower’ by ‘LuKremBo’: https://www.youtube.com/watch?v=aoL_cSU1Xtw
‘Chocolate’ by ‘LuKremBo’: https://www.youtube.com/watch?v=9P4wn4ITE_Q
‘Branch’ by ‘LuKremBo’: https://www.youtube.com/watch?v=xPOy2pkImiU
‘Rose’ by ‘LuKremBo’: https://www.youtube.com/watch?v=XIhyti2Qa74
‘Butter’ by LuKremBo: https://www.youtube.com/watch?v=Ua7Qfc1xu90
‘Onion’ by LuKremBo: https://www.youtube.com/watch?v=KGQNrzqrGqw
‘Animal Friends’ by LuKremBo: https://www.youtube.com/watch?v=ag2RePNqwCs
‘Snow’ by LuKremBo: https://www.youtube.com/watch?v=wYiNao04Wg0
‘Affogato’ by LuKremBo: https://www.youtube.com/watch?v=YTUF1o9Sf3E

Many thanks to LuKremBo

#stablediffusion #aiart #tutorials #techtutorials #promptcrafting #install #installation #researchpaper