Extracting Training Data from Diffusion Models
by Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace · arXiv
A USENIX Security 2023 study that recovers over a thousand memorized training images from Stable Diffusion, Imagen and DALL-E 2, including photographs of identifiable people and trademarked logos, and measures which design choices worsen memorization. It quantifies how often models reproduce training images and reports the copyright status of what it extracted.
More resources on Generative Image & Video Models
Hands-On Generative AI with Transformers and Diffusion Models
A practitioner's book from the Hugging Face Diffusers team covering how diffusion models work, Stable Diffusion internals, sampling and guidance, fine-tuning with LoRA and DreamBooth, ControlNet and inpainting workflows, and chapters on audio and video generation.
Practical Deep Learning for Coders, Part 2: From Deep Learning Foundations to Stable Diffusion
Over thirty hours of free video lessons rebuilding Stable Diffusion from scratch, starting at matrix multiplication and autograd and working up through DDPM, DDIM, conditional sampling, textual inversion and DreamBooth, with paper-reading practice throughout. The only free course taking a reader from tensor primitives to a working diffusion implementation without hand-waving, and the depth means the understanding survives model churn. Long and demanding.
Hugging Face Diffusers Documentation
Official documentation for the Diffusers library: pipelines for image, video and audio generation, scheduler choices, LoRA and adapter loading, ControlNet usage, DreamBooth and textual-inversion training scripts, quantization and memory-offloading guides, and conceptual explanations of each component. The practical half of the topic — running models, choosing schedulers, applying LoRA and ControlNet, fine-tuning — has no better free source, and the maintainers' own docs avoid the huge volume of blog tutorials written against long-dead API versions.
Adding Conditional Control to Text-to-Image Diffusion Models (ControlNet)
The paper introducing ControlNet, which adds spatial conditioning such as edges, depth, segmentation and human pose to a frozen text-to-image diffusion model using trainable copies of its encoder joined by zero-initialized convolutions.
High-Resolution Image Synthesis with Latent Diffusion Models
The CVPR 2022 paper behind Stable Diffusion. It runs the diffusion process in the latent space of a pretrained autoencoder and adds cross-attention conditioning, cutting compute enough for consumer hardware while handling text, layout, inpainting and super-resolution. Every practical tool in this topic — SD 1.5, SDXL, and their descendants — is an implementation of this paper.
What are Diffusion Models?
A continuously revised technical survey covering forward and reverse diffusion processes, the DDPM training objective, score-based formulations, classifier and classifier-free guidance, latent diffusion, ControlNet, diffusion transformers, and acceleration methods including DDIM and consistency models. The map of the whole field in one page, with consistent notation across papers that each use their own. Kept updated since 2021 rather than abandoned, so it now covers guidance, latent diffusion, ControlNet and DiT in the same framework. The reference you return to after the Nakkiran tutorial teaches the core derivation.