Generative AI: images, video, audio
GANs, diffusion, Stable Diffusion, voice synthesis: how AI creates new content.
On this page
1. Producing rather than just classifying
Discriminative vs. generative
"Classic" (discriminative) AI answers "is this image a cat?". Generative AI does the opposite: "draw me an astronaut cat". It doesn't copy an existing image: it learned the statistical patterns of millions of images and produces new ones that resemble them.
What does a generative AI do, as opposed to a discriminative one?
An AI-generated image is assembled from pieces of images stored in the model.
2. First generation: GANs (2014)
A duel between two networks
In 2014, Ian Goodfellow and colleagues proposed generative adversarial networks (GANs): a generator produces fake images, a discriminator tries to tell real from fake. Through this back-and-forth, the generator becomes very realistic.
Limits that opened the door to diffusion
GANs popularized synthetic faces of people who don't exist, but remain hard to train and hard to control from text — they've been largely superseded by diffusion models.
In a GAN, what is the "discriminator's" role?
What year were GANs proposed?
3. Diffusion: the art of removing noise
How a diffusion model works
During training — Noise is gradually added to a real image until it becomes pure random noise. The model learns to reverse each step: remove a bit of noise.
When you generate — The model starts from pure random noise and "denoises" it step by step, guided by your text description.
The link to text — A second model (CLIP-like) turns your description into numbers that guide the denoising.
Stable Diffusion (August 2022)
Diffusion models underlie DALL-E 2 (2022), Midjourney, and Stable Diffusion, released openly in August 2022. Each generation starts from different noise, which is why the same prompt gives a different image each time.
Put the 3 steps of diffusion-based generation in order
Why does the same prompt produce a different image each time it's generated?
4. Voice, music, video
Other domains, the same idea adapted
| Domain | How it works |
|---|---|
| Image | Text-guided diffusion |
| Voice | The model learns a voice's characteristics to read any text |
| Music | Generates an audio signal from a text describing style and lyrics |
| Video | Diffusion applied to a sequence of images, with consistency over time |
Match each description to its domain
Keeping the same face and setting throughout an AI-generated video is now perfectly solved, with no remaining difficulty.
5. Copyright and deepfakes
Ongoing lawsuits
Models were trained on huge collections of images and text, often copyrighted. Several lawsuits are ongoing: Getty Images v. Stability AI (2023), The New York Times v. OpenAI (December 2023).
If you publish AI-generated content, disclose it. Check the tool's terms for commercial use, and never reproduce a person's face or voice without their consent.
Which December 2023 lawsuit illustrates tensions around copyright and generative text AI?
A deepfake can be used both for content creation and for disinformation or identity theft.
Boss challenge: GAN or diffusion?
Sort each statement according to whether it describes GANs or diffusion models.