Lecture 05 · 29 September 2026
Making the Model Yours.
Adaptation, creative workflows, and authorship: how to teach a model your character, object, or style, and what that means for the artists whose work models learn from.
80 slides · English script / 中文讲解 · Three assigned readings · Five-question quiz
One artist · one art-and-tech paper · one technique paper
Three readings for this week.
All three connect directly to tonight's slides and are short: one news article and two project pages, where the figures are the reading. For Quiz 05, reflect on any one of them.
Reading 1 / Art
Melissa Heikkilä · MIT Technology Review | September 16, 2022
Read the article (about 10 minutes). Connect it to the “memorized style” slides.
Greg Rutkowski is a Polish illustrator known for fantasy landscapes painted in a classical style. His name was used as a prompt about 93,000 times with Stable Diffusion, far more than famous names such as Picasso. At first he thought it might bring him new viewers; then he found AI images carrying his name that he never made, and worried that his real work would become hard to find.
Questions to keep in mind
- Why did so many people type Rutkowski's name into their prompts?
- What exactly worries him: copying, credit, income, or something else?
- Is using an artist's name in a prompt homage, mimicry, or something new?
Tonight's slides showed Stable Diffusion imitating his style, and then a model that was made to forget it.
This is a news report from 2022. The legal cases and the tools have changed since.
Reading 2 / Art + tech
Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu · ICCV 2023 | Project page
Read the abstract on the project page and look closely at the before-and-after figures. You do not need to read the method.
The paper shows how to make a text-to-image model “forget” one thing — a living artist's style, a copyrighted character, or a memorized photo — while keeping everything else. Ask for Grumpy Cat and you get an ordinary cat; ask for a painting in Greg Rutkowski's style and the style is gone.
Questions to keep in mind
- Pick one before-and-after pair. What exactly changed?
- Who should decide what a model is made to forget?
- Does removing an artist's name from one model protect the artist?
This is the method behind tonight's R2D2, Snoopy, Van Gogh and Greg Rutkowski slides.
The equations are optional. The figures are the reading.
Reading 3 / Technique
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu · CVPR 2023 | Project page
Read the abstract on the project page and look at the results. Compare Custom Diffusion with DreamBooth and Textual Inversion.
Custom Diffusion teaches a model a new concept — a pet, an object, a style — from only a few photos, by changing just a small part of the model and adding a new word such as “V* dog”. Because the change is small, two separately learned concepts can be combined in one image.
Questions to keep in mind
- How many images does the model need to learn a new concept?
- What is the new word V* for?
- Which example would be most useful for your own film?
Most of tonight's customization slides — the moongate, Jun-Yan's dog, the wooden pot — come from this paper.
You are not expected to understand the optimization.
Download the reading guide
