Noise as Material
To teach a neural network to draw, you need clear exercises: a problem and its correct answer. For text this was simple — you hid the next word of a book and set the model to guess it. But how do you teach a machine to draw a whole image out of nothing? The brilliant answer of the diffusion models was to turn the problem upside down: if we do not know how to create an image from scratch, we certainly know how to destroy one. Random noise, the old enemy of data transmission, becomes here the clean clay out of which any form is born.
Key terms: the forward process (forward diffusion) noise level the noise schedule the direct leap to step t
A task born of its own destruction
When we train a language model, the nature of text gives us an unhoped-for gift: every sentence contains its own teacher. We take any article or book in the world, cover the word at the end and ask the network: “what comes next?”. For the visual world, however, there is no such catalogue. A painter does not lay the paint down from left to right, pixel by pixel, and nobody has photographed billions of half-finished paintings alongside their finished versions.
The great unlocking came from an idea borrowed from statistical physics: instead of trying to paint a masterpiece at one stroke, we take a clear photograph and degrade it gradually, on a rigorous plan. We add to it a minute powder of digital fleas (random noise), then a little more, and a little more, step by step, until every trace of the original image is entirely wiped out. This controlled journey towards chaos is called the forward process (or forward diffusion) and it calls for no intelligence at all: it is a simple mathematical recipe.
The magic of this deliberate destruction is that it automatically produces billions of perfect exercises, with no human effort. At every intermediate step we know two things for certain: what the image partly eaten by fleas looks like, and exactly how much noise we have just sprinkled over it. Whoever teaches a neural network to look at the dirtied image and guess the noise added gives it the very key with which the road back can later be retraced.
Imagine a painting left in an old room, on which a microscopic thread of dust settles every second. Nobody scatters the dust on purpose; it falls by a known physical rule. And precisely because the rule is exact, we can reckon what the canvas looked like a single second earlier.
How the dust is added: the recipe for a careful dilution
If we simply threw bright pixels over a photograph at every step, the values would grow out of control and the canvas would soon become a blinding white. To keep everything in hand, the forward process works like a careful dilution: at every step the existing image is first faded a little (shrunk by a fraction), and then a quantity of fresh noise, measured to the millimetre, is added to fill exactly the room left free.
The calculations are designed for images whose variation (the spread of the values) is equal to one. The simple image in the drawing below is gentler at the start, so its variation climbs gradually towards one as the noise takes the place of the useful content, instead of staying fixed. The figures under the picture show this gradual handing-over of the baton clearly.
The part still surviving from the initial image at step t defines the so-called noise level. At step zero the useful signal is 100% and the noise is non-existent. As the steps flow by, the starting image thins out continuously, until at step one thousand nothing is left of it but a mathematical ghost: the whole screen is taken up by pure, standard noise.
The noise schedule: the tempo of dissolution
How much noise is added at each step is not left to chance. The series of these proportions is settled in advance and is called the noise schedule (or noise schedule). On this rhythm depends where along the way the great dissolution of the forms takes place.
What really matters for recognition is the ratio between how much of the image is left and how much noise has gathered. The place where this ratio passes through one marks the watershed of the process: up to there the image weighs more than the dust; after it, chaos takes command. A well-made schedule gives the network steps enough both to sort out the large forms and to learn the fine textures.
The direct leap: a thousand steps in a single blink
The forward process hides a remarkable mathematical property, without which training would be impracticably slow. To reach step eight hundred, we do not have to simulate eight hundred successive additions laboriously. Because Gaussian noise added several times remains Gaussian noise, the dust accumulated can be compressed into a single huge step.
We can write the dirtied image at any step t directly, by a simple formula linking the starting photograph to a fresh sample of noise. For every training exercise the computer chooses a step t at random, leaps there instantly and asks the network to guess the noise hidden in the mixture.
Why we stop in clean noise: the universal gate
At the end of the thousand steps we do not obtain an image merely slightly blurred or partly destroyed, but something that cannot be told apart in any way at all from a screen of pure noise generated from scratch. This complete loss is not a failure but the very keystone of generation.
If, pushed to the end, all the images in the world arrive at exactly the same statistical point — a uniform sea of fleas — then the road back can begin from anywhere! We can take a fresh sheet of random noise, with no connection to any photograph of the past, and walk down the learned path towards a coherent, entirely new image. All the roads have dissolved into the same universal chaos, so from that chaos one can set out towards any of them.
The experiment below takes two completely different images, leads them both to the end of the forward process and measures whether the results differ in any way. At the end, their origin is entirely wiped out.
The road forward is pure mechanical arithmetic: the degradation follows a simple formula and calls for no learning. The road back, by contrast, cannot be deduced directly: out of clean noise billions of possible images could be born, and nothing on the screen says where it started from. The next lesson reveals what the network learns in place of certainty, and how, out of a mere ant-heap, a form begins to take shape.