The Transformer Block

The conveyor belt: the file that travels unhindered

In the old models, every layer destroyed the previous representation in order to replace it with a new one. The Transformer architecture changes this logic radically. Every word sets out as a rich vector of coordinates, and this vector travels from input to output along an unbroken motorway called the residual stream.

The most intuitive analogy is that of a working file that circulates through a series of specialised offices: each office reads the file, adds a new sheet with useful observations or a stamp, and sends it further along the belt, but no office throws the file away! What attention discovered on the first floor stays preserved and can be consulted intact on the fortieth floor as well.

Every Transformer block has two great workshops that read from this belt and add their results back onto it: the first is the attention mechanism (which exchanges ideas between different words), and the second is a small factory of individual thought for each word separately.

The workshop of thought: each word’s dense network

Once attention has gathered the clues from the neighbours, the block’s second engine comes into action: the network at every position (known as the feed-forward network, or MLP). It works like an office of individual reflection: it takes a word’s enriched vector, projects it into an internal space four times wider where it combines the nuances, applies a non-linear activation function and brings it back to the standard size.

Two features distinguish it essentially from the attention mechanism. First, it acts entirely independently at each position: the word at position 3 is processed in its own cubicle without peeping at the neighbours. Second, it uses exactly the same formula of computation for all the words, ensuring perfect consistency throughout the text.

This division of labour also explains the consumption of resources: attention compares every word with all the others (quadratic cost), while the dense network is applied once per word (linear cost). In long texts, attention makes the greatest demand on the computer; the dense network, on the other hand, is what stores two thirds of the permanent memory and the raw facts the model has learned.

Attention moves the pieces on the board, but it is the dense network that holds the greater part of the model’s parameters. It is like a library of encyclopaedic knowledge stored directly in the weights, consulted at every step.

The bypass lane: saving the signal

A golden rule of the Transformer block is that no module rewrites the vector from scratch. Every module produces only a fine addition (a small numerical adjustment), which is simply added on top of the belt’s existing content. The direct route that lets the vector bypass the transformations and arrive intact at the output is called a skip connection (or skip connection).

This device brings an immense relaxation to the process of learning. If a given storey has nothing new to add to a simple sentence, its neurons need only produce zero, and the information continues unhindered. Without this bypassing footbridge, the layer would have been forced to learn laboriously how to copy every bit of information faithfully without altering it.

What is more, the bypass settles once and for all the problem of the fading signal during training. When the error propagates back, it has a high-speed lane on which there is no repeated multiplication at all. Even if we stack fifty or a hundred storeys, the error-correcting signal reaches the first words clean and unaltered.

The shock absorber: layer normalisation

Because every storey adds new numerical values onto the conveyor belt, there is a risk that the numbers will grow unchecked to gigantic sizes. If a layer receives at its input values hundreds of times larger than those it was calibrated for, the mathematics becomes unstable and runs wild.

Here the mechanism of layer normalisation (Layer Normalization) comes in. At every step, before the data enter the computation, the vector is calibrated: its mean is brought to zero and it is divided by its internal variation, then finely adjusted by a learned scaling. In this way the values are brought back into a temperate and predictable range, however many layers have accumulated beneath them.

One essential architectural detail: the normalisation is applied to what enters the workshop, not to the conveyor belt itself! The main motorway stays free to grow and to accumulate nuances from every storey, but each specialist receives a nicely calibrated copy before setting to work.

The anatomy of the complete block: step by step

All these pieces form a harmonious choreography, repeated twice inside every block: first we normalise and apply attention, adding the result onto the belt; then we normalise and apply the dense network, adding onto the belt again.

In the teaching animation above, the numerical values are generated in order to illustrate the order of the operations and the stability of the magnitudes through the stages of normalisation and addition.

Stacking upwards: the storeys of deep thought

A single Transformer brick is powerful, but the real magic appears when we build a skyscraper. The second block reads the belt left by the first and adds its own reflections; the third continues where the second left off. Compact models have 12 storeys, while today’s giants stack between 40 and 100 blocks identical in structure.

This depth is not mere engineering greed. Certain linguistic inferences require several steps of logical deduction. For instance, a first storey of attention may identify the link between a pronoun and a proper name; the next storey may extract the action associated with that name from another sentence. Every storey stacked represents an extra round of thinking and abstraction before the final answer is given.

The fine mechanics that keep the tower standing

At first sight, skip connections and normalisation look like mere technical plumbing. In reality they make the difference between a network that works brilliantly and one that does not work at all. In the interactive table below you can test the behaviour of the learning signal as we add storeys.

Without skip connections, the gradient signal collapses exponentially: after thirty storeys it becomes a microscopic fraction indistinguishable from zero. With skip connections and rigorous normalisation, the signal reaches the first layer as vigorous as if it had passed through a single storey!

These invisible components do not necessarily increase the theoretical intelligence of the formulas; they make the network trainable in reality. It is a fundamental philosophical distinction in computing: one thing is what a formula can represent on a mathematical blackboard, and quite another what an algorithm can learn stably in practice.

How big the model is: the formula 12·d²

The total size of a Transformer block is computed astonishingly simply, resting on a single basic measure: the width of the feature vector, written in the literature as d.

The attention mechanism uses four square matrices (for queries, keys, values and the combining of outputs), which means 4d² parameters. The dense processing network contains an expansion matrix from d to 4d and one for the return, coming to 8d² parameters. Adding the two workshops, every block contains exactly 12d² parameters! If we multiply this value by the number of storeys stacked, we obtain with impressive accuracy the official size of the great models in the press (those of the GPT or LLaMA families, say).

And yet this almost perfect structure still lacks one elementary sense: it knows nothing of time or succession! The attention mechanism compares all the words at one stroke; if we shuffle the order of the words in a sentence, attention will produce exactly the same scores, as if looking at a disordered heap of stones rather than a living sequence. In the next lesson we shall learn how the model is given a sense of direction and of word order.