The Stored Program

One single memory

Before modern computers were born, computing machines had their “program” cast directly into cables and physical switches. To change the task of a colossus such as ENIAC — to move from computing the trajectory of a shell to forecasting the weather, say — a whole team of engineers had to unplug hundreds of thick cables, rearrange them on huge panels and turn hundreds of switches by hand, a manual labour that took days. In those times a program was not a text you typed but a physical scaffolding that had to be rebuilt for every use.

The great turning point came in 1945, with the famous report of the mathematician John von Neumann on the EDVAC project. The idea of the stored program is disarmingly simple: why treat commands as fixed wiring when we can write them as plain numbers and place them in the same common memory where we keep the data? Once that principle is adopted, reconfiguring the machine no longer asks for pliers and wire, only for the rewriting of a few bytes in the memory boxes — an operation the processor can carry out by itself in an instant.

The consequence of this unification goes far beyond mere technical convenience; it is a philosophical revolution. A hammer stays a hammer for ever; a lathe can only be a lathe. But a computer with a stored program has no identity nailed down. Load the rules of chess into its memory and it becomes a chess master; load a ledger and it becomes a tireless accountant; load a sound synthesiser and it becomes a musical instrument. It is the first object in the history of human civilisation whose innermost nature is precisely to have no fixed nature: it becomes whatever machine the program stored in its memory dictates.

The same number, three readings

If we glance inside a memory chip, a byte comes with no label saying what it stands for. Eight bits of silicon are just eight transistors either under voltage or not. The same string of eight bits can be seen as a whole number, as an instruction to add, as a letter from a novel, as a shade of colour on the screen or as a musical note in a song. The meaning does not lie imprisoned inside the byte; the meaning depends entirely on the eye that reads it and on the context in which it is called.

Eight bytes, read in three ways at once

When the program counter (PC) stops at an address, the processor blindly treats the byte there as an instruction to carry out. When an addition operation reads the same address, it treats the byte as a number to reckon with. In the memory itself nothing at all changes. That is why, if a program goes astray by accident and starts reading a photograph as though it were a list of instructions, the processor will try in all seriousness to execute the pixels of the image, reading the colours as orders to compute until it locks up in an error.

In the interactive figure above, the letters follow the convention we settled on in the first part of the course: the first five bits pick a character out of an alphabet of 32 signs, while the first three bits — which in an instruction are the operation code — stay invisible to the text.

Programs that write programs

Here is a spectacular consequence of the von Neumann architecture: if programs are only strings of numbers, and computers are expert at handling numbers, then a computer can handle, generate and alter other programs. This reflexive loop — programs that build other programs — is the secret engine that allowed the whole modern digital world to grow.

Nobody today programs by typing agonising strings of 0s and 1s. The first step to freedom was the creation of a text with human abbreviations that are easy to remember ( LOAD, ADD, JUMP, for instance). That text is called assembly language, and the program that goes through it and translates it line by line into the binary code of the machine is called an assembler. The translation is direct and rigorous: to every line written by a human corresponds, with clockwork precision, one binary byte understood by the processor.

Write instructions with names; the numbers come out by themselves

what it showed

—

But the ascent does not stop at the assembler. One step higher we find the the compiler: a far more sophisticated program, able to read high-level languages (such as Python, C or JavaScript), in which we write elegant mathematical concepts, logical conditions and natural formulas. The compiler takes these abstract human ideas apart and translates them into thousands of small binary instructions for the machine. And the compiler is itself a program once written by another compiler, building a dizzying ladder of technical intelligence.

A program can write its own instructions

Since instructions share the same living memory with data, nothing stops a program from computing a value and saving it right over its own future lines of code, changing its behaviour as it goes. In the dawn of computing this technique of “self-modifying code” was used intensively to save the tiny memories of those days: to go through a long list, the program altered at each turn the address written in its own reading instruction.

In modern systems the practice is viewed with the greatest caution, for two pragmatic reasons. First, today’s processors anticipate future steps and preload instructions into an ultra-fast store (the cache); if the program suddenly rewrites the memory behind it, the whole store must be thrown away, which destroys performance. Second, a program’s ability to rewrite its own code is the dream of every computer virus. For that reason modern operating systems strictly separate the regions of memory, forbidding writing into the space reserved for instructions.

The spirit of the idea lives on triumphantly in another guise, though: the “Just-In-Time” (JIT) compilers that give life to contemporary web browsers analyse JavaScript code while we browse the internet, generate fresh machine instructions right there in memory and execute them on the spot at astonishing speed.

The ladder

Look back at the road we have travelled in this second part of the course: we have raised an impressive ladder of nine conceptual rungs. Every rung is built entirely on the shoulders of the one below, but has the wonderful gift of hiding the complexity underneath completely. The engineer who writes an algorithm need not think about transistors, and the one who designs the adder need not know what program will run on it in ten years. This separation into protective layers is called abstraction — and it is the only reason the human mind can master systems made of tens of billions of parts.

From the bottom up; click any rung

Every rung is built out of the rung below it and needs nothing from further down.

The ladder goes on rising far above the edges of this lesson: on assembly languages were built compilers, on those operating systems, planetary networks and vast platforms of distributed computing. A gigantic pyramid of ideas, at whose modest base sits, as quiet and obedient as on the first day, a simple switch that either does or does not let an electric current through.

The machine we have perfected so far is a marvel of deterministic logic: it is obedient, fast and never wrong if the recipe is right. But all its power depends on one enormous condition: that an intelligent person has written down beforehand, step by step, the necessary algorithm. What are we to do, then, when we meet problems that any child solves in a fraction of a second but for which no mathematician can set down a mechanical recipe — recognising a mother’s face in a photograph, say, or understanding a joke? Before we come to that great riddle of artificial intelligence, we must settle the last great concept of the classical world: what an algorithm actually is.