RomânăEnglish

What a Machine Knows

How does a switch come to write sentences? The course answers from the ground up, with no step skipped: from what it means to learn something, to the way an electric current holds a bit, to the machine that learns from examples and, in the end, to the models that speak.

Every lesson uses only what the earlier ones have built, and every mechanism is drawn and set in motion on the page: the boxes go dark, the code builds itself, the current runs through the switch. The figures compute for real; nothing in them is staged.

Contents

  1. Part 1Information

    1. 01The Measure of Informationbit · log₂ · binary code
    2. 02Entropysurprise · Huffman · redundancy
    3. 03Error Correctionparity · Hamming · capacity
    4. 04Sampling and QuantisationNyquist · pixels · loss

  2. Part 2The Computer

    1. 05The Switchgates · transistor
    2. 06The Addercarry · two's complement · overflow
    3. 07The Flip-Floploop · latch · clock · register
    4. 08The Processorinstruction · cycle · datapath
    5. 09The Stored Programvon Neumann · assembler · the ladder
    6. 10The Algorithmsorting · complexity

  3. Part 3Learning

    1. 11Learning from Examplesrule with thresholds · the change of plan
    2. 12The Cost Functionparameters · cost · landscape
    3. 13Gradient Descentgradient · learning rate
    4. 14Overfittingtraining · test · regularisation
    5. 15The Artificial Neuronweights · boundary · sigmoid
    6. 16The Layershidden layers · depth
    7. 17Backpropagationblame · the chain rule
    8. 18Convolutionfilter · activation map

  4. Part 4Language

    1. 19Vector Representationvectors · cosine
    2. 20Tokenisationtokens · merging pairs
    3. 21The Next Wordlanguage model · perplexity
    4. 22The Neural Language Modelvectors · shared weights
    5. 23The Recurrent Networkhidden state · the vanishing slope
    6. 24Attentionquery · key · value
    7. 25The Transformer Blockresidual stream · bypass · normalisation
    8. 26Word Orderpositional mark · sinusoids
    9. 27Training at Scalepre-training · scaling laws

  5. Part 5The Assistant

    1. 28How the Word Is Chosentemperature · cutting the tail
    2. 29From Continuation to Assistantfine-tuning · reward model
    3. 30What Happens While It Answerskey-value memory · context window
    4. 31Why It Inventsconfabulation · calibration
    5. 32In-Context Learningprompt · examples in context

  6. Part 6Images

    1. 33Noise as Materialforward process · noise schedule
    2. 34The Way Backsampling step · diffusion
    3. 35Diffusion on a Handful of Pointsa network trained on the page · the spiral
    4. 36The Text That Pulls the Imageconditioning · guidance
    5. 37Working in the Compressed Spaceautoencoder · latent diffusion
    6. 38Close Relativesadversarial network · flow matching

  7. Part 7Today's Systems

    1. 39One Space for Image and Textshared space · multimodal
    2. 40Tools and the Agent Looptool call · loop
    3. 41Search Before the Answerretrieval · vector store
    4. 42What an Answer Costsmixture of experts · quantisation · distillation
    5. 43How It Is Measuredevaluation · contamination
    6. 44What Can Be Seen Insideprobe · direction · intervention

  8. Part 8The Open Questions

    1. 45The World Modelhidden state · probe · intervention
    2. 46What “Smarter” Meansgenerality · jumps · the ruler
    3. 47The Goal and Its Measurebest of k · Goodhart · alignment
    4. 48The Ladder, End to Endfifteen stops, each made of the last