Tools and the Agent Loop

What comes out badly from a single pass

Exact arithmetic with several digits runs against the fundamental nature of a language model. A neural network holds no internal silicon registers and cannot remember “the carry held in mind”; it predicts the next token of text on the basis of learned statistical associations. If we ask it to solve 47 × 83, it is forced to hit all four digits of the result directly, in a single pass.

The experiment below compares the same small model, trained in two different ways. In the first case the model tries to compute by itself and to emit the digits of the answer directly. In the second case we no longer ask it to do arithmetic, only to identify what is being asked: the mathematical operation and the two numbers involved. The actual computation is taken over instantly by a small classical program on the page.

The contrast speaks for itself: while simple additions can be memorised through frequent association, multiplication fails without an outside tool. Look at the right-hand column: the model often guesses the order of magnitude or a few digits, but almost never all four at once. And in mathematics, a result with a single wrong digit is entirely wrong.

The second kind of training does not make the network cleverer at arithmetic; it gives it a task suited to its nature: a problem of language. The model reads the sentence, understands the person’s intention and emits a structured command. This is the essence of any architecture with tools: we move the burden from what the network does unreliably to what it masters remarkably well — understanding and dialogue.

What the request looks like

But how, in practice, does the model communicate with an outside tool? The request is not a free conversation but a text formatted rigorously in a settled convention. When the network recognises that it needs a computation, it generates this specific format, which the surrounding program intercepts, decodes and sends to the tool. This request is called a tool call (tool call).

The result computed by the tool comes back as text and is added to the end of the conversation. From the model’s point of view, this new information becomes an ordinary piece of context — a fresh note on its working pad, from which it can take up the thread of its thinking again.

The program that coordinates this continual to-and-fro — the model writes an intention, the outside program carries it out, the result comes back onto the page, and the model reads and continues — is called a loop, and the whole articulated system is called an agent. The word may sound spectacular, but the mechanism is clean and pragmatic: a band of text that lengthens step by step.

The requests on the interactive band are issued by the model trained above, and the computations are carried out in real time by the browser. The difference from a fully autonomous agent is that here the succession of steps is suggested by the structure of the question; in an advanced system the model decides by itself which tool to invoke at each step, but the dynamics of the loop stay the same.

What the turning costs

Every iteration of the loop adds new text: the command written by the model, the answer produced by the tool and the new inferences. The working pad thickens continuously, and the model must keep the whole history in its attention at every step.

Here we rediscover the importance of the key-value memory (KV cache), studied when we looked at generating text. Without this fast memory, at every turn of the loop the computer would have to reread all the pages accumulated before from the beginning, and the computing effort would grow with the square of the number of steps. Thanks to this optimisation, every step pays only for processing what is new and for attention spread over what has already gathered.

Where it breaks

It is essential to remember that the agent loop confers no magical intelligence on the network; it only gives it access to the outside world.

The model has no infallible sense for telling whether the data received from a tool are correct; it reads them like any other text and builds on them. If the initial request was vague or wrong, the result returned will be just as faulty — but it will wear the coat of an official answer. And from that moment on, the whole later logical scaffolding rests on a crack.

A major vulnerability of any autonomous system is that the mistakes add up. Even if every individual step has an excellent success rate of 90%, after a chain of ten consecutive steps the probability that the whole mission comes out perfectly falls below 35% (0.9 multiplied by itself ten times comes down fast). That is why agents carrying out complex plans need checks, confirmations and self-corrections at every turn, not only at the end.

In practice the most used tool is not a pocket calculator but a system able to search for information in huge libraries of data and to lay it on the model’s working table before it formulates an answer. In the next lesson we shall explore precisely that technique — retrieval-augmented generation (RAG) — a genuine open-book examination.