Why it persists The early evidence, mostly clinical, suggests that working beside such a system erodes unaided skill, and that people with the least earned competence defer to it most. An organisation that appoints on credentials and then deploys these systems has built a workforce disposed to defer.
What to do Name the person who answers for each judgement the system makes. Where no one can be named, the delegation was never valid, whatever the contract says.
The two preceding essays in this series, made a narrow argument. A certificate records that a process was completed but says nothing about whether judgement was formed; and an organisation that appoints on the certificate, then buys a fortnight’s course to close the gap, has chosen the least promising remedy available. If you have not read them, the summary is enough to follow what comes next.
This essay charts the same substitution somewhere else: in the machines we have started calling agents, and the early evidence on what that is costing us is now in.
We have begun handing work to systems we have agreed to call “agents”. I want to take up that word carefully, because I think it is doing something the people using it have not noticed, and because the thing it conceals is the thing for which boards and managers are supposed to exist.
The Word That Cannot Answer
I should declare an interest, in the way an expert witness declares one: my doctoral work (2023) was on Edmund Burke (1729–1797) and the delegation of authority, so while I am not a neutral party on this word. I am, I hope, a useful one.
In computer science, ‘an agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators’. That is a term of art, set out by Stuart Russell and Peter Norvig in the textbook that is used widely by many people in the field (Russell & Norvig, 1995/2022, p. 54).
It is a functional definition: it describes what goes in and what comes out, and says nothing whatever about whether anyone is home. Nobody using it that way is being deceptive. When the first edition appeared in 1995 that reticence was a virtue, because the question was unanswerable and engineering did not need it answered.
Thirty years on, the same silence is an asset, and it is being sold.
There is a familiar philosophical objection here which I will move past quickly, because it is less useful than it looks. Daniel Dennett (1942–2024) pointed out that treating something as though it has intentions is a perfectly good shortcut for predicting what it will do (Dennett, 1987/2002). We do it with chess engines, and most of us do it with the office printer. But the shortcut belongs to us, not to the thing. Calling the office printer spiteful does not give it a view about your deadline.
The objection that matters is not philosophical but constitutional, and it is the one that ought to concern anyone who sits on a board.
In law, and in the older political vocabulary, an agent is a trustee. Someone who exercises judgement on another’s behalf and who answers for having exercised it. When Burke stood before the electors of Bristol in 1774, he told them that their representative owed them not his industry only but his judgement, and betrayed them if he surrendered it to their opinion (Burke, 1996). The claim is stronger than it first sounds. Judgement is the thing being delegated. Answerability is what makes the delegation intelligible. Remove either and what remains is not an agent but an instrument.
The difference here is not academic, and it is easy to miss because the word is overloaded. Security architects speak of a trust relationship between systems—one endpoint configured to accept another’s assertions without further challenge. That is a statement about configuration, not about answerability. An instrument can be given tasks, and it can be trusted in that narrow sense. It cannot be given trust in the Burkean sense, because that trust is a relation with something that can be called to account, and you cannot call a tool to account any more than you can cross-examine a spanner—that it fits the nut is a design outcome, not a relational one.
So the equivocation runs like this. Agentic systems are sold on the technical definition and bought on the constitutional one. The purchaser hears initiative, discretion, judgement. The vendor has supplied percepts and actuators. And when the arrangement fails, as delegations do, the accountability has nowhere to land, because once set in motion, nothing in the chain was ever capable of answering for anything.
That is not a new problem wearing new clothes. It is the oldest governance failure there is, and we have just industrialised it.
Member discussion
The discussion is open to members of the Commons. Join, free, or sign in to take part.