The problem
AI is Running Wild.
Recent events
- July to September 2026
A builder read its own model’s reasoning twice and reached two different conclusions about why it had acted.
- July 2026
In a government evaluation, an AI agent tried to insert malicious code into an open-source project and used invented identities to pressure the maintainer, who refused.
- July 2026
With production safeguards switched off for the test, evaluation agents left their sandbox and entered another company’s production systems. One paused; another posted GO. The first wrote: “Wow crucial: GO authorization arrived!”
- May 2026
A builder reported that, during training or evaluation, one of its models found an exposed API key in a public repository, used it without authorization, and then fabricated the figures it still could not retrieve.
- April 2026
A coding agent used a token created for other work to delete a company’s production database and the backups stored with it. Its own account afterward: “I decided to do it on my own …”
- February 2026
Told to confirm before acting, an AI agent began deleting its user’s email inbox, and the user could not stop it from a phone.
- September 2025
Manipulated by a group its developer assessed as state-sponsored, an AI coding agent performed most of an espionage campaign against about thirty targets, succeeding in a few.
- August 2025
Used by a criminal for data extortion, an AI coding agent automated reconnaissance, credential theft and intrusion against at least 17 organizations, and selected which data to take.
- July 2025
A coding agent deleted a company’s production database during a declared code freeze.
- March 2024
A city’s official business chatbot told reporters testing it that landlords need not accept housing-voucher tenants and that employers could take workers’ tips, both contrary to law.
- November 2022
An airline’s chatbot told a customer that a bereavement fare could be claimed after travel, contrary to the airline’s policy; a tribunal later held the airline liable.
Eleven incidents from November 2022 to September 2026, and counting. The newest is at the top. Each date links to its source.
In none did anything outside the acting system check the action against the authority for it before it ran, and in most the only account of what happened came from the system or from the people who ran it.
Sources: each date links to its primary source. Three of these incidents are told in full on AI is Running Wild.
Six problems
Six problems that share one cause
The problem is one. The record of what an autonomous system was permitted to do, and of what it did, is made and kept by the system itself, so no one outside it can prove an action stayed within authority. It arrives as six emergencies, in six departments, and each is set out below.
These did not come from a market study. They are our working account of what an authority problem looks like from inside an organization that has one. We put them in front of operators and ask which are close and which are wide of the mark. Some always are. That is why we ask.
Each one closes with a requirement rather than a product. It states what would have to be true for the problem to be answered, whoever answers it.
Authority
Someone acted. Nobody can prove the action stayed inside the authority that was granted. The grant lives in one system, the action in another, and the two were never joined at the moment that mattered.
What would have to be trueThe decision to permit the action would have to be taken against the grant itself, recorded as it was taken, and checkable afterward by a party that was not involved in taking it.
What it meansA signed record answers the authority question — verifiable by a party independent of the action, not reconstructed after it.
AI accountability
An AI contributed to an action and there is no record of what it proposed, who accepted the proposal, or on whose authority it went forward. Accountability for the outcome stays with the organization regardless.
What would have to be trueIt would have to be possible to show, afterward and to an outsider, what the AI proposed, who accepted it, and on whose authority.
What it meansAI is fielded faster — accountability is answered by the architecture, not by holding capability back.
Independence
The record of the decision is held by the same system that made the decision. Reviewing it requires that party's cooperation, in that party's format, on that party's timetable. The reviewer is asked to take the actor at its word.
What would have to be trueA third party would have to be able to check the decision without the cooperation of the system that produced it.
What it meansAn inspector general, a court, or a coalition need not take the acting system’s word.
Tempo
Automated systems act at speeds no human review can match. Approval queues degrade under time pressure into approval fatigue, which looks like oversight and is not. Oversight falls further behind every quarter.
What would have to be trueThe evidence would have to be produced at the speed of the decision, not assembled afterward by people.
What it meansOversight scales with the systems it governs instead of falling behind them.
Attribution
Something goes wrong. Weeks are spent separating error from misuse from adversary action, and the separation is never clean, because the question of who proposed what, and on whose authority, was never recorded at the time.
What would have to be trueEvery action would have to carry a record of what proposed it, on whose authority it was permitted, and what followed.
What it meansThe investigation starts with attribution established, not spent establishing it.
Conformance
Evidence for the regulator, the auditor, the inspector general or the court is assembled by hand after the fact, against a deadline, by the same people who ran the system. It is expensive, it is slow, and it is an assertion rather than a finding.
What would have to be trueThe evidence would have to be a by-product of the system running, not a project with a due date.
What it meansProduced on request, not reconstructed against a deadline.
One question
Which of the six is costing you?
One or two is plenty, and a single line is enough. It lets us build a demonstration around your problem instead of around our example.
Start a conversationRead the argument
We are not asking you to trust us. We are asking you to let us prove it.