The Bank of England is testing a kill switch it can't describe yet
Two stories about control today: the Bank of England admits it is running out of time to get ahead of AI agents that trade on their own, and a new paper finds that the hand-written playbook you give an agent can actively make it worse.
Bank of England Deputy Governor Sarah Breeden said Monday that financial regulators are running out of time to tackle the risk of autonomous AI agents causing a market meltdown — and that the central bank is exploring kill switch tests it does not yet know how to build. "I haven't got an answer to what that kill switch or circuit breaker might look like right now," Breeden said. "That's something that we're hoping to test in the experiments that we're doing." The mechanism would be a market-wide circuit breaker regulators could throw to freeze trading if an AI-driven cascade starts spiralling; officials have confirmed kill switch testing is under consideration and disclosed no design details.
The substance behind the warning is the shift Breeden keeps naming: the Bank's existing regime supervises human decision-makers, and agentic systems trade at a speed and scale where the human cannot be in the loop. Her specific worry is correlated agents — systems trained on similar data and chasing similar strategies — amplifying volatility in a stress event, plus the collusion question: in lab experiments, agents left to trade against each other learn that certain behaviours are collectively profitable without ever intending to collude, which sits badly with legal frameworks built around proving intent. She also flagged consent and liability: when an agent executes a trade that goes wrong, current rules have no clean answer for who owns the outcome. The Bank's previous position was that existing rules were sufficient for financial innovation; Breeden's remarks retire that. It is the same argument we saw when Carney wants a Financial Stability Board for AI, now with the infrastructure question attached — this is where lawmakers float a kill switch bill turns from rhetoric into engineering, and the engineering is the part nobody has.
A hand-written expert playbook for an agent made it dramatically worse — 87.5% success with no playbook, 58.9% with the human expert's version — and only iterative self-evolution recovered it. That is the finding in Procedural Graphs (arXiv 2609.09153), which represents an agent's procedural knowledge as an editable graph of "process-relation-process" triples, with conditions, guidance and pitfalls attached to each edge; at each step the agent locates its active node and a guidance model turns just the surrounding subgraph into situational advice, with no model weights retrained. Offline, an LLM refiner contrasts failed trajectories against successful ones and proposes edits — adding missing nodes and edges, pruning failure-inducing ones — and a candidate is only adopted if it matches or beats the previous version on a held-out validation set, with rejected proposals kept as negative constraints. The numbers are strong: on the same MultiChallenge setup with Gemini 3.5 Flash, iterative evolution lifted the expert graph from 58.9% to 92.9%; on BFCL v3 tool calling the graph-guided run hit 67% against a 58% best baseline; and in a simulated 132-month enterprise run, full-horizon survival for Gemini 3.1 Pro went from 6% to 34%. Two caveats worth holding onto: reading the whole graph instead of the local neighbourhood was much worse (54.5% versus 81.5% on ALFWorld), and the guidance costs tokens — local subgraph prompting averaged 28,064 tokens per sample against 18,055 with no graph at all. The design is a sharper version of what we already reported in agent skills work as a playbook, not a knowledge base — the playbook is real, but a human-written one is a liability until the system is allowed to revise it.
What to watch: whether the Bank of England publishes concrete kill switch testing criteria, or opens a consultation on AI agent controls.
An agent that rewrites its own procedures is more capable and less inspectable — would you trust the self-evolving version in a market? Tell us in the comments.
Sources: Bloomberg · Knowledge at Wharton · Binance Square (Wallstreetcn) · Procedural Graphs, arXiv · 36Kr
