Signed, Sealed, Self-Policed

Or: How Marking Your Own Company Homework is a Policy

When I wore the white cap of the RAF Police, one of the first truths you absorbed, usually before your boots were properly broken in, was that nobody has ever written themselves a ticket. Not the junior airman with the dodgy tax disc, not the officer parked across two bays outside the mess, and certainly not the station commander, whose car appeared to exist in a legal dimension all of its own. People follow rules reasonably well when someone else is holding the notebook. Hand them the notebook and something remarkable happens to their eyesight.

I thought about that notebook a great deal on Tuesday evening, reading the news from Washington in the cave, on a phone charged by a solar panel that works harder than most governments. Trump announced that he and a large group of AI company leaders had signed a voluntary accord, one that will include internal and external reviews of the technology, following a meeting at the White House. His verdict on the arrangement was that “they understand that they have to self-police”, which is the sort of sentence I’d have underlined twice in a witness statement. audacyaudacy

The timing deserves its own paragraph. Earlier that same day, he had promised the US government wouldn’t limit the development of artificial intelligence or support calls for guardrails or other restrictions, and he did it while unveiling America.gov, an AI chatbot meant to help Americans navigate government services. So the morning was no rules, and the afternoon was a signed promise to follow some rules, voluntarily, as judged by the people following them. Somewhere between lunch and the press gaggle, the notebook changed hands. audacyaudacy

And what a lunch it was. Nvidia’s Jensen Huang sat at Trump’s right, according to a seating chart the president posted on Truth Social, with Musk on his left, and Sundar Pichai, Mark Zuckerberg and Satya Nadella also at the table, alongside Anthropic’s Dario Amodei, OpenAI president Greg Brockman and Jeff Bezos. Publishing your own seating plan is a very particular flex. It tells you who matters, in what order, and who gets to pass the salt. audacyaudacy

When asked about oversight, Trump pointed out that the country automatically has regulation through the Department of Justice and the FBI. I spent a career on the enforcement side of the fence, so let me gently explain the difference. Regulation happens before the harm. Law enforcement arrives after it, with a clipboard and a sympathetic face. Calling the FBI your AI regulator is like calling the coroner your GP… technically they’ll both take an interest in your health, but only one of them is any use to you. audacy

What makes this more than a routine piece of Washington theatre is who was sitting in the room and what they’d been saying. Many of the executives in Washington had called for a slowdown after several high-profile incidents of AI agents going rogue. Amodei had called earlier this month for leading firms to slow down, and Musk and OpenAI’s Sam Altman also called for safety measures to be given time to catch up. The day before the lunch, OpenAI halted the rollout of a new model over safety concerns. Meanwhile, Huang and Zuckerberg have played down fears of runaway AI and stressed that each company is responsible for the safety of its own products. So the table held the people pulling the fire alarm and the people insisting it’s only toast, and they all signed the same piece of paper. I’m not sure what that paper says, but I’d love to know what it costs to break it. Trump vows to never ‘stifle’ AI as leading executives urge caution +3

Because that’s the question nobody seems to have asked out loud. The accord includes external reviews, but the reporting doesn’t say who the external reviewers are, what powers they have, or what happens when a company fails. In my old world, an inspection you invite, schedule and can politely decline isn’t an inspection. It’s a visit. You put the kettle on, you tidy the front room, and you close the door to the spare bedroom where the real mess lives.

Then there are the data centres, which is where the abstract becomes very concrete, very fast, and usually very thirsty. Trump said the executives had agreed to soften the bipartisan opposition to data centres springing up in local communities by giving more financial support to local schools or taking steps to cut energy costs. And if that doesn’t work? He suggested that if the opposition carried on, the companies would be forced to go overseas or elsewhere. Read that twice. Nobody is proposing to address the water, the power draw or the noise. The offer is to pay for the objection, with a gentle gesture towards the exit if the objection won’t stay paid for. I’ve written before about what these places do to aquifers, and living where I live, with every litre accounted for, I can tell you a new roof on the primary school doesn’t refill a well. audacyaudacy

The final ingredient is the race. Trump framed AI development as largely a contest between the US and China, with America in front, and said there would be a winner and a loser with no second place. The two countries agreed last week, during Xi Jinping’s visit to Washington, to set up a channel for handling AI-related incidents, but Trump stressed he didn’t want to cooperate too much, asking why he’d want to do anything when America is leading. I grew up in a service family during the Cold War, some of it in Germany, some in Norway, close enough to the fault line to feel it hum. The adults around me understood something that seems to have gone out of fashion: the Washington to Moscow hotline wasn’t a sign of weakness. It was an admission that the race itself was the dangerous part, and that the winners would still need a way to say “hang on, that wasn’t us” before the other side did something irreversible. Race logic turns every safeguard into a handicap. It’s a very efficient way of making sure nobody breaks. audacyaudacy

In fairness, and I do try, there’s a serious case on the other side. Plenty of sensible people argue that lawmakers barely understand the technology, that heavy regulation would freeze it in place while less scrupulous rivals carry on regardless, and that the engineers building these systems are better placed than a committee to spot the dangers. A signed accord is more than nothing, and more than existed last week. If you believe the companies are genuinely frightened by what they’ve built, and some of them say they are, then asking them to hold each other to account isn’t absurd.

But voluntary is doing a lot of heavy lifting in that sentence. A voluntary commitment lasts precisely as long as it’s cheaper to keep than to break, and the sums being spent here are measured in hundreds of billions. There’s also the small matter of how this administration treats the companies themselves. In February, it ordered every US agency to stop using Anthropic’s technology and imposed other major penalties, after months of negotiations broke down over two exceptions the company wanted on the lawful use of its model: mass domestic surveillance of Americans and fully autonomous weapons. Just two weeks before Tuesday’s lunch, Trump lashed out at Amodei’s call for a slowdown, and told the country “Don’t kill the Golden Goose!” And now there’s a seat at the table. You can think whatever you like about any of the firms involved, and I’d encourage a healthy scepticism towards all of them, but that isn’t a regulatory framework. It’s patronage. You’re in favour or you’re out, and the rules shift with the mood. Rules that bend to the mood of one man aren’t rules at all. They’re weather. Trump orders US agencies to stop using Anthropic technology in clash over AI safety +2

In my old job, the signature at the bottom of a memo was where accountability began. You signed, and from that moment your name was attached to what came next. On Tuesday, a room full of the most powerful people in technology signed something too. I’m still waiting to find out whose name gets attached when it goes wrong, and I have a nagging feeling the answer is going to be ours.

Until Next Time

Dominus Owen Markham