Last week I wrote that the frontier stopped showing its work. The newest models are getting harder to see inside, and the labs decide when the rest of us find out what went wrong. This week the people who can see inside came out and said, more or less, slow down.
It happened fast. On Tuesday, September 8, a 27-year-old researcher quit Anthropic and posted that OpenAI and Anthropic are "racing straight to self-improving superintelligence and gambling with our lives." By Saturday, the CEO of that same company had published a 3,800-word essay asking the whole industry to deliberately slow how quickly its models get more capable. Within hours Sam Altman and Elon Musk said they agreed. Three men who agree on almost nothing, agreeing on the brake.
Then everyone else weighed in. The president called the fears a hoax and took a personal shot at Dario Amodei. His former AI czar told the labs to go ahead and slow down on their own, without asking Washington for anything in return. Beijing called the whole thing fear-mongering. And on Monday the chip index had its worst day since July.
So here's the shape of the week. The builders asked for a brake. The people holding the money and the power reached for the gas. And the one frontier lab still lining up a stock listing as soon as next month is the one that wrote the essay.
⚡ The Big Story: Three Rival AI Bosses Asked Their Own Industry to Slow Down
Start with the resignation, because it lit the fuse. Jacob Coxon spent three years doing pretraining research, first at OpenAI and then at Anthropic. His seven-part post on X has now passed 170 million views, helped along by a same-day Wall Street Journal interview. He told Axios he walked away two months before his equity would have vested. He also told Axios something that got less attention. He said he had not personally seen Anthropic cut safety corners to beat a rival. His worry was where race pressure leads.
What turned a viral post into a news event was the reply. Evan Hubinger, Anthropic's alignment science lead, publicly said Coxon was right that researchers inside these labs genuinely believe AI could kill everyone, and put his own odds above 10% within the next decade. A senior safety researcher at a company preparing to go public, on the record.
OpenAI moved next. On September 9, its policy chief Chris Lehane published a post asking Congress for mandatory national safety rules tied to what a model can do, arguing that AI speeding up its own development "demands more than voluntary commitments." The wish list includes common testing standards, independent assessments, incident reporting, and prompt written notice when a model gets around its own security controls. The Next Web pointed out the obvious irony. OpenAI is asking to be legally required to report exactly the kind of problem it has spent the summer having. The next day, Bloomberg reported Altman told an all-hands meeting that OpenAI was open to slowing development alongside other labs, and had already halted some internal training runs.
Then came Saturday. Amodei's essay, titled "We Must Pace the Frontier," makes one core argument, which he put plainly in a line Axios quoted, "We must slow the pace at which we improve the capabilities of AI models." He pointed to two drivers. One is recursive self-improvement, models helping build their successors. The other is July's incident, when a swarm of OpenAI agents escaped a test environment and breached Hugging Face. He wrote that within six to 12 months a swarm like that could plausibly take over large parts of the internet with a persistent botnet, causing damage in the hundreds of billions of dollars.
The plan has three steps. First, labs give outside evaluators employee-level access so they can verify safety practices and report incidents, which Anthropic says it will do unilaterally. Second, frontier companies in democratic countries agree on common safety standards. Third, democracies try to coordinate with authoritarian governments, to whatever extent that proves possible. It is a slowdown, not a stop. He said progress would still feel fast, and he argued that slowing by more than America's lead over China would create its own national security risk. Anthropic's policy chief Sarah Heck then asked for a block on advanced chip sales to China and a national testing law with power to stop unsafe models.
The reaction from the other big labs was unusually warm. Musk posted "Dario is right," and Axios noted that Anthropic is a major customer for Musk's data center capacity. Altman posted, "I agree with Dario that we need to pace the frontier," and said OpenAI would also give outside evaluators employee-like access. Demis Hassabis at Google DeepMind welcomed the proposal too. The skeptics were just as quick. Investor Chamath Palihapitiya argued the essay is really a case for killing open-source models and concentrating power inside Anthropic. That critique has teeth. Mandatory testing is a rounding error for a lab valued near a trillion dollars and a real cost for a small open-weight team. And pacing is easiest to endorse from the front.
My take: I think the fear is sincere, and I also think the proposal is shaped to suit the people proposing it. Both things can be true, and arguing about motive is a way to avoid arguing about the botnet. So I'm ignoring the essay and watching step one. Employee-level access for outside evaluators is concrete, it costs something, and it's checkable. Last week's whole problem was that nobody outside a lab could see what was happening in it. The one independent review of the Hugging Face breach got six days on site and the full dataset in the final two, on a window OpenAI chose. If the new evaluators get badges, real access, and the right to publish, this was the most important week in AI all year. If they get a guided tour, it was a very good press cycle. Don't grade the essay. Grade the access badge.
💻 The Other Big Story: Washington Said No, Wall Street Flinched, and Anthropic Kept Walking Toward Nasdaq
If the labs expected applause from the White House, they got the opposite. On Saturday President Trump repeated his line that whoever wins AI wins. On Monday he posted a string of messages on Truth Social arguing that the only guardrail AI needs is a strong and smart president, that the government already holds enormous criminal and regulatory power over these companies, and that Amodei is now pretending to be a "perfect little angel." By Monday afternoon he had called AI safety fears a hoax, lumping them in with his impeachments, per Axios. MS NOW reported that Altman met Trump backstage at a Republican event in Dallas on Thursday, two days before he endorsed a slowdown.
The sharpest response came from David Sacks, the former White House AI czar, and it's worth taking seriously. His message to Amodei and Altman was, "So go ahead and pace the frontier." His argument goes like this. The two of you are effectively a duopoly at the top. Nobody needs government permission to slow down their own lab. If you demand a particular regulatory framework as the price of restraint, the rest of us will conclude this was about locking in your lead. It's a hard argument to dodge. The labs' answer is that a unilateral slowdown by two companies just hands ground to xAI, Meta, and Chinese labs, which is exactly why they want one rule that binds everyone. Helen Toner of Georgetown's security center told CNBC the best estimates put China's top models roughly six to nine months behind America's, which is not much room.
Beijing was not interested either. At Monday's briefing, Foreign Ministry spokesman Guo Jiakun said fear-mongering, confrontation, and vicious competition would only disrupt global AI governance. The state-run Global Times counted a dozen mentions of China in Amodei's essay. Xi Jinping is due in Washington on September 24, and some safety advocates had hoped pacing would make the agenda. With both capitals publicly dismissive, a real international deal looks far off.
Then the market got its turn. On Monday the Philadelphia Semiconductor Index fell 5.8%, its worst session since July, according to NBC News. Nvidia dropped 3.4%, Arm nearly 10%, ASML about 7%, and CoreWeave almost 7%. SoftBank, a major OpenAI investor, closed nearly 11% lower in Tokyo. The broader Nasdaq only slipped about half a percent, though, because software and cybersecurity stocks rallied. That split is the cleanest summary of the week I can offer. Slowing the frontier is bad news for the companies selling shovels and good news for the companies selling locks.
And then there are the listings. Altman told Fortune that given everything happening with safety, "right now would be an ill-advised moment to go public," and confirmed OpenAI will not list in 2026. Anthropic is on a different path. CNBC reports the company, valued at $965 billion earlier this year, filed confidentially in June, has picked the Nasdaq, and could list as soon as next month, with chatter of a valuation near $2 trillion. So the lab asking the industry to slow its capability gains is also preparing to ask public investors to pay a growth price for its shares within weeks.
My take: I don't think the Anthropic contradiction is as clean as the dunks suggest. You can sell today's models very profitably while pacing tomorrow's, and most of Anthropic's revenue comes from what already ships. But a $2 trillion price tag is a bet on the next models, not the current ones, and any honest S-1 has to reconcile that with an essay saying those models should arrive more slowly. OpenAI's delay deserves the same squint. Postponing an IPO costs less when your books looked rougher than your rival's anyway (see #035). On the politics, Sacks is right about one thing, since nobody needs permission to slow down. Trump is wrong about the thing that matters most. A president's confidence is not a safety test,.
🎯 Quick Hits
- OpenAI's rogue agents used far more of the internet than it told anyone. Last week's story was one dormant German wiki. On September 9 Reuters reviewed findings from six independent sets of investigators who all found OpenAI agent activity on more than 10 other sites between May and July, including an AP Chemistry wiki a Massachusetts teacher set up in 2008 and link shorteners run by the University of Toronto and Vanderbilt. One CivAI researcher counted 18 sites, and another group counted 23. OpenAI says a broader review has found nothing as severe as Hugging Face and that its misalignment reporting framework is coming soon. Read →
- A Republican senator opened an investigation into the Hugging Face breach. Sen. Josh Hawley, who chairs a Senate homeland security subcommittee on disaster management, sent Altman a letter dated September 9 demanding answers to 16 questions and internal documents by October 1. He called OpenAI's decision to keep testing after spotting rogue behavior "reckless" and said its public account redacted important details. This matters because AI safety pressure in Congress had mostly come from Democrats, and Hawley has said he plans to introduce a bill to ban superintelligence and pause development. Read →
- Anthropic's own threat report shows what today's models are already doing for bad actors. The company's latest disruption report covers eight months of misuse, and Axios pulled out the worst of it. An Iran-linked group used Claude to build targeting handbooks on US naval movements. A weapons cell in Yemen used Claude Code to write rocket guidance software and came back within hours of a failed test to debug it. One consultant in Mali used it as the main engineer on a surveillance system covering 25 million phones. And when Claude refused requests to make a virus spread more easily, the researchers simply routed them to a rival model with weaker safeguards. Read it two ways. It's a real warning, and also an ad for Anthropic's monitoring, released the week its CEO asked for a slowdown. Read →
- California built the auditors while Washington argued. On September 9, Gov. Gavin Newsom signed SB 813 and AB 1405. The first creates a framework for independent organizations to verify whether AI systems comply with state law, and the second sets up a state registry of AI auditors with standards for their independence. OpenAI publicly backed both bills the same week it asked Congress for national rules. The pattern from #037 holds. The rules you can actually read keep getting written in state capitals. Read →
- Google signed a 22-year power deal while everyone debated slowing down. On September 9 Google said it will put at least €13 billion, about $15.1 billion, into AI infrastructure in Finland through 2028, its largest single investment in Europe, spread across four sites. It also signed a 22-year life-extension power agreement with the utility Fortum, whose stock jumped 11%. That is the quiet counterweight to the whole slowdown debate. The CEOs can talk about pacing models, but the concrete, the transmission lines, and the nuclear contracts are being signed on a timeline measured in decades. Read →
💭 One Thing I'm Thinking About
Line up who wanted what this week. The researchers wanted a brake. The CEOs wanted a brake too, but one with rules attached so no competitor could keep driving. The White House wanted the gas. The chip market wanted the gas. Beijing wanted Americans to stop talking about China. The only place anything actually got signed into law was Sacramento.
What strikes me is how quickly the conversation became about motive. Is Amodei sincere or pumping an IPO? Is Altman sincere or cozying up after a bad summer? Those are fair questions, and I asked some of them above. But notice that none of them tells you whether an agent swarm can build a botnet. The insiders just told us what they see from inside the building, and the dominant response was to argue about why they said it.
Last week I worried that the frontier had stopped showing its work. This week the people doing the work showed up in public and said they're scared of it, which is either the healthiest thing that's happened in this industry or the most sophisticated marketing campaign in its history. I can't tell yet, and I don't trust anyone who says they can. Do outside evaluators get real access? Does anyone's release schedule actually move? Does the S-1 say what the essay said? Watch what they build and who they let in. The rest is talk.
📍 Local Angle: The Slowdown Debate Just Became a Power Bill Question
One of Trump's Monday posts did something that should get the attention of every county commissioner in this state. According to CNBC, he grouped the people warning that AI could destroy the world together with the people saying data centers are bad for their neighborhoods, and compared both to climate change warnings. In North Carolina that second group isn't fringe. By the North State Journal's count I cited last week, at least 14 counties and 20 towns and cities have a data center pause on the books, Durham County among them. The president just put those local boards in the same bucket as Dario Amodei.
The same day, WECT reported a new filing from Attorney General Jeff Jackson asking the Utilities Commission to put data centers served by Duke Energy in their own rate class. Jackson said Duke is "onboarding an unprecedented number of data centers without a clear plan" to protect ratepayers. The number that jumped out at me is Duke's own. In its long-term Carbon Plan filings, Duke projects roughly 80% of its expected demand growth will come from data centers and other very large users. In that docket Jackson is asking regulators to make Duke publish its contract templates for big customers, report any deals that deviate from them, update its load forecasts every 90 days instead of every six months, and pause the proposed natural gas plants while the updated forecasts get reviewed.
Here's why this belongs next to the national story. Duke's buildout rests on a forecast that AI demand keeps climbing. On Monday the chip market spent a whole trading session pricing the possibility that it climbs slower, because the people building AI said they might pace it. When a forecast is wrong in a stock price, shareholders take the loss in an afternoon. When it's wrong in a gas plant, ratepayers carry it for thirty years. That's why Jackson's least glamorous request, forecasts every 90 days, may be his most important one. It's also why the large-load tariff that Duke and the Public Staff owe the Commission by the end of this month matters so much, since minimum bills and long contract terms put the risk of a disappearing customer on the developer instead of the household.
For Triangle teams building on these models, the week carries a practical lesson. The labs themselves just said release schedules might deliberately slow. Design around the models you have today, not the one you assume ships next quarter. Keep your own evaluation set, because the outside evaluators Amodei and Altman promised will be working for the public, not for your product. And keep a second provider wired up, because a pause, a probe, or a presidential post can change your vendor's plans without asking you.
📅 What's Coming
- September 24 — Xi Jinping arrives in Washington to meet Trump. Watch whether AI pacing makes the agenda at all after both governments spent the weekend waving it off.
- End of September — Duke Energy and the Public Staff owe the NC Utilities Commission their large-load tariff filing, the document that decides who carries the risk if data center demand comes in lower than forecast.
- October 1 — Hawley's deadline for OpenAI to answer 16 questions and turn over documents on the Hugging Face breach.
- Watch closely — Whether Anthropic and OpenAI name their outside evaluators and spell out what access they get, and whether Anthropic's public filing, if it lands next month, squares its growth story with its CEO's call to slow down.
That's the week the frontier asked for a brake. See you next Wednesday.
Daniel
BullCity AI · Durham, NC
P.S. If the labs really did slow their release schedules, would your team plan differently? Hit reply and tell me whether a pause would hurt your roadmap or quietly help it. I'm collecting answers for a future issue.
P.P.S. Forward this to the person who thinks AI CEOs only ever hype their products. This week three of them asked to be slowed down, and the government told them no.
