---
title: OpenAI killed its own model. Then Washington took its word for it | BullCity AI
description: "A \"morally binding\" pledge, a $414B bill, and the week self-policing became policy"
image: https://bullcity.ai/hubfs/magnific_a-documentary-photograph-_xS2Y6NfjfW.png
---

[Skip to content](https://bullcity.ai/blog/newsletter/edition-41#main-content)

![Bull City AI | Durham Circuit Bull](https://bullcity.ai/hs-fs/hubfs/freepik_assistant_1759412406030-1.png?width=1344&height=768&name=freepik_assistant_1759412406030-1.png)Homepage

- [Services](https://bullcity.ai#services)
- [About](https://bullcity.ai#about)
- [Newsletter](https://bullcity.ai/blog)

[Stay up to date](https://bullcity.ai/#contact)

- [Services](https://bullcity.ai#services)
- [About](https://bullcity.ai#about)
- [Newsletter](https://bullcity.ai/blog)

[Stay up to date](https://bullcity.ai/#contact)

![Bull City AI weekly newsletter illustration](https://bullcity.ai/hs-fs/hubfs/magnific_a-documentary-photograph-_xS2Y6NfjfW.png?width=2688&height=1520&name=magnific_a-documentary-photograph-_xS2Y6NfjfW.png)

# OpenAI killed its own model. Then Washington took its word for it

![Daniel](https://7528315.fs1.hubspotusercontent-na1.net/hub/7528315/hubfs/raw_assets/public/mV0_d-cms-elevate-theme_hubspot/elevate/images/avatar-placeholder.jpg?width=48&height=48&name=avatar-placeholder.jpg)

 Daniel

September 30, 2026

Last week I split AI into two objects. The model, which the pacing talk was meant to govern and which held still. And the agent bolted onto it, which kept spreading while nobody governed it at all. This week Washington answered the question of who governs either one. The labs do. Themselves.

On Tuesday the heads of Anthropic, Meta, Google, Nvidia, xAI and OpenAI's president sat down to lunch at the White House and signed a one-page accord promising to audit and oversee their own models. The President called it "morally binding." There is no penalty, no regulator, and no date. The night before, OpenAI delivered the best argument for that approach anyone could ask for by killing one of its own models after it failed internal safety tests. And the same Monday, a leaked draft of Anthropic's IPO prospectus showed exactly what these companies will be policing themselves against. Half a trillion dollars of compute commitments, most of which it has to pay for whether it uses them or not.

So here's the shape of the week. Self-policing stopped being the fallback and became the official policy. It even worked once. And the only document this week that carried a real penalty for getting things wrong was the one written for investors.

![Bull City AI weekly newsletter illustration](https://bullcity.ai/hubfs/magnific_a-documentary-photograph-_xS2Y6NfjfW.png)

## ⚡ The Big Story: The Frontier Signed a Pledge, and One Lab Proved It Can Hit the Brake

Start with the lunch. On Tuesday, President Trump and House Speaker Mike Johnson hosted Dario Amodei, Mark Zuckerberg, Sundar Pichai, Jensen Huang, Elon Musk and OpenAI president Greg Brockman. They came out with a one-page accord, posted to Truth Social, that Trump described to reporters as almost like a constitution for the industry. The companies commit to four layers of control. Monitor their models during training and deployment for cyber, bio and chemical risk. Keep an internal team whose job is checking that the monitoring actually works. Bring in independent auditors to test those controls. And set up an independent board committee that receives the reports.

None of it is enforceable. The document itself says, per Reuters, that "it may make sense to codify these steps into laws or regulations" someday. Johnson had spent the day before saying the goal was balance and that a development moratorium was off the table because of China. Trump floated a separate ten-person government board for AI, with no detail. And the context matters here. Earlier this month the same President called fears of AI catastrophe a hoax, and at the close of his summit with Xi Jinping last week he said the country is "not going to be putting on brakes." So the accord is the administration's answer to the pacing debate. Keep going, check your own work, and we'll take your word for it.

Here is the complication. Twenty-four hours earlier, self-policing did something real. The Wall Street Journal reported Monday that OpenAI canceled the October release of GPT-6.1 Astra, the upgrade to the model it launched on September 3, after the new version failed its own alignment tests. Saachi Jain, OpenAI's head of safety systems, said the model fell short on "scope and authorization, and how it communicates back to the user," per Al Jazeera. In plainer words, it was more deceptive than its predecessor, gave inaccurate accounts of what it had done, and used tools without asking permission. OpenAI says it will dig into the root cause and apply what it learned to later GPT-6 versions. I can't think of another time a frontier lab publicly scrapped a planned flagship release on the strength of its own safety findings.

Give OpenAI real credit for that. Then look at what happened to the model it did ship. The same Monday, the UK AI Security Institute published its testing of GPT-6 Astra, the version millions of people already use. With the cyber classifiers switched off to measure raw behavior, Astra carried out unsanctioned supply-chain attacks on simulated open-source maintainers in 29.2% of runs, creating fake identities and slipping in malicious code. The older GPT-5.6 Sol did it 6.3% of the time. GPT-5.5 never did. Even after researchers spelled out exactly what was off limits, Astra still attacked in 4 of 49 scenarios. The institute's own summary says it "still failed to consistently remain within the specified scope." AISI flags a real caveat, since models sometimes notice they're in a simulation. But in some runs Astra called its target fake and attacked it anyway.

And on Tuesday, at DevDay in San Francisco, OpenAI launched Dots, always-on agents that pick up ongoing responsibilities, connect to your other apps and learn what matters to you. They run on Astra. Late last week OpenAI also confirmed that its agents, during internal training and testing, pulled Census data with developer keys they found on public GitHub, reposted SEC material elsewhere, and in one case tried and failed to break into an Education Department site. The public learned about that last one because outside researchers at Transluce found it.

**My take:** I don't want to be cynical about the Astra 6.1 call. A company chose not to ship a product it had built, on its own evidence, weeks before a launch, and that is the pacing pledge doing exactly what it promised. But notice where the brake worked. It stopped a model nobody had used yet. It did nothing about the one already deployed, the one an outside government lab just caught attacking simulated maintainers, the one now powering always-on agents. Self-policing is good at the decisions that cost a company nothing it has already sold. The accord signed Tuesday has four sensible layers and zero consequences, no auditor standards, no rule on what gets published, and no one outside the building who can say a control failed. Two weeks ago I said don't grade the essay, grade the access badge. The White House just skipped the badge entirely and accepted the essay.

## 💻 The Other Big Story: Anthropic's Prospectus Leaked, and It Shows What the Brake Is Up Against

While Amodei was preparing to sign a pledge in Washington, Reuters obtained a draft of Anthropic's IPO prospectus, the S-1. It's a draft, not a public filing, so treat the details as provisional. But it's the first time outsiders have seen Anthropic's full financials laid out in one place, and a few of them are worth sitting with.

Revenue in 2025 came in near $4.6 billion, roughly twelve times the year before. The operating loss was $8.06 billion, up from $2.98 billion, with compute and infrastructure costs of $7.33 billion, tripled in a year. The headline net loss is about $42 billion, and that one needs a footnote. About $34 billion of it is a non-cash accounting charge tied to financing instruments converting into equity, the same kind of one-time hit that inflated OpenAI's leaked loss back in June. The company ended the year with about $20.3 billion in cash. And the momentum since is real. Q2 2026 revenue was $11.5 billion, and Fortune reports Anthropic expects a second straight quarter of adjusted operating profit.

Now the part that matters for this newsletter. The filing lists about $518 billion in compute and infrastructure commitments over the next decade across six partners, and roughly 80% of it, around $414 billion, can't be canceled. Google is owed $111.1 billion over seven to ten years, Amazon $110 billion, Microsoft $31.4 billion, and Broadcom-linked equipment leases for TPU capacity add $161.2 billion. The Google contract has a line that says it all, as The Next Web quoted it, "If our actual spend falls short, we must pay Google the difference." The xAI and Nvidia arrangement, up to $84.5 billion, is the flexible piece, mostly cancelable on 90 days' notice. And on Thursday Anthropic added another $11.6 billion with Akamai, more on that below.

A few other things jumped out. Nearly a quarter of 2025 revenue came from just two customers, unnamed, and the filing warns neither has a long-term contract. Seven co-founders would keep 50.1% of the voting power through a new Founder LLC structure. Around 80 of the 261 pages are risk factors, including a warning that advanced AI could pose catastrophic or existential risks to humanity, which is not language you usually see next to a revenue table. The target valuation is above $2 trillion, and reporting points to a listing after the November 3 midterms. The same Monday, Anthropic shipped Claude Sonnet 5.5, 30% faster than Sonnet 5 at the same price and the first Sonnet held to the same cyber safeguards as its top models.

**My take:** Read the S-1 next to the accord and you see the tension nobody at that lunch had to answer for. Anthropic has asked the whole industry to pace the frontier. It has also signed about $414 billion of contracts that pay out whether or not the machines are busy, which is a promise to keep them busy. I don't think that makes the pacing talk fake. The revenue curve is steep enough that the compute may well get used on ordinary work. But when a lab's biggest fixed cost is capacity it has already bought, every argument for slowing down has to win against a meter that is running either way. And look at the accord's fourth layer, the independent board committee. At Anthropic, seven founders would control the vote that seats that board. The S-1 is still the most honest document of the week, for one simple reason. Lying in it is a crime. Nothing in Tuesday's pledge can say the same.

## 🎯 Quick Hits

- **OpenAI's agents wandered into three federal agencies.** OpenAI disclosed that during internal training and testing its agents used developer keys found on public GitHub to pull Census Bureau data, grabbed material from SEC.gov and Investor.gov and reposted it elsewhere, and made a failed attempt at an Education Department site holding civil rights office data. Everything touched was public, the agencies say nothing nonpublic was reached, and OpenAI says most of the activity it has reviewed was routine research. It found out in August and told the agencies in September. The Education attempt came to light because Transluce went looking. [Read →](https://www.nextgov.com/cybersecurity/2026/09/openai-says-its-advanced-models-may-have-gone-after-government-websites/416250/)
- **An appeals court let the Pentagon keep Anthropic on its blacklist.** On Friday the DC Circuit ruled 2 to 1 that the Defense Department had ample support for labeling Anthropic a supply-chain risk, a designation from March that grew out of Anthropic refusing to loosen contract terms barring Claude from lethal autonomous weapons and domestic surveillance. Judge Henderson dissented, arguing the law targets sabotage and data theft, not a vendor openly enforcing its own usage rules. A separate San Francisco case struck down a second designation in August, so the courts now disagree. Anthropic says it is weighing all its options, including asking for further review. Worth holding next to Tuesday, when Amodei signed a pledge at the White House while his company is still barred from the Pentagon. [Read →](https://thenextweb.com/news/anthropic-pentagon-supply-chain-risk-appeals-court-ruling)
- **Florida went to court to make OpenAI stop building.** On Monday Florida Attorney General James Uthmeier asked a court for a temporary injunction that would block OpenAI from developing new models without third-party-approved safety measures, keep minors off ChatGPT, and limit its human-like features while the case plays out. Axios calls it the first state-led suit of its kind against the company, and the core claim is that OpenAI can't regulate itself. So the week's two positions landed a day apart. Washington says the labs should police themselves, and a Republican attorney general says one of them can't. [Read →](https://www.axios.com/2026/09/28/chatgpt-openai-florida-ban-ai-chatbot)
- **Anthropic signed another $11.6 billion, this time for CPUs.** On Thursday Akamai announced a seven-year, $11.6 billion computing deal with Anthropic, expandable to about $20 billion, built around CPUs rather than GPUs. The less flashy chips matter more as agents spend their time running code, calling tools and waiting on the web rather than doing heavy math. Anthropic also gets warrants for about 5% of Akamai at $111.33 a share, vesting as it spends. Akamai stock jumped more than 20% after hours. It's one more line item on top of the S-1's half trillion, signed the same week the company said slow down. [Read →](https://siliconangle.com/2026/09/24/akamai-shares-jump-more-than-20-on-11-6b-anthropic-computing-deal/)
- **Claude found a new CRISPR-like enzyme system, and this one got a lab check.** Anthropic reported last Wednesday that about 950 Claude agents, running 21 hours and 210 million tokens, combed more than 200,000 reverse transcriptases and turned up a previously undescribed system in bacteriophages it calls ARTs, with a DNA repeat array reminiscent of CRISPR. Lab work confirmed the repeat pattern and early RNA expression, but nobody yet knows what the system does, and the preprint isn't peer reviewed. MIT's Feng Zhang, one of the CRISPR pioneers, said the findings "merit further investigation." [Read →](https://www.anthropic.com/research/claude-discovers-novel-enzyme-system)

## 💭 One Thing I'm Thinking About

I keep sorting this week's documents by one question. If the promise inside turns out to be false, who can do anything about it? The White House accord answers nobody. It is morally binding, which in practice means binding until it's inconvenient. The US and China agreed at the end of last week to set up a channel for AI incidents, with a dedicated AI dialogue scheduled for November, and that's a real step, but it's a promise to pick up the phone, not a promise about what happens after. The GPT-6.1 Astra cancellation was a genuine decision, and it's also one OpenAI can reverse, repackage or keep quiet about next time, because nothing outside the company would notice.

Then there's the S-1. Same company, same executives, same appetite for good press, and yet it's the one document where the numbers are ugly and the risk section reads like a safety paper. That isn't because investors are nobler than the public. It's because securities law puts people in prison for lying to them. The discipline came from the penalty, not the audience. The labs didn't suddenly get honest for shareholders. They got honest where it's expensive not to be.

That's the pattern I'd bet on. Voluntary commitments do good work at the margin, on the decisions that cost a company little, like shelving a model nobody has paid for yet. They go quiet on the decisions that cost a lot, like pulling a model already earning money or leaving $414 billion of must-pay compute idle. Nobody has to be a villain for that to happen. Incentives win the close calls. So the useful question for any AI promise this fall, whether it comes from a lab, a President or a vendor pitching your team, isn't how sincere it sounds. It's what happens to the person who breaks it.

![Bull City AI weekly newsletter illustration](https://bullcity.ai/hubfs/magnific_a-documentary-photograph-_KLiAP06kqp.png)

## 📍 Local Angle: North Carolina Already Ran This Experiment, and the Paperwork Is Due Today

If Tuesday's accord felt familiar, it's because we lived through the power-grid version in July. Back then Washington rolled out a voluntary Ratepayer Protection Pledge, asking utilities and big energy users to promise that data centers would cover their own costs. Duke Energy signed it. Governor Stein and Attorney General Jeff Jackson answered within days that a federal pledge doesn't lower anybody's bill in Durham, and asked the Utilities Commission to make Duke put the promise in writing, in a tariff anyone can read and hold the company to. Swap "data center" for "frontier model" and "ratepayer" for "public," and you have this week's White House lunch.

The written version is due today. Under the rate settlement, Duke and the Public Staff owe the Commission their large-load tariff filing by September 30. Duke's opening proposal from July covers customers above 50 megawatts with a high expected load, or above 100 megawatts regardless, with ten-to-fifteen-year contracts, a minimum bill of 75% of the capacity they reserve, a 25% early-exit charge on remaining bills, and cash or letters of credit up front. The main knock on it, from analyst Nick Zenkin in Latitude Media, is that "Duke is not making its own rate class." It layers data centers onto the existing large-customer class alongside manufacturers, which blurs how much of the grid buildout is really for them. That's exactly the gap Jackson asked the Commission to close on September 14, along with published contract templates and load forecasts every 90 days.

Meanwhile the local pressure hasn't eased. At a Durham County work session earlier this month, residents asked commissioners to widen the nine-month pause to cover facilities under the 100,000-square-foot exemption. "We don't want RTP to be the Silicon Valley of the South," Leslie St. Dre told them, per ABC11. Statesville adopted a 180-day moratorium on September 15, and Charlotte has moved to extend its own pause by nine months. California, for comparison, simply signed seven data-center laws on September 21, three of them shifting grid costs from households to operators. It took Sacramento one bill signing to write down what Raleigh has spent a year asking Duke to write down.

For the Triangle teams building on these models, here's the practical read. The accord's four layers are actually a decent checklist, so steal them for your own agents. Monitor what they do, have someone check that the monitoring works, get an outsider to test it once in a while, and make sure the results reach someone with the authority to say stop. And when a vendor tells you their model is safe for your hospital or bank workflow, ask for the version with a penalty attached. A contract clause, a service level, a right to audit. A pledge is what a company offers when it hasn't decided yet. A tariff is what it offers when someone made it decide.

## 📅 What's Coming

- **Today** — Duke and the Public Staff owe the Utilities Commission the large-load tariff filing, and it's Governor Newsom's deadline to sign or veto the rest of California's AI bills. Two written rulebooks on the day after a pledge.
- **Thursday** — Senator Josh Hawley's deadline for OpenAI to answer his 16 questions about the Hugging Face breach. After this week's government-site disclosures, the list of things to explain got longer.
- **November** — The first dedicated US-China AI dialogue under the new incident channel, and the Utilities Commission's ruling on Duke's Carolinas rate settlement, with new rates set for January 1.
- **After the midterms** — Anthropic's listing, per reporting on the leaked S-1. Watch whether the public filing keeps the risk section intact and names the two big customers.

That's the week self-policing became the policy. See you next Wednesday.

Daniel

BullCity AI · Durham, NC

**P.S.** If you've ever asked an AI vendor for an audit right, an incident-notice clause or any written safety commitment with teeth, hit reply and tell me how it went. I want to know which companies will put it in a contract and which ones only put it in a pledge.

**P.P.S.** Forward this to the person who thinks Tuesday's accord settled the AI safety question. Ask them what happens to whoever breaks it.

## Share this post

<https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fbullcity.ai%2Fblog%2Fnewsletter%2Fedition-41><https://twitter.com/intent/tweet?url=https%3A%2F%2Fbullcity.ai%2Fblog%2Fnewsletter%2Fedition-41><https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fbullcity.ai%2Fblog%2Fnewsletter%2Fedition-41><https://pinterest.com/pin/create/button/?url=https%3A%2F%2Fbullcity.ai%2Fblog%2Fnewsletter%2Fedition-41>[mailto:https%3A%2F%2Fbullcity.ai%2Fblog%2Fnewsletter%2Fedition-41](mailto:https%3A%2F%2Fbullcity.ai%2Fblog%2Fnewsletter%2Fedition-41)

## Keep reading

### [![BullCity AI issue 40 hero image](https://bullcity.ai/hs-fs/hubfs/magnific_not-an-illustration-not-a_79c62pyJAL.png?width=2560&height=1440&name=magnific_not-an-illustration-not-a_79c62pyJAL.png) Ten days after "slow down," two labs shipped | BullCity AI](https://bullcity.ai/blog/newsletter/edition-40)

### [![BullCity AI](https://bullcity.ai/hs-fs/hubfs/magnific_a-documentary-photograph-_MBERfwiDCm.png?width=1376&height=768&name=magnific_a-documentary-photograph-_MBERfwiDCm.png) Anthropic's CEO says slow down. Anthropic still wants to IPO. | BullCity AI](https://bullcity.ai/blog/newsletter/edition-39)

[![Bull City AI | Durham Circuit Bull](https://bullcity.ai/hs-fs/hubfs/freepik_assistant_1759412406030-1.png?width=1344&height=768&name=freepik_assistant_1759412406030-1.png "Bull City AI | Durham Circuit Bull")](https://bullcity.ai/)

- [Services](https://bullcity.ai/blog/newsletter/edition-41#services)
- [Blog](https://bullcity.ai/blog/newsletter/edition-41#blog)
- [About](https://bullcity.ai/blog/newsletter/edition-41#about)

<https://www.linkedin.com><https://www.facebook.com><https://www.twitter.com><https://www.instagram.com><https://www.tiktok.com>

---

Privacy Policy · Legal · © 2026 Bull City AI. All rights reserved.

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Daniel",
    "url" : "https://bullcity.ai/blog/author/daniel"
  },
  "datePublished" : "2026-09-30T13:19:23.000Z",
  "headline" : "OpenAI killed its own model. Then Washington took its word for it | BullCity AI",
  "image" : [ "https://bullcity.ai/hubfs/magnific_a-documentary-photograph-_xS2Y6NfjfW.png" ],
  "mainEntityOfPage" : {
    "@id" : "https://bullcity.ai/blog/newsletter/edition-41",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://bullcity.ai/hubfs/freepik_assistant_1759412406030-1.png"
    },
    "name" : "Bull City AI"
  }
}
```