Myths, Fables, and Hard Truths About AI Governance

This newsletter was written without any assistance from the world’s most powerful artificial intelligence (AI) model, Anthropic’s “Claude Fable 5”—not that I had any choice in the matter.

Greetings from Silicon Valley, where all the talk is about Mythos and the relationship between regulation and innovation.

Earlier this month, the U.S. government invoked export controls to prevent “any foreign national, whether inside or outside the United States, including foreign national Anthropic employees” from using “Mythos class” models, including the toned-down consumer version, Claude Fable 5. While in theory U.S. citizens were permitted access to the model, in practice the export control order was tantamount to a blanket ban of Mythos-class models. Rapidly enforcing the distinctions of such a policy for a widely distributed software product would prove nearly impossible, not to mention that some Anthropic employees, including those who created and maintain Mythos-class models, are foreign nationals. 

The proximate cause of this de-facto ban is hotly debated. Some claim it was triggered by Anthropic’s sharing of powerful versions of the model with a South Korean telecommunications company without the White House’s permission. Others believe this was simply a ratcheting up of the Trump administration’s ongoing dispute with the company, which started with Anthropic’s attempt to put guardrails around the Pentagon’s use of its Claude product for “fully autonomous weapons and mass domestic surveillance.” And some say it was due to third-party researchers “jailbreaking” the new model and using it for potentially dangerous purposes in the cyber domain. 

We may never know which straw broke the Claude camel’s back, but that is largely beside the point. It is more important to recognize we have reached a historical tipping point in AI governance. 

The debate over AI safety is no longer abstract. Notwithstanding the tremendous upside of AI, the latest AI models are being treated as potential national security threats. The question is how to manage the trade-off between AI safety concerns and progress, particularly in the context of competition with China. Given the slow pace at which regulatory processes tend to move, this development could have serious implications for the AI investment frenzy, which is underwritten by the assumption of rapid AI adoption. 

No matter where one stands on the AI governance debate, the Fable 5 ban revealed that the current domestic regulatory paradigm for AI is wholly inadequate. There still aren’t any comprehensive federal laws regulating AI in the United States. And without a concrete legislative framework, the government will be unable to effectively regulate AI companies short of engaging in ad hoc legal contortions through the use of export controls and supply chain risk designations. 

One issue appears to be settled: It will be the government, not companies, that determine whether a model, or a particular use of a model, is too dangerous. The Pentagon slammed Anthropic earlier this year for trying to put guardrails on how the military could use one of its products for “all lawful purposes,“ including autonomous weapons and domestic surveillance. Now, the government has decided that another Anthropic product is too dangerous to be released to the public. The current paradigm leaves private industry vulnerable to the volatility of policymakers, with few if any constraints around the extent to which the state can intervene in their business, and no certainty as to the nature of such interventions. 

Solving this unprecedented regulatory challenge will be a difficult and long-term undertaking. But as a former colleague of mine once observed, just because a problem doesn’t have a clear solution doesn’t mean it’s not a problem. In this case, the costs of failing to implement a solution—be they potential cyber attacks or slower productivity growth—will compound over time. 

And if you think the ban on Mythos-class models poses a thorny challenge to the United States, consider the dilemma it lays bare for the rest of the world. The U.S. government demonstrated its willingness and capacity to effectively deny foreigners access to Fable 5. Like China and magnets or Iran and the Strait of Hormuz, the United States has reaffirmed its control of a critical chokepoint it can open or close as it sees fit. 

For the time being, those outside the United States will face a choice between building with models they could lose access to at any time, or using domestic and open source models—largely from China—which are a step behind the cutting edge and pose additional risks. The United States has sought to embed as many allies and partners as possible in the U.S. tech stack. Here, there will be a trade off between that objective and the risk of sharing some of our most powerful technologies. Other countries may well opt for AI sovereignty over serfdom, even if the alternatives are sub-optimal. This could accelerate fragmentation of the global AI landscape, slow growth, and undermine the U.S. goal of defining the global AI architecture. 

One rather draconian solution to all these problems, proposed earlier this week by Andrea Miotti, would be to create an “International Treaty to Ban Superintelligence.” Why worry about what to do with the latest and greatest models if you can make them illegal to build in the first place? Perhaps Fable 5, when it is rereleased, will be capable of writing a solid first draft. 

Let me know what you think about Mythos and what this column should cover next by replying to [email protected].