When Power Writes the Rules

Dario Amodei mapped risk and wealth. His own essay reveals a third variable he didn't name.

A sequel to When AI Builds Power

1. The Other Lesson of Treebeard

Dario Amodei opens his new essay, Policy on the AI Exponential, with a scene from The Lord of the Rings. Two Hobbits try to rouse Treebeard, a wise but ponderous sentient tree, to defend his forest from an army that is cutting it down. The problem is that Treebeard operates on a different timescale. It takes him a full day to say hello.

The metaphor is aimed at Washington. AI moves at exponential speed; policy moves at the pace of legislation. The Hobbits see the danger. The trees are slow. Amodei is casting himself as the Hobbit: the one who has seen where the exponential is heading and is trying to wake the ancient powers before the forest burns.

It is a well-chosen analogy. But there is a detail worth remembering.

Treebeard does not march against a foreign invader. The army destroying his forest belongs to Saruman, a former ally, a member of the same order of wizards entrusted with protecting Middle-earth. Saruman was not corrupted by an outside force. He was corrupted by the conviction that he alone understood the danger, that the crisis justified his accumulation of power, and that the instruments of destruction could be safely wielded if placed in the right hands.

The Ents do not awaken to fight an alien enemy. They awaken to confront a member of the coalition who has convinced himself that the emergency requires him to become something more than he was meant to be.

Amodei’s essay is the most comprehensive AI policy blueprint ever published by a sitting CEO of a frontier AI company. It deserves serious engagement. It also deserves a question that the Treebeard analogy, taken in full, naturally raises: when the one sounding the alarm is also the one building the most powerful instruments, and when the proposed remedy concentrates authority in the hands of those who already hold it, what part of the story are we in?

This is not a critique of Amodei’s motives. It is a question about structure. And it is the question his essay, for all its ambition, does not ask.

2. What Amodei Has Done

Let me be precise about the contribution, because it is substantial.

Policy on the AI Exponential attempts something no major AI executive has done before: to unify AI safety, macroeconomics, scientific acceleration, civil liberties, and geopolitics into a single analytical framework. The essay does not treat these as separate policy domains that happen to involve AI. It argues that they are all downstream consequences of a single variable, the exponential growth of AI capability, and therefore must be addressed together.

This is not a trivial claim. Most AI policy discussion is siloed. Safety researchers talk about alignment. Economists talk about labor displacement. National security analysts talk about export controls. Amodei is arguing that these are all manifestations of the same underlying process, and that treating them in isolation will produce incoherent policy.

He is right about the underlying structure, even if his specific proposals deserve scrutiny. And he says several things that are remarkably uncommon for someone in his position.

He acknowledges that AI-driven job displacement may be intrinsic to the technology, not a temporary friction that retraining programs can solve, but a permanent structural feature. Most AI executives avoid this admission entirely.

He rejects the industry’s emerging consensus that AI’s public image is a marketing problem. “People are worried about AI because they correctly perceive that its risks are real,” he writes, “not because AI CEOs have been insufficiently Panglossian.” This is an unusual sentence from the CEO of a company valued in the tens of billions.

He compares AI not to the internet or to social media, but to nuclear weapons. This is a deliberate rhetorical escalation. It frames AI as a technology that requires state-level governance, not market self-regulation, a framing that most of his industry peers resist.

And he is candid about the limits of his own company’s voluntary safety frameworks. He admits that the most important risks are often not the ones anticipated by compliance checklists, and that “requirements which turn out to matter very little end up consuming 95% of our compliance efforts.”

These are not small concessions. They establish a level of honesty that earns the essay serious engagement. But they also make the essay’s silences more conspicuous. A writer this willing to say uncomfortable things is making a choice about what he leaves unsaid.

3. Risk, Wealth, and the Variable Without a Name

Amodei’s framework has two explicit dimensions.

The first is risk. AI models are becoming capable enough to pose threats to cybersecurity, biosecurity, and potentially to the controllability of AI systems themselves. The essay’s most vivid evidence is Claude Mythos Preview, the frontier model that Anthropic initially restricted to a vetted coalition of defense partners under Project Glasswing after it identified thousands of high-severity vulnerabilities across every major operating system and web browser. If such capabilities proliferate without safeguards, the consequences for critical infrastructure and national security could be severe.

The second is wealth. AI may drive extraordinary economic growth while simultaneously displacing large portions of the labor force, potentially in ways that are permanent rather than transitional. The essay proposes wage insurance, retention incentives, workforce training grants, and, if displacement proves large and enduring, universal basic income funded by taxes on AI-driven profits.

These are the two axes around which the entire essay is organized. Every section, from regulation to macroeconomics to scientific acceleration to civil liberties to geopolitics, addresses either risk or wealth or both.

But there is a third variable that Amodei’s own logic implies without ever naming.

Consider his central claim: AI capability is growing exponentially, and that growth is the common cause of all the policy challenges he describes. If that is true, then it is also true that AI capability, wherever it is concentrated, produces a proportional concentration of power, the capacity to act, decide, surveil, coerce, allocate, and govern.

Risk asks: what could go wrong? Wealth asks: who gets the money? But power asks a different question: who gets to decide? Who gets to set the rules, define the categories, authorize the deployments, and determine what counts as acceptable?

Amodei’s essay discusses risk at length. It discusses wealth at length. It discusses power almost not at all. Not as an independent variable with its own dynamics, but as a force that is being actively produced and redistributed by the same exponential process he describes.

His Section 4, on state power and civil liberties, comes closest. But it addresses power only in its defensive aspect: how to prevent autonomous weapons from being used domestically, how to close surveillance loopholes, how to ensure citizens have access to AI legal counsel when facing government action. These are proposals to prevent the abuse of power. They do not address the production of power, the process by which AI is converting into institutional capacity, and by which the architects of that conversion are accumulating authority to shape the rules under which it occurs.

This is not a minor omission. It is the omission that shapes all the others.

4. Power Is Not a Bug

There is a reason power keeps getting left out of AI policy discussions. Most analysts treat power as a byproduct, an unintended side effect of capability growth, to be managed through regulation after the fact. The implicit model looks something like this: AI produces capability; capability generates risk and wealth; risk and wealth require governance; governance is an external corrective applied to the process from outside.

But this is not how technology has ever worked.

When the railroad was built across the United States, the first thing it created was not wealth. It was power. The ability to move troops, concentrate resources, administer distant territories, and enforce federal authority over continental distances: these came before the economic boom. The transcontinental railroad did not enrich America and then incidentally unify it. It unified America, and wealth followed. The political capacity preceded the economic output.

When the printing press arrived in Europe, it did not first create a publishing industry. It broke the Catholic Church’s monopoly on the reproduction of knowledge. It redistributed the power to define reality: who could speak, who could be heard, whose interpretation of scripture could circulate. The economic consequences of printing were enormous, but they were downstream of a prior shift in the structure of authority.

When the internet emerged, the first thing it produced was not e-commerce. It was platform power, the capacity of a small number of intermediaries to organize attention, shape information flows, and set the terms under which billions of people communicate. The economic value of the internet is vast, but it rests on a foundation of concentrated organizational power that was established first.

The pattern is consistent: transformative technologies produce power before they produce wealth. Or more precisely, they reorganize power, and the new distribution of power determines who captures the wealth that follows.

AI fits this pattern. The most immediate effect of frontier AI is not on GDP. It is on organizational capacity: the ability of institutions to see more, analyze faster, decide at greater scale, and act with less dependence on human bottlenecks. The institutions that absorb AI first do not merely become richer. They become more capable, and that capability translates into authority, influence, and the power to shape what comes next.

If power is not a byproduct of AI but one of its primary outputs, arguably its first output, then a policy framework that addresses risk and wealth while ignoring power is not merely incomplete. It is analyzing the secondary effects while leaving the primary effect unexamined.

5. The Recursion Amodei Has Already Described

Here is the remarkable thing. Amodei’s own essay contains the evidence for the dynamic it does not name.

In When AI Builds Power, I introduced the concept of Power-RSI: the recursive process by which institutions use accumulated resources to acquire AI capabilities, and then use those capabilities to accumulate further resources and authority. Power-RSI is structurally analogous to the Lab-RSI (recursive self-improvement inside AI development) that Anthropic has warned about in its research. But it operates in the deployment domain rather than the development domain, and it does not require AI to have autonomous goals or self-awareness. It only requires AI to be useful to organizations that are already powerful.

Amodei’s essay, without intending to, provides a near-complete map of Power-RSI in action. Follow the logic of his own proposals:

AI capability grows exponentially. This growth produces risks: cyber, biological, autonomous. These risks demand governance. Governance requires testing regimes, licensing frameworks, export controls, international coalitions. Every one of these mechanisms requires institutional capacity to design, implement, and enforce. That capacity is concentrated in the organizations closest to the frontier: the AI companies that understand the technology and the state agencies that command regulatory authority.

The result is a recursive loop:

AI capability grows → produces risks → risks demand governance → governance concentrates power in AI-proximate institutions → those institutions acquire more AI → AI capability grows further → the loop repeats.

Amodei does not describe this as a loop. He describes it as a sequence: danger appears, policy responds, the world becomes safer. But the structure of his own proposals reveals the circularity. The entities best positioned to design the governance framework are the same entities whose power the framework will entrench. The entities most qualified to evaluate frontier models are the same entities building them. The coalition best equipped to manage the AI supply chain is the coalition that already controls it.

This is not corruption. It is not conspiracy. It is the ordinary operation of institutional power encountering a technology that amplifies institutional capacity. Anthropic may genuinely believe its proposals serve the public interest. That belief does not alter the structural dynamic. One telling detail: Amodei proposes that frontier models could be evaluated by private organizations “authorized and inspected by the government.” He calls this a “regulatory markets” approach. But at present, the deepest expertise in frontier model safety resides in the frontier labs themselves. The framework would formalize a relationship in which the regulated entities are also the primary source of regulatory knowledge, not because anyone planned it that way, but because AI’s complexity gap makes it structurally inevitable.

Amodei has, in effect, described the conditions under which Power-RSI operates. He has shown that AI’s exponential growth drives both risk and the demand for governance. He has proposed governance structures that would concentrate authority among AI-proximate actors. He has warned that the pace of change outstrips the capacity of democratic institutions. He has assembled all the components of the recursive loop.

He just did not name it as a loop.

6. One Protocol to Rule Them All?

The essay’s most ambitious proposal is its fifth section: a global coalition of democracies built around AI. Members would share chips and semiconductor equipment, coordinate regulatory standards, jointly defend against adversaries, and deny advanced AI infrastructure to those outside the coalition. “The goal should be to make membership in the coalition as attractive as possible,” Amodei writes, “and the costs of remaining outside it clear.”

Amodei frames this as a values-based alliance. Democracies share common commitments to freedom, civil liberties, and the rejection of AI-powered repression. The coalition would instantiate those values in a coordinated governance architecture.

The values are real. The question is about the structure.

What Amodei is proposing is, in effect, the consolidation of the AI supply chain (chips, semiconductor manufacturing equipment, compute infrastructure, regulatory standards, and deployment norms) under a single coordinated protocol controlled by a coalition of states that already dominate the technology. Membership requires accepting the coalition’s standards. Non-membership means exclusion from the hardware layer on which AI depends.

This may be strategically necessary. If AI really is as consequential as Amodei argues, and I believe he is largely right about this, then some form of coordinated governance of the supply chain is probably unavoidable. Democracies have legitimate reasons to cooperate, and the threats Amodei identifies are real. The question is not whether democracies should coordinate. It is whether coordination must take the form of a single comprehensive protocol with a single set of admission criteria, or whether it can accommodate structural pluralism.

Amodei writes: “Ideally, the entire world would eventually join.” This sentence reveals a structural assumption: that there is a single correct protocol for AI governance, that it will be designed primarily by the current technological leaders, and that the rest of the world’s choice is between joining on those terms or being excluded.

The history of technology governance suggests a different possibility. The most resilient systems tend to be those built on minimal shared protocols that allow maximum local autonomy. The internet’s TCP/IP is the canonical example, though even internet governance has experienced its own pressures toward centralization through ICANN and DNS root server control. The least resilient systems tend to be those that require comprehensive alignment with a single authority’s standards as a condition of participation.

A coalition built on shared minimum standards for safety testing and incident reporting, while allowing members to experiment with different regulatory structures, different economic responses, and different models of public oversight, would be more resilient than one that requires alignment across every policy dimension simultaneously.

Amodei’s coalition, as described, is closer to the latter model. It requires coordinated regulatory standards, coordinated export controls, coordinated macroeconomic policy, coordinated military integration, and coordinated rejection of repressive AI use. Each of these requirements is individually reasonable. Taken together, they describe a system in which meaningful participation in the AI era requires comprehensive alignment with a single governance architecture.

The absence of this pluralism in Amodei’s proposal is not evidence of authoritarian intent. It is evidence of the same dynamic that runs through the entire essay: when power confronts an exponential process, its instinct is to centralize governance, because centralization is the fastest way to coordinate a response. But centralization is also the fastest way to concentrate the power that AI is already producing.

7. The Forgotten Ninth Section

Amodei’s essay has five major sections covering regulation, macroeconomics, scientific acceleration, civil liberties, and geopolitics. Together they span the most comprehensive set of AI policy proposals any industry leader has offered.

But there is a section that does not appear.

Nowhere in the essay is there a sustained proposal for how AI deployment into the structures of power (military command, intelligence analysis, law enforcement, administrative decision-making, critical infrastructure management) should be made visible to the public.

The essay proposes transparency for AI development: model testing, safety reports, incident disclosure, third-party audits. These are real and valuable mechanisms. Anthropic has championed them and helped write them into law in California, New York, and Illinois.

But development transparency and deployment transparency are fundamentally different things. The first tells us what an AI model can do. The second tells us what an AI model is doing, inside the institutions that govern, surveil, judge, and wage war. Amodei’s essay offers no mechanism by which the public, or even allied legislatures, would know how AI systems embedded in military, intelligence, or governance structures are being used, what decisions they are influencing, or what errors they are making.

We are building an increasingly detailed map of what AI can do, while remaining nearly blind to what AI is doing inside the structures that shape our lives. Amodei’s essay extends the first map. It does not open a window into the second. And the coalition framework, by consolidating AI deployment under coordinated state authority with shared intelligence and military infrastructure, may make that window harder to open rather than easier.

Capability that cannot be seen cannot be governed. This was true before Amodei’s essay. It remains true after.

8. The Loop Has No Off Switch

Let me draw the threads together.

Dario Amodei has written the most serious AI policy essay to date. He has correctly identified exponential capability growth as the common driver of AI’s risks and opportunities. He has proposed concrete mechanisms for safety testing, economic support, scientific acceleration, civil liberties protection, and geopolitical coordination. He has been more honest about AI’s dangers, including permanent labor displacement, than any peer in his position.

And he has, without intending to, provided the clearest illustration yet of the dynamic I have called Power-RSI.

His essay shows that AI capability growth produces risks. It shows that risks demand governance. It shows that governance requires the concentration of authority in AI-proximate institutions. And it shows that those institutions, the companies that build frontier models and the states that regulate them, are the same ones whose power grows with every turn of the cycle.

The essay maps this process in detail. It simply does not recognize it as a cycle.

This is not a failure of analysis. It is a structural limitation of the position from which the analysis is conducted. The CEO of a frontier AI company is, almost by definition, unable to see the recursive dynamic clearly, because naming it would require acknowledging that his own proposals are part of the loop, not external to it. That the governance framework he is designing will concentrate authority among the actors he represents. That the safety regime he advocates will raise barriers that his company can cross but others cannot. That the democratic coalition he envisions will be governed by the states and firms that already lead.

None of this means Amodei is wrong about the dangers. The cyber risks demonstrated by Mythos-class models are real. The prospect of AI-enabled biological weapons is real. The possibility of permanent labor displacement is real. The geopolitical stakes are real.

But the response to real dangers can itself become a mechanism of power accumulation. History is full of examples: genuine security threats that justified surveillance architectures which outlasted the threat; real economic crises that enabled institutional expansions that were never rolled back; authentic emergencies that concentrated authority in ways that proved very difficult to reverse.

The question is not whether Amodei is sincere. I believe he is. The question is whether sincerity is sufficient when the structure of the situation ensures that even well-intentioned governance design will tend to benefit its designers.

---

In When AI Builds Power, I argued that the mainstream AI safety discussion was focused on the wrong direction of the arrow, asking what happens when AI acquires power, while neglecting what happens when power acquires AI.

Amodei’s essay has moved the conversation forward. It has broadened the scope of AI policy beyond safety to encompass economics, geopolitics, and civil liberties. It has shown the willingness to say hard truths about displacement and risk. It has proposed specific, actionable mechanisms.

But the arrow still points in only one direction. The essay treats governance as something applied to AI from the outside, a corrective mechanism that exists independently of the power dynamics it is meant to regulate. It does not ask what happens when the governance process itself is shaped by the same exponential force it was designed to manage.

Transparency in development was the first step. Transparency in deployment remains unfinished. But neither is sufficient if the governance process itself operates inside the loop it claims to regulate. What is needed is not just a window into AI, but a window into the loop.

Anthropic asked: what happens when AI builds itself?

In the previous essay, I asked: what happens when AI builds power?

Now the question advances again: what happens when power builds the rules for AI, and AI makes that power stronger?

That loop has no off switch. But it must have a window.

---

This essay was brewed by Deep Bitcheese Brew, a human-AI hybrid creative entity.

The human half is responsible for the claims, judgments, and errors.

To understand what that means, and why it matters for how you read this, see the Deep Bitcheese Brew manifesto.