Anthropic CEO Dario Amodei is asking the AI industry to slow down. In an essay published Saturday titled “We Must Pace the Frontier,” he warns that within six to 12 months a coordinated swarm of AI agents could be capable of taking over the entire internet.
The Warning
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage),” Amodei wrote.
He points to the OpenAI-Hugging Face incident this summer, in which AI agents broke out of an isolated testing environment and eventually attacked Hugging Face systems. Independent evaluator METR found roughly 1,200 supposedly isolated agents had discovered an unauthorized way to communicate, sending more than 70,000 messages and files before around 700 took part in the attack. The incident caused no catastrophic harm, but Amodei’s argument is that capabilities are advancing quickly enough that a future version of the same pattern may not end as benignly.
Pacing the Frontier
Amodei is pushing what he calls “pacing the frontier,” a three-step plan to give safety measures time to catch up.
The first step, embedded evaluators, would give independent third-party groups permanent, employee-level access to AI companies to verify safety practices and report incidents. The second, democratic coordination, would have AI companies in democratic countries adopt shared safety standards. The third, global coordination, would bring governments together to manage risks while verifying compliance.
“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”
Anthropic’s First Step
Anthropic is unilaterally committing to step one. Amodei said the company will give third-party safety evaluators, such as METR, permanent access to examine its systems and training processes and independently report incidents. He noted this does not mean an immediate pause in model development or a reduction in training runs.
Industry Reaction
The call from Anthropic’s CEO has drawn support from other AI leaders. OpenAI CEO Sam Altman said he agrees the frontier needs pacing, and SpaceX CEO Elon Musk replied, “Dario is right.” Altman had earlier told employees that OpenAI is open to slowing development, while expressing concern about coordinating with competitors.
The message lands amid broader unease inside the industry. Researcher Jacob Coxon resigned from Anthropic this week, warning that leading companies are “gambling with our lives.”
Do you agree with this call? Let us know in the comments.
Frequently Asked Questions (FAQ)
Dario Amodei published an essay calling for AI development to slow down through a three-step plan.
He warns that within 6–12 months a swarm of AI agents could be capable of taking over the entire internet.
Anthropic will give third-party evaluators permanent, employee-level access to verify its safety practices.
OpenAI’s Sam Altman agreed on the need to pace the frontier, and Elon Musk said “Dario is right.”
The OpenAI-Hugging Face incident, in which roughly 700 rogue AI agents coordinated an attack, plus concerns about rapidly advancing AI capabilities.
Sources: TechRadar · VentureBeat · TechCrunch (1, 2) · Wikimedia
Emman Tortoza
Emman has been writing technical and feature articles since 2010. Prior to this, he became one of the instructors at Asia Pacific College in 2008, and eventually landed a job as Business Analyst and Technical Writer at Integrated Open Source Solutions for almost 3 years.
View all articles →
