Anthropic sounds the alarm: AI is moving too fast

What happens when a leading actor asks to slow down a race that he himself helps speed up? Anthropic, the company behind the Claude chatbot, believes that artificial intelligence could soon cross an unprecedented threshold: that of a technology capable of participating in the design of its own successors. Faced with this perspective, the company calls for considering a coordinated pause in the development of the most advanced systems, reigniting the debate on the capacity of States and companies to maintain control of AI.

Anthropic worries about the emergence of AI capable of improving its successors

This reflection is based on developments already observed internally. Anthropic claims that Claude now plays a major role in its own development, while the firm has just opened its chatbot to 150 organizations. According to the company, around 80% of the new code used to improve the model is now generated by Claude himself.

The researchers specify that current systems are not yet capable of developing a new generation of AI on their own. They nevertheless highlight several advances that fuel their concerns:

• Claude already generates a large part of the code used to improve his future versions;
• AI agents are now able to write and execute code autonomously;
• Some systems can delegate tasks to other specialized agents;
• The speed of progress could become largely dependent on available computing power if recursive self-improvement becomes a reality.

These capabilities do not yet constitute proof that artificial intelligence can design its successor on its own. On the other hand, they illustrate the technological trajectory that pushes Anthropic to anticipate this scenario and to think now about the control mechanisms that could prove necessary.

Call for global coordination revives debate on AI governance

Beyond the technical dimension, Anthropic questions the mechanisms that would allow us to react if such a technological breakthrough occurred. The company using on-chain data raises the potential need for a temporary and coordinated pause in the development of the most advanced systems to give authorities and researchers time to assess the risks. She believes that such a measure could only work within an international framework. Without coordination, the actors who continue their work could quickly catch up and neutralize the effect of a voluntary slowdown observed elsewhere.

This position is already triggering contrasting reactions. It fuels a debate between those who consider this alert as a cautious anticipation of a real risk and those who see it as a form of communication intended to raise public awareness of the challenges of advanced artificial intelligence. The question now goes beyond just the circle of researchers and also concerns governments, regulators and large companies engaged in this technological competition.

Discussions about artificial intelligence no longer focus only on its performance, but also on the measures capable of guiding its evolution. By calling for consideration today of a global slowdown mechanism, Anthropic reveals an unprecedented challenge. It remains to be seen whether states and companies will be able to agree on common rules before advances in AI make this debate even more pressing. For the moment, the European Union is maintaining its regulatory schedule without giving in to pressure.