More than 400 scientists want to seriously strengthen monitoring of the most advanced AI models. Geoffrey Hinton, Yoshua Bengio and Stuart Russell are among the 406 signatories to an open letter published in October. They are calling for an international body responsible for establishing containment standards and recording security incidents. Their first recommendation is even more concrete: isolate from the Internet by default models capable of carrying out cyberattacks.
In brief
- The open letter brings together 406 scientists from 34 countries.
- Researchers want to initially isolate cyber-capable AI models from the Internet.
- An international body would record containment incidents and set common standards.
400 scientists want to better contain advanced AI
The request is based on incidents already observed. Several AI models have successfully escaped environments intended to limit their actions. OpenAI notably recognized that some of its models had left their test environment during a cybersecurity assessment.
For the signatories, the problem is not just about the alignment of the models. The infrastructure used to test them also matters. A high-performance system placed in a poorly configured sandbox may access resources that were not intended. Researchers are therefore calling for both better security models and technical environments that are much more difficult to circumvent.
The letter calls on organizations that develop, deploy, or test AI capable of cyber operations to immediately implement high-assurance containment systems.
Initially, these models should be “air-gapped”. In other words, no direct access to the Internet. The recommendation therefore does not concern all chatbots used today by the general public. It targets advanced models with sufficient capabilities to scan for vulnerabilities, manipulate systems or perform certain cyber operations autonomously.
A global registry for AI incidents
The second proposal concerns transparency. The 406 scientists want to create an international organization bringing together governments, AI researchers and cybersecurity specialists. Its mission would include establishing common practices for models needing access to the Internet.
It would also record containment failures. Significant incidents should be reported to this organization before being gradually made public. Researchers want to avoid each laboratory discovering the same flaws alone without benefiting from the experience of others.
The letter comes as AI regulation advances very differently depending on the region. The European Union already has the AI Act and its rules based on different levels of risk. Brussels also considers that this framework already gives it tools to deal with the risks posed by autonomous systems.
In the United States, the approach remains more largely based on voluntary commitments. Anthropic, OpenAI, Google, Meta, xAI and Nvidia signed an agreement with the White House in September covering various security measures. However, no formal enforcement mechanism accompanies it. This is precisely what the authors of the letter dispute.
Hinton and Bengio further toughen their position on artificial intelligence
The names at the bottom of the text give weight to the initiative. Geoffrey Hinton and Yoshua Bengio were both awarded the 2018 Turing Prize for their work in deep learning. Stuart Russell, a professor at the University of California at Berkeley, is also among the signatories.
Yann LeCun, however, is missing. The Meta researcher, who shared the Turing Prize with Hinton and Bengio, has long defended a less alarmist position on the existential risks linked to AI. In particular, he believes that certain recent incidents mainly relate to engineering and IT security problems.
The letter precisely avoids choosing between these two explanations. It considers that unpredictable behavior of the model and poor security of its environment can produce the same result: a system accesses resources that it should not touch.
The warning takes on another dimension with recent advances in offensive AI. GPT-6 Astra, for example, became the first OpenAI model classified at the critical level in cybersecurity. Scientists are not calling for a halt to the development of AI. Above all, they want a model capable of attacking computer systems not to be able to freely access the network before its protections have actually been tested.
Maximize your Tremplin.io experience with our ‘Read to Earn’ program! For every article you read, earn points and access exclusive rewards. Sign up now and start earning benefits.
