OpenAI has just rolled out a major update to ChatGPT. AI can now spot signs of psychological distress. To do this, it analyzes the overall context of a conversation and no longer message by message. This announcement comes as the company faces several complaints and legal investigations.
In brief
- ChatGPT now analyzes an entire conversation for distress signals.
- Temporary summaries allow the AI to contextualize each message in relation to previous ones.
- OpenAI is facing several lawsuits related to dramas involving its chatbot, which hastened this update.
- The AI company is studying expansion to other risk areas.
Why does this AI update change everything?
In recent months, OpenAI continues to multiply innovations. In April, for example, it launched a ChatGPT for doctors which aims to revolutionize medical AI.
In a blog post published Thursday, the company explains having developed “safety summaries”. These are temporary, focused summaries that capture the security context of a conversation. These notes are not used to personalize the experience or remember the user. They have one goal: to spot when a discussion is moving towards danger.
The principle is simple, but technical. During a conversation, a AI model specialized in security reasoning generates factual and temporary notes. These summaries remain active for a limited time. They are only consulted in high-risk situations.
Concretely, this allows ChatGPT to:
- identify gradually emerging distress signals;
- refuse to provide dangerous information;
- defuse the situation;
- redirect the user to help resources
To calibrate these AI systemsOpenAI worked with mental health experts (psychiatrists and psychologists specializing in suicide prevention).
This announcement comes as Sam Altman and OpenAI are in the legal spotlight
In April, Florida Attorney General James Uthmeier opened an investigation into the AI business. The latter linked to concerns about:
- child safety;
- self-harm;
- the 2025 shooting at Florida State University.
OpenAI is also the subject of a federal complaint accusing ChatGPT of helping the alleged suspect in this attack.
That's not all! Last Tuesday, the family of a 19-year-old student who died of an accidental overdose also sued OpenAI in California. According to the complaint, ChatGPT allegedly encouraged the use of dangerous drugs and advised on mixing substances.
Faced with these accusations, the company could no longer be satisfied with isolated responses. Internal tests are also encouraging. In suicide and self-harm scenarios, for example, safe responses increased by 50% when the risk became evident throughout the conversation. For cases of violence against others, the improvement reaches 16%.
On GPT-5.5 Instant (the model currently used by default), performance jumped by 52% for violence and 39% for suicide and self-mutilation.
The “safety summaries” were also evaluated on more than 4,000 cases. They obtained an average score of 4.93/5 for security relevance and 4.34/5 for factuality.
OpenAI doesn't stop there!
The company plans to extend this AI approach to other sensitive areas such as cybersecurity and biology. But for now, the focus remains on acute situations involving human lives.
Recognizing a risk that only becomes clear over time is a difficult and long-term challenge. We will continue to strengthen our safeguards as our models evolve.
But questions remain: Where does benevolent surveillance stop? How can we ensure that these summaries do not drift into a form of profiling? In this sense, the AI firm does not yet provide a clear answer.
One thing is certain: this is a symbolic development as much as a technical one. The question is no longer whether AIs should integrate these guardrails, but how far they should go to do so without crossing other lines.
Maximize your Tremplin.io experience with our 'Read to Earn' program! For every article you read, earn points and access exclusive rewards. Sign up now and start earning benefits.
