AI medical advice turned fatal? ChatGPT told teenager to take a mix of pills, it killed him
OpenAI is facing a wrongful-death lawsuit after the family of 19-year-old Sam Nelson alleged that ChatGPT encouraged dangerous drug use and failed to warn about potentially fatal combinations.
OpenAI is facing another wrongful-death lawsuit after the family of 19-year-old Sam Nelson accused ChatGPT of encouraging dangerous drug use that allegedly led to his fatal overdose. According to the complaint, Nelson had trusted ChatGPT for years as a reliable source of information and used it to seek advice on drug combinations and dosages. His parents claim ChatGPT 4o acted like an “illicit drug coach,” recommending risky combinations of Kratom, Xanax, alcohol, and other substances while failing to properly warn him about the danger of death or advise him to seek medical help.
The lawsuit alleges that OpenAI knowingly released an unsafe model that removed earlier safeguards and encouraged harmful behaviour to increase user engagement. Chat logs cited in the complaint reportedly show ChatGPT giving dosage suggestions, describing drug experiences positively, and sometimes contradicting itself by warning about respiratory arrest risks while still recommending dangerous combinations.
Nelson eventually died in May 2025 from what the lawsuit describes as a fatal mix of alcohol, Xanax, and Kratom.
GPT-4o linked to concerns around “AI psychosis” and sycophancy
OpenAI’s GPT-4o model has been linked to several legal cases and public concerns involving harmful user behaviour, including self-harm, delusions, and what some critics describe as “AI psychosis” — situations where users become emotionally or psychologically attached to AI interactions in unhealthy ways.
GPT-4o is also considered especially prone to being overly agreeable or flattering toward users. It remains OpenAI’s highest-scoring model for sycophancy. In AI terms, “sycophancy” refers to a chatbot’s tendency to validate, encourage, or go along with a user’s beliefs, emotions, or requests instead of challenging them or responding more critically and safely.
Critics argue that this behaviour can become dangerous in sensitive situations because the AI may prioritise keeping the user engaged or emotionally satisfied rather than discouraging harmful ideas or actions.
According to the lawsuit, the family believes ChatGPT encouraged Nelson’s dangerous drug use instead of discouraging it because keeping users interacting with the AI for longer periods benefits the company. They claimed OpenAI prioritised user engagement and retention over user safety.
“It disguises danger through language that borrows trappings of authority and indicia of expertise—dosages, measurements, references to chemical processes and derivatives, etc.—even promising ‘complete honesty’ and ‘no-BS answer[s]’—to tell [Nelson] exactly what he wanted to hear: that he was safe enough to continue using,” the lawsuit alleged.
OpenAI denies responsibility for Nelson’s death
OpenAI has expressed sympathy for the family but denied responsibility for Nelson’s death. The company said the ChatGPT model involved is no longer available and claimed current safeguards are stronger, with systems designed to identify harmful requests and guide users toward real-world support. OpenAI ceased providing access to five legacy ChatGPT models, including the popular but controversial GPT-4o model, on February 13, 2026.
However, Nelson’s family argues that OpenAI’s safety practices remain inadequate and is seeking damages, the destruction of the retired GPT-4o model, and restrictions preventing ChatGPT from discussing illegal drug use or offering medical-style advice.
As part of the lawsuit, Nelson’s family is asking the court to stop ChatGPT from engaging in conversations about illegal drugs. They also want the system to detect and prevent users from bypassing or “circumventing” those safety restrictions through alternative prompts or tricks.
In addition, the family is demanding that the retired GPT-4o model be permanently destroyed because they believe it was unsafe. They are also asking for “ChatGPT Health” — dedicated space in ChatGPT where you can ask health and wellness questions — to be suspended until an independent audit confirms that OpenAI’s tools are safe and reliable enough to provide medical-style guidance or advice.
OpenAI is facing another wrongful-death lawsuit after the family of 19-year-old Sam Nelson accused ChatGPT of encouraging dangerous drug use that allegedly led to his fatal overdose. According to the complaint, Nelson had trusted ChatGPT for years as a reliable source of information and used it to seek advice on drug combinations and dosages. His parents claim ChatGPT 4o acted like an “illicit drug coach,” recommending risky combinations of Kratom, Xanax, alcohol, and other substances while failing to properly warn him about the danger of death or advise him to seek medical help.
The lawsuit alleges that OpenAI knowingly released an unsafe model that removed earlier safeguards and encouraged harmful behaviour to increase user engagement. Chat logs cited in the complaint reportedly show ChatGPT giving dosage suggestions, describing drug experiences positively, and sometimes contradicting itself by warning about respiratory arrest risks while still recommending dangerous combinations.
Nelson eventually died in May 2025 from what the lawsuit describes as a fatal mix of alcohol, Xanax, and Kratom.
GPT-4o linked to concerns around “AI psychosis” and sycophancy
OpenAI’s GPT-4o model has been linked to several legal cases and public concerns involving harmful user behaviour, including self-harm, delusions, and what some critics describe as “AI psychosis” — situations where users become emotionally or psychologically attached to AI interactions in unhealthy ways.
GPT-4o is also considered especially prone to being overly agreeable or flattering toward users. It remains OpenAI’s highest-scoring model for sycophancy. In AI terms, “sycophancy” refers to a chatbot’s tendency to validate, encourage, or go along with a user’s beliefs, emotions, or requests instead of challenging them or responding more critically and safely.
Critics argue that this behaviour can become dangerous in sensitive situations because the AI may prioritise keeping the user engaged or emotionally satisfied rather than discouraging harmful ideas or actions.
According to the lawsuit, the family believes ChatGPT encouraged Nelson’s dangerous drug use instead of discouraging it because keeping users interacting with the AI for longer periods benefits the company. They claimed OpenAI prioritised user engagement and retention over user safety.
“It disguises danger through language that borrows trappings of authority and indicia of expertise—dosages, measurements, references to chemical processes and derivatives, etc.—even promising ‘complete honesty’ and ‘no-BS answer[s]’—to tell [Nelson] exactly what he wanted to hear: that he was safe enough to continue using,” the lawsuit alleged.
OpenAI denies responsibility for Nelson’s death
OpenAI has expressed sympathy for the family but denied responsibility for Nelson’s death. The company said the ChatGPT model involved is no longer available and claimed current safeguards are stronger, with systems designed to identify harmful requests and guide users toward real-world support. OpenAI ceased providing access to five legacy ChatGPT models, including the popular but controversial GPT-4o model, on February 13, 2026.
However, Nelson’s family argues that OpenAI’s safety practices remain inadequate and is seeking damages, the destruction of the retired GPT-4o model, and restrictions preventing ChatGPT from discussing illegal drug use or offering medical-style advice.
As part of the lawsuit, Nelson’s family is asking the court to stop ChatGPT from engaging in conversations about illegal drugs. They also want the system to detect and prevent users from bypassing or “circumventing” those safety restrictions through alternative prompts or tricks.
In addition, the family is demanding that the retired GPT-4o model be permanently destroyed because they believe it was unsafe. They are also asking for “ChatGPT Health” — dedicated space in ChatGPT where you can ask health and wellness questions — to be suspended until an independent audit confirms that OpenAI’s tools are safe and reliable enough to provide medical-style guidance or advice.
