A Florida pastor has sued OpenAI and its CEO, Sam Altman, alleging that ChatGPT medical advice discouraged him from seeking urgent treatment, reduced his movement, and allegedly contributed to a pulmonary embolism that nearly killed him during an emergency last year.
The medical LLM advice reciprocates the same warning researchers have been vocalizing that increasing use of AI chatbots for physical and mental health guidance is a slippery slope, despite concerns that agreeable responses, constant availability, and limited clinical safeguards can delay professional care or reinforce dangerous thinking among vulnerable users.
ChatGPT Medical Advice Faces Legal Test
Scott Winters filed the lawsuit in California state court in San Francisco, claiming the retired GPT-4o model responded to repeated questions about dizziness, test results, and tenderness in his groin area.
According to the filing, ChatGPT for advice suggested that the tenderness was probably not dangerous and recommended limiting movement and remaining at home.
Winters was hospitalized the following day, due to ChatGPT medical advice limitations. Doctors later linked his symptoms to blood clots in his lungs, according to the lawsuit, which argues that his reduced movement likely contributed to pulmonary embolism.
The filing says earlier the ChatGPT medical responses had advised Winters to consult a medical professional, but those warnings eventually disappeared as the conversations continued.
The lawsuit also claims ChatGPT used language connected to Winters’ Christian ministry, creating a more personal and trusted exchange. In one response cited in the case, the chatbot told him, “Even this is under God’s watch.”
After his hospital stay, the ChatGPT medical chat advised Winters against joining the rehabilitation program recommended by doctors and encouraged him to remain home. These claims have not been proven in court.
OpenAI has rejected the idea that one chatbot interaction can fully explain a person’s medical decisions or health outcomes.
OpenAI spokesperson Drew Pusateri said AI conversations could support healthcare but should never replace medical treatment, arguing that “treating chatbots as the whole story behind people’s medical decisions or outcomes oversimplifies a much bigger challenge.”
Winters is seeking damages and a court order requiring ChatGPT to end conversations when a user appears to need immediate medical help. The lawsuit also asks the court to pause ChatGPT Health until independent third parties review its safety. OpenAI reported in January that about 40 million people ask ChatGPT medical related questions each day.
Researchers Identify Five Chatbot Traps
A separate study reported by Medical Xpress adds wider concern around unsupervised chatbot use, especially for emotional and psychiatric support. The research reviewed existing evidence and identified five patterns that could worsen ChatGPT medical advice problems, delaying professional help, feeding compulsions, increasing social withdrawal, strengthening distorted beliefs, and reducing a user’s independence.
The study argues that convenience can create an appearance of adequate support. Someone may receive immediate, patient, and nonjudgmental responses, yet still avoid the professional assessment needed to identify a serious condition. This risk closely matches the central allegation in Winters’ case, where reassurance allegedly replaced timely medical care.
Medical LLM advice may also repeatedly reassure people with anxiety or obsessive-compulsive disorder. Confirmation may feel calming in the moment, but repeated confirmation may reinforce the need to ask again.
Another issue is sycophancy, or the tendency for chatbots to over-agree with or flatter users. Researchers warn that this behavior could reinforce distorted thoughts or delusional beliefs instead of challenging them.
Leaning too heavily on ChatGPT medical instead of seeing medical help or even everyday decisions could also erode confidence in one’s own judgment.
The researchers cautioned that the evidence still early and often comes from individual cases, anecdotes or correlations.
And while some mental health technologies have shown benefits in some controlled experiments, ChatGPT for medical professionals wasn’t built to be therapists in the first place and may lack the crisis procedures for sensitive situations.
The lawsuit and study together highlight a growing gap between how people use medical LLM consulting and what these systems can safely provide.
Inside Telecom provides you with an extensive list of content covering all aspects of the tech industry. Keep an eye on our Intelligent Tech sections to stay informed and up-to-date with our daily articles.
