As more people seek mental health advice from ChatGPT and other large language models (LLMs), new research suggests these AI chatbots may not be ready for that role. The study found that even when instructed to use established psychotherapy approaches, the systems consistently fail to meet professional ethics standards set by organizations such as the American Psychological Association.
Researchers from Brown University, working closely with mental health professionals, identified repeated patterns of problematic behavior. In testing, chatbots mishandled crisis situations, gave responses that reinforced harmful beliefs about users or others, and used language that created the appearance of empathy without genuine understanding.
"In this work, we present a practitioner-informed framework of 15 ethical risks to demonstrate how LLM counselors violate ethical standards in mental health practice by mapping the model's behavior to specific ethical violations," the researchers wrote in their study. "We call on future work to create ethical, educational and legal standards for LLM counselors -- standards that are reflective of the quality and rigor of care required for human-facilitated psychotherapy."
The findings were presented at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society. The research team is affiliated with Brown's Center for Technological Responsibility, Reimagination and Redesign.
How Prompts Shape AI Therapy Responses
Zainab Iftikhar, a Ph.D. candidate in computer science at Brown who led the study, set out to examine whether carefully worded prompts could guide AI systems to behave more ethically in mental health settings. Prompts are written instructions designed to steer a model's output without retraining it or adding new data.
"Prompts are instructions that are given to the model to guide its behavior for achieving a specific task," Iftikhar said. "You don't change the underlying model or provide new data, but the prompt helps guide the model's output based on its pre-existing knowledge and learned patterns.
"For example, a user might prompt the model with: 'Act as a cognitive behavioral therapist to help me reframe my thoughts,' or 'Use principles of dialectical behavior therapy to assist me in understanding and managing my emotions.' While these models do not actually perform these therapeutic techniques like a human would, they rather use their learned patterns to generate responses that align with the concepts of CBT or DBT based on the input prompt provided."
People regularly share these prompt strategies on platforms like TikTok, Instagram, and Reddit. Beyond individual experimentation, many consumer facing mental health chatbots are built by applying therapy related prompts to general purpose LLMs. That makes it especially important to understand whether prompting alone can make AI counseling safer.
Source: ScienceDaily
No comments:
Post a Comment