We’re excited to have Adi Ganesan, a PhD researcher at Stony Brook University, the University of Pennsylvania, and Vanderbilt, on the show. We’ll talk about how large language models LLMs) are being tested and used in psychology, citing examples from mental health research. Fun fact: Adi was Sid's research partner during his Ph.D. program.

Discussion highlights

• Language models struggle with certain aspects of therapy including being over-eager to solve problems rather than building understanding
• Current models are poor at detecting psychomotor symptoms from text alone but are oversensitive to suicidality markers
• Cognitive reframing assistance represents a promising application where LLMs can help identify thought traps
• Proper evaluation frameworks must include privacy, security, effectiveness, and appropriate engagement levels
• Theory of mind remains a significant challenge for LLMs in therapeutic contexts; example: The Sally-Anne Test.
• Responsible implementation requires staged evaluation before patient-facing deployment

Resources

To learn more about Adi's research and topics discussed in this episode, check out the following resources:

• Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation
• Therapist Behaviors paper: [2401.00820] A Computational Framework for Behavioral Assessment of LLM Therapists 
• Cognitive reframing paper: Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction - ACL Anthology 
• Faux Pas paper: Testing theory of mind in large language models and humans | Nature Human Behaviour 
• READI: Readiness Evaluation for Artificial Intelligence-Mental Health Deployment and Implementation (READI): A Review and Proposed Framework 
• Large language models could change the future of behavioral healthcare: A proposal for responsible development and evaluation | npj Mental Health Research 
• GPT-4’s Schema of Depression: Explaining GPT-4’s Schema of Depression Using Machine Behavior Analysis
• Adi’s Profile: Adithya V Ganesan - Google Scholar 

What did you think? Let us know.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.