Too Positive to Be True? Sentiment Analysis of AI-Guided vs. Human-Led Debriefings in Virtual Reality Counseling Training

Debriefings are a crucial component of simulation-based learning, as they allow participants to critically reflect on their performance and translate experiences into knowledge. With the rise of artificial intelligence (AI), large language models (LLMs) are increasingly being explored as conversational agents in debriefings. However, concerns have been raised that AI debriefers may display sycophantic tendencies, characterized by overly positive sentiment and a lack of critical feedback. Such tendencies could affect learners’ self-evaluations and limit the educational value of AI-supported debriefings. In this study, 45 university students participated in a virtual reality (VR) counseling simulation, followed by either an AI-guided or a human-led debriefing. Sentiment analysis of 958 debriefing statements (350 AI, 608 human) was conducted using a German sentiment classification model based on Google’s BERT. Results show that AI debriefers expressed significantly more positive sentiment and substantially fewer negative statements compared to human debriefers, corroborating the assumption of a sycophancy effect. However, sentiment was not found to predict self-rated counseling competence, or self-efficacy. These findings highlight both the promise and the limitations of AI-supported debriefings in counseling education. While AI debriefers provide consistent encouragement, their lack of critical feedback raises questions about their long-term effectiveness for professional skill development. Future research should explore hybrid approaches that combine AI scalability with human expertise in constructive reflection.