Video and Text-Based Affect Analysis of Children in Play Therapy

Abstract

Play therapy is an approach to psychotherapy where a child is engaging in play activities. Because of the strong affective component of play, it provides a natural setting to analyze feelings and coping strategies of the child. In this paper, we investigate an approach to track the affective state of a child during a play therapy session. We assume a simple, camera-based sensor setup, and describe the challenges of this application scenario. We use fine-tuned off-the-shelf deep convolutional neural networks for the processing of the child's face during sessions to automatically extract valence and arousal dimensions of affect, as well as basic emotional expressions. We further investigate text-based and body-movement based affect analysis. We evaluate these modalities separately and in conjunction with play therapy videos in natural sessions, discussing the results of such analysis and how it aligns with the professional clinicians' assessments.