Identifying, Analyzing and Modeling Emotions in Collaborative Problem Solving

Abstract

Collaborative problem-solving (CPS) is an educational paradigm where two or more individuals work together to solve a problem. In such settings, individuals experience a range of emotions that influences the collaboration process. Automatically detecting these emotions could better support CPS, but existing affect detection approaches are understudied in this space. In this thesis, I investigate several key aspects involved in identifying, analyzing and modeling emotions during CPS. I begin by collecting and exploring a dataset of internal monologues from individuals who completed CPS tasks in groups using retrospective cued-recall. This data shows common epistemic emotions prevalent in such settings and highlights that there is little overlap with the conventional basic emotions generally used in most affective computing research. I then analyze how these emotions evolve throughout collaboration using Ordered Network Analysis, showing that collaborative emotions emotions exhibit an epistemic core -- with strong transitional tendencies among the productive states of confusion, curiosity and optimism. I also show that this network varies significantly across slow and fast groups, with fast groups reporting less struggle emotions but more disengagement. I then evaluate several facial emotion recognition systems trained on prototypical, frame-level basic emotions and find limited alignment with these reported epistemic emotions, suggesting that conventional affective representations may not transfer cleanly to CPS settings. Finally, I investigate multimodal approaches using facial behavior and speech features across different temporal windows, showing that different collaborative emotions are expressed through different modalities and timescales, unlike the single-frame unimodal datasets most detectors are trained on. Together, these findings highlight important limitations in current affect detection approaches for CPS.