Automatically Detecting Affect in Computer-based Learning Environments: A Systematic Literature Review

Abstract

Affect detection is increasingly viewed as an essential component of computer-based learning systems because it aims to improve learner outcomes by adapting to the learner’s affect. However, most computer-based learning environments used across formal and informal educational contexts do not respond to students’ affective needs. Moreover, it is not clear which affective states should be assessed and which states have a positive or a negative effect on student learning. The aim of this review is to examine how affect is automatically detected and analyzed via affect-sensitive computational systems in educational settings. This systematic literature review analyzes 36 peer-reviewed publications that focus on finding relationships between affect and learning in computational applications. Evidence from the reviewed articles shows that most studies (1) were published in the last four years; (2) mainly used facial expressions to detect affect; (3) identified engagement, boredom, frustration, and confusion as the most frequent affective states in learning settings; and (4) used supervised machine learning algorithms to classify learner emotions. The present review identifies the following gaps in the related literature. First, it revealed that there is a paucity of studies in non-STEM domains and that sample K-12 students and participants from countries other than the US, given that two-thirds of the reviewed studies sampled university students, almost half of the studies sampled participants from North America, and almost three quarters of the studies focused on STEM contexts. Second, it identified facial expression as the most common physiological and behavioral data channel, with system log data being the most frequent performance-related channel. Third, it found that few studies examined both affect and achievement measures. Finally, it revealed that few studies employed unsupervised learning techniques or supervised learning regressors, given that supervised learning classifiers were overwhelmingly employed to predict affective states. This research provides recommendations on how to address these gaps, including the need for more methodological approaches, both theory- and data-driven, in capturing and analyzing affect. This review suggests the exploration and development of adaptive intelligent educational interfaces that use affective and behavioral states to provide a better learning experience by offering suitable responses. Likewise, the review suggests the exploration of creating affective datasets to improve existing machine learning affect detecting models.