The Potential of Adaptive Learning Systems to Enhance Learning Outcomes: A Meta-Analysis

Abstract

Adaptive learning systems (ALSs) serve as a way to personalize learning experiences for students to improve their learning outcomes. Many studies have been conducted to develop and examine ALSs in terms of their effects on student learning outcomes compared to the traditional classroom settings. Comparisons of these studies revealed that different ALSs exhibited varying degrees of success in promoting learning achievement (i.e., discrepant magnitudes of system effectiveness). However, little work has empirically examined factors that impact the effectiveness of ALSs, thus constraining their application in practice. This study performed a meta-analysis of 46 studies on ALSs in order to identify factors significantly accounting for the variation in system effectiveness. The analyses of 77 effect sizes confirmed substantial heterogeneity in system effectiveness (Mean = 1.48; Range = .09-9.06). Using three-level correlated and hierarchical effects modeling, the heterogeneity was explained by the variability both within publications (i.e., 12.06%) and between publications (i.e., 83.47%). Specifically, the magnitudes of system effectiveness were significantly moderated by learner characteristics and modeling approaches. Moreover, in comparison with other subject areas such as mathematics and computer science, ALSs used to support the learning of the English language were likely associated with higher system effectiveness. No evidence of publication bias was detected in these data. Findings from the present research facilitate the understanding of what and how system components are associated with the effectiveness of ALSs and inform decision making on their design. Implications for pedagogical theories and practice, limitations of this research, and future directions for developing and implementing ALSs in educational settings are discussed.