Experiments with Word Embeddings for Sequential Questioning

Abstract

As a student learns to program, there will be gaps in the student's knowledge that must be addressed for the student to gain a full understanding of the material. A student's answer to a single question may provide some insight into the student's level of understanding. However, a well-chosen sequence of questions might more accurately identify any misunderstandings. For example, the popular 20 Questions game relies on a sequence of well-chosen questions for one player to guess what another player is thinking. Inspired by the 20 Questions game, we suggest a method to select the next question in a sequence to identify gaps in a student's understanding. We model introductory computing science terms with word embeddings trained from a collection of Python course notes and textbooks. We also introduce a test suite of computing science concepts. Each of the 17 tests is an algebraic equation and each term in the equation is represented by one of our word embeddings. Thus, a test can be evaluated to produce a result that corresponds to another word embedding in the model. The test suite represents a collection of concepts and skills that an introductory computing science student must learn. We demonstrate that we can represent a computing science concept by adding relevant substituent concepts and removing irrelevant concepts. We then posit that this ability can be used to diagnose the gap in understanding and recommend a relevant next question based on a student's answers so far.