Specific Machine Curiosity

Abstract

Curiosity appears to motivate and guide effective learning in humans, which has led to high hopes in the machine learning community for machine analogues of curiosity. While a variety of machine curiosity algorithms have been introduced, they are rarely compared with other existing curiosity algorithms. With a new family of experimental domains—‘Curiosity Bandits’—I provide a means of observing curiosity methods on an even playing field, manipulating the curiosity mechanism while controlling for the learning algorithm and its environment. Observations using these domains, along with the study of human and animal curiosity, allowed me to clarify five properties that would offer important benefits for machine learners but have not yet been well-explored in machine intelligence—directedness towards inostensible referents, cessation when satisfied, voluntary exposure, transience, and coherent long-term learning. I further demonstrate how three of these properties can be implemented together in a proof-of-concept reinforcement learning agent. As a whole, this work presents a novel view into machine curiosity and how it might be integrated into the behaviour of goal-seeking, decision-making machine agents in complex environments.