Reinforcement Teaching

Abstract

While traditional machine learning algorithms learn to solve a task directly, meta- learning aims to learn about and improve another learning algorithm’s performance. However, existing meta-learning methods either only work with differentiable algo- rithms or are handcrafted to improve a specific component of an algorithm. There- fore, we develop a unifying meta-learning framework called reinforcement teaching to improve the learning process of any algorithm. Within the reinforcement teaching framework, a teaching policy is learned through reinforcement to improve a student’s learning. To effectively learn such a teaching policy, we develop a reward function based on learning progress, allowing the teacher’s policy to maximize the student’s performance more quickly. Further, we introduce a parametric-behavior embedder that learns a representation of the student’s learnable parameters from its input- output behavior. Finally, to demonstrate the effectiveness of reinforcement teaching, we perform a case study applying reinforcement teaching to the automatic curricu- lum learning domain. In this setting, a curriculum policy is learned that selects sub-tasks for a reinforcement learning student, outperforming handcrafted heuristics and previously proposed reward functions. To that end, reinforcement teaching is a framework capable of unifying different meta-learning approaches while effectively leveraging existing tools from reinforcement learning research.