Beyond Clever Hans: Learning From People Without Their Really Trying

Abstract

Facial expressions and other body language are important for human commu- nication. They complement speech and make the process of communication simple and sustainable. However, the process of communication using existing approaches to human-machine interaction is not intuitive as that of human communication. Specifically, the existing approaches to human machine inter- action do not learn from whatever subtle non-verbal cues produced by a user. Many of the existing approaches map body language cues to instructions or re- wards and use them to train a machine. These mappings are not learned from ongoing interactions and are assumed to be defined by some external source. As a consequence, the communicative process through these approaches can cause significant cognitive load on the user. This is an important problem that needs to be addressed if we are to amplify our existing cognitive and physical capabilities through intelligent machines. Towards addressing this, we intro- duce our idea that allows machines to learn from whatever subtle cues people produce during the process of interaction. Particularly, the agent learns a value function that maps the userâs non-verbal cues to later rewards. By using this value function, the user can teach their agent to complete a task according to their preferences, and consequently maximize their satisfaction. We demon- strate this by training an agent using facial expressions. Furthermore, we show that these learned value functions can be successfully transferred across tasks. In conclusion, our approach is the first that allows people to teach their machines using whatever subtle cues they produce and could take us far in achieving sustainable forms of human-machine interaction.