A Framework for Associating Mobile Devices to Individuals Based on Identification of Motion Events

Abstract

The ubiquity of the Internet-of-Things (IoT) devices in everyday life allows various sensors to be utilized in networked systems for solving a number of real-world problems. Models utilizing specific sensing modalities achieve impressive performance in understanding human activity and are used in systems developed for monitoring and improving indoor living conditions. A combination of multiple sensors could even allow a better understanding of the environment. Nevertheless, certain sensing modalities may not have a direct correlation in their measurements, hence, making the fusion of the sensor data quite challenging. This thesis studies the feasibility and design of a sensor fusion system that can associate two unrelated sensing modalities, namely radio frequency and visual domains, by identifying and associating events, human motion, that leaves a signature in both domains. We present a holistic framework for associating a mobile device unique identifier to an individual holding it during a certain activity. We study different motion detection methods that rely on the analysis of Received Signal Strength Identifier (RSSI) combined with state-of-the-art Computer Vision approaches to object tracking. We run field experiments to evaluate the performance of different motion detection methods and use the proposed framework to associate mobile devices to individuals who hold or carry them. Our results indicate that an accuracy of 75% can be achieved in the device-to-individual association task.