Human and machine recognition of transportation modes from body-worn camera images

Richoz, Sebastien (Wearable Technologies Lab, University of Sussex, Brighton, United Kingdom) ; Ciliberto, Mathias (Wearable Technologies Lab, University of Sussex, Brighton, United Kingdom) ; Wang, Lin (Centre for Intelligent Sensing, Queen Mary University London, London, United Kingdom) ; Birch, Phil (Engineering and Informatics, University of Sussex, Brighton, United Kingdom) ; Gjoreski, Hristijan (Faculty of electrical Engineering and Information Technologies, Ss Cyril and Methodius University, Skopje, Macedonia) ; Perez-Uribe, Andres (School of Management and Engineering Vaud, HES-SO // University of Applied Sciences Western Switzerland) ; Roggen, Daniel (Wearable Technologies Lab, University of Sussex, Brighton, United Kingdom)

Computer vision techniques applied on images opportunistically captured from body-worn cameras or mobile phones offer tremendous potential for vision-based context awareness. In this paper, we evaluate the potential to recognise the modes of locomotion and transportation of mobile users, by analysing single images captured by body-worn cameras. We evaluate this with the publicly available Sussex-Huawei Locomotion and Transportation Dataset, which includes 8 transportation and locomotion modes performed over 7 months by 3 users. We present a baseline performance obtained through crowd sourcing using Amazon Mechanical Turk. Humans infered the correct modes of transportations from images with an F1-score of 52%. The performance obtained by five state-of-the-art Deep Neural Networks (VGG16, VGG19, ResNet50, MobileNet and DenseNet169) on the same task was always above 71.3% F1- score. We characterise the effect of partitioning the training data to fine-tune different number of blocks of the deep networks and provide recommendations for mobile implementations. Index Terms—Activity recognition, Body-worn camera, Computer Vision, Deep learning, Crowd sourcing, Mechanical Turk.

Conference Type:
full paper
Ingénierie et Architecture
IICT - Institut des Technologies de l'Information et de la Communication
Washington, WA, USA, 26-29 April 2019
Washington, WA, USA
26-29 April 2019
6 p.
Published in:
Proceedings of ABC 2019 : Activity and Behavior Computing Conference, 26-29 April 2019, Eastern Washington University, WA, USA
External resources:
Appears in Collection:

Note: The status of this file is: restricted

 Record created 2019-07-16, last modified 2020-10-27

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)