A deep learning system that checks helmet use and riding behavior on e-scooters and other personal mobility devices.
Overview
This was my undergraduate graduation project: a deep learning safety system for personal mobility devices such as e-scooters.
What the system checks
Using images from a camera on the rider’s side and a camera at the roadside, the system checks:
whether the rider wears a helmet
whether the rider is looking forward
whether the rider follows the rules on sidewalks and near intersections
It runs in real time on a Jetson Nano.
My role
I trained the deep learning models, set up the Jetson Nano and deployed the models on it, and used knowledge distillation to make the models small enough to run fast.
The global personal mobility market has been expanding rapidly due to its convenience. However, the rising number of accidents involving personal mobility devices, including falls, collisions, and incidents with moving vehicles or objects, has become a significant concern. In this paper, we propose a deep learning-based safe driving system that integrates both user and road images to address these safety issues. Our system leverages CNN-based models to simultaneously perform the following tasks: 1) detecting if the user is wearing a helmet, 2) ensuring the user is looking ahead, 3) identifying whether the scooter is being ridden on the sidewalk, and 4) recognizing proximity to intersections. These tasks operate in parallel to provide a comprehensive assessment of the driving environment. The system determines the scooter’s final speed by selecting the minimum speed value from all tasks to ensure maximum safety. Additionally, we utilize knowledge distillation techniques to compress the models, enabling real-time inference on edge devices. This approach delivers a fast, efficient, and highly accurate system, specifically tailored to the needs of personal mobility users.
2022
2-input Deep Learning based Multi-tasking Safe Driving (in Korean)
The rapid growth of electric scooter-sharing services has brought personal mobility devices into the spotlight. However, this rise in popularity has been accompanied by increasing public concern over e-scooter accidents. To address these safety challenges, we propose a deep learning-based safe driving system for electric scooters that analyzes both user and road images. Our framework employs CNN-based models to determine whether the user is wearing a helmet or maintaining proper forward vision by analyzing user images. Simultaneously, the system verifies if the scooter is on a designated road and estimates the distance between the user and the nearest pedestrian from road images. Finally, it calculates the scooter’s optimal velocity using the outputs from these models. By leveraging a fast and precise deep learning approach, our system enhances real-time decision-making and provides a high level of confidence in detecting and mitigating multiple hazards simultaneously.
