Deep learning-based safety monitoring system for personal mobility devices, focusing on helmet usage and driving behavior
Overview
This project was conducted as part of my graduation requirements, focusing on developing a deep learning-based safety system specifically designed for personal mobility devices.
Key Features
Utilized both user-side and roadside images to monitor and detect various safety factors:
Helmet usage
Forward-looking behavior
Adherence to driving rules on sidewalks and near intersections
Designed for real-time operation using Jetson Nano hardware.
Role
Overall Training: Developed and trained deep-learning models.
Hardware Integration: Set up and embedded models on Jetson Nano.
Knowledge Distillation: Applied knowledge distillation techniques to optimize model performance.
The global personal mobility market has been expanding rapidly due to its convenience. However, the rising number of accidents involving personal mobility devices, including falls, collisions, and incidents with moving vehicles or objects, has become a significant concern. In this paper, we propose a deep learning-based safe driving system that integrates both user and road images to address these safety issues. Our system leverages CNN-based models to simultaneously perform the following tasks: 1) detecting if the user is wearing a helmet, 2) ensuring the user is looking ahead, 3) identifying whether the scooter is being ridden on the sidewalk, and 4) recognizing proximity to intersections. These tasks operate in parallel to provide a comprehensive assessment of the driving environment. The system determines the scooter’s final speed by selecting the minimum speed value from all tasks to ensure maximum safety. Additionally, we utilize knowledge distillation techniques to compress the models, enabling real-time inference on edge devices. This approach delivers a fast, efficient, and highly accurate system, specifically tailored to the needs of personal mobility users.
2022
2-input Deep Learning based Multi-tasking Safe Driving (in Korean)
The rapid growth of electric scooter-sharing services has brought personal mobility devices into the spotlight. However, this rise in popularity has been accompanied by increasing public concern over e-scooter accidents. To address these safety challenges, we propose a deep learning-based safe driving system for electric scooters that analyzes both user and road images. Our framework employs CNN-based models to determine whether the user is wearing a helmet or maintaining proper forward vision by analyzing user images. Simultaneously, the system verifies if the scooter is on a designated road and estimates the distance between the user and the nearest pedestrian from road images. Finally, it calculates the scooter’s optimal velocity using the outputs from these models. By leveraging a fast and precise deep learning approach, our system enhances real-time decision-making and provides a high level of confidence in detecting and mitigating multiple hazards simultaneously.
