Car accidents occur every 60 seconds. To that end, every driver needs any help they can get, to point out obstacles and dangerous conditions. Precarious circumstances on the road place drivers and autonomous vehicles in dangerous situations. Difficult contexts include complex driving scenes, speeding, and distracted driving. All of these situations and more increase the likelihood of accidents which can lead to damage, injury, or even death.
With an eye towards improving safety, researchers in Stony Brook’s College of Engineering and Applied Sciences are working on an AI-driven solution. Professor Zhaozheng Yin of the Department of Biomedical Informatics and Department of Computer Science, Professor Ruwen Qin, and PhD candidates Muhammad Monjurul Karim and Yu Li, of the Department of Civil Engineering have presented a multi-faceted system of deep neural networks for driving scene analysis: Multi-Net.
“The creation of Multi-Net is motivated by the complexity of driving scene analysis for crash risk assessment and prevention,” says Qin.
Human drivers and autonomous vehicles alike are at a great risk as soon as they hit the road. Research on traffic crashes, especially with an AI-driven focus, is sparse, which serves to the detriment of both human drivers and autonomous vehicles. Without ample research and solutions, car accidents continue to perpetuate in high volumes. Therefore, in order to jumpstart work with Multi-Net, two brand new datasets of annotated images were created and presented made public online.
The creation of two large-scale datasets is an invaluable contribution to research efforts. These datasets make it possible for other researchers to expand upon current literature, with the hopes to make driving safer for human drivers and autonomous vehicles alike.
Dash cameras coupled with deep learning is the foundation for Multi-Net. Within this deep learning system, dash cameras are of chief importance in order to extract valuable information from in-road scenarios. With this on-scene view in place, AI is set up to take the wheel.
“The proposed system can be integrated into autonomous vehicles and human-driving cars with a dashcam to assess the fatal crash risk of driving scenes,” says Qin.
Deep learning methods fuel Multi-Net, a two-pronged approach in assessing driving scenes to supplement risk assessments. Multi-Net consists of two neural networks which classify and provide multiple labels for driving scenes. The use of Convolutional Neural Networks (CNNs) in the context of Computer Vision (CV) tasks has proved itself deeply efficacious and versatile.
The two deep neural networks which constitute this system are namely YOLO v3 and DeepLab v3. YOLO v3 is an object detector for things like pedestrians and vehicles. DeepLab v3 is an image segmentation algorithm that functions to survey the driving area, in being able to segment the driveable area, and recognize lanes and traffic signs. Multi-Net exists parallel to these two neural networks on its own, classifying and studying road function, weather, and time of day.
Overall, Multi-Net is responsible for identifying four possible labels for each driving scene: 1. pre-crash, crash, no-crash, 2. arterial, collector, interstate, local, 3. rainy, snowy, clear, overcast, foggy, and 4. daytime, nighttime, dawn/dusk. The system’s keen ability to identify and classify each of these risk indicators serves in support of crash risk assessment and crash prevention.
Crash prevention is integral in making our roads safer. This amplified safety is rooted in this system’s ability to expand the situational awareness for drivers and autonomous vehicles alike. The system is efficacious as it accurately pinpoints obtrusive objects and segments the road ahead.
The work done by these Stony Brook faculty, through the Multi-Net, is renowned and viable for improving driving safety. The Multi-Net was accepted and presented at the 100th Transportation Research Board annual meeting, with the potential to captivate the attention of other transportation institutions.
There is more work to be done on the road ahead. Vision sensor-based driving scene analysis could benefit from being able to recognize and predict traffic density in tandem with other on-scene analyses and would give a more apt risk assessment.
“The Multi-Net will augment drivers’ risk awareness or assist them in decision-making. To form an effective collaboration between the Multi-Net and drivers or Advanced Driver Assistance Systems (ADAS), we are calibrating the strengths and weaknesses of AI-powered safety enhancement systems against humans. Furthermore, we envision the Multi-Net will be integrated with other AI methods to enhance traffic safety from various aspects. Recently, we developed a network with Dynamic Spatial-Temporal Attention (DSTA) to predict if an accident would occur shortly. The accident risk awareness provided by the Multi-Net and the accident anticipation from the DSTA network complement each other on the time dimension,” says Qin.
-Alyssa Dey, Communications Assistant