Face analysis using ensemble convolutional neural networks for driver drowsiness detection

Abstract

Driver drowsiness is one of the major factors which lead to very severe and fatal accidents in day to day life. Globally millions of people have died in road accidents annually, on average, globally 3 700 people lose their lives everyday on the road. Young adults aged between 15-30 account for more than half of all road accident deaths. Many more lose their lives due to accidents every day in other forms of travel. Drowsiness and distractions are the main causes of these fatalities. Behavioral or facial features like eyes closing or too much blinking, yawning and nodding are considered important features for recognizing one's drowsiness state. It has been found that the available Deep Learning models are not able to perform well for facial features detection like eyes detection (blinking and closing) and mouth detection (yawning) and also for pose variations and occlusions (wearing sunglasses). Therefore, a specialized Ensemble Convolutional Neural Network (CNN) configured with three CNN models which are better than one single CNN has been developed in this work that is invariant to pose and occlusion. The systematic methodology of drowsiness detection has been described in detail with their results in this study. The experiment has been carried out on Yawning Detection Dataset (Yaw DD) for yawning detection and the University of Texas at Arlington Real-Life Drowsiness Dataset (UT A-RLDD) for blink feature detection, to examine the extent of drowsiness depending on the frequency of yawning and blinking with certain pose and occlusion variation from these datasets. Results of the FI Score that were computed from the Confusion Matrix of CNN 1, CNN 2 and CNN 3 of YawDD are 0.92, 0.90 and 0.912 and that from UTA-RLDD are 0.901, 0.921 and 0.942 respectively and our proposed Ensemble CNN configuration of Yaw DD and UTA-RLDD yielded Fl score of 0.935 and 0.954 respectively. The proposed Ensemble CNN configuration showed more promising results than individual CNN's.