Hostname: page-component-7dd5485656-wlg5v Total loading time: 0 Render date: 2025-10-30T04:38:43.752Z Has data issue: false hasContentIssue false
  • English
  • Français

Face salient points and eyes tracking for robust drowsiness detection

Published online by Cambridge University Press:  07 September 2011

J. Jimenez-Pinto  and
M. Torres-Torriti
J. Jimenez-Pinto
Affiliation:
Dept. of Electrical Engineering, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Casilla 306-22, Santiago, Chile. E-mail: jejimenp@puc.cl
M. Torres-Torriti*
Affiliation:
Dept. of Electrical Engineering, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Casilla 306-22, Santiago, Chile. E-mail: jejimenp@puc.cl
*
*Corresponding author. E-mail: mtorrest@ing.puc.cl
Get access Rights & Permissions [Opens in a new window]

Summary

Measuring a driver's level of attention and drowsiness is fundamental to reducing the number of traffic accidents that often involve bus and truck drivers, who must work for long periods of time under monotonous road conditions. Determining a driver's state of alert in a noninvasive way can be achieved using computer vision techniques. However, two main difficulties must be solved in order to measure drowsiness in a robust way: first, detecting the driver's face location despite variations in pose or illumination; secondly, recognizing the driver's facial cues, such as blinks, yawns, and eyebrow rising. To overcome these challenges, our approach combines the well-known Viola–Jones face detector with the motion analysis of Shi–Tomasi salient features within the face. The location of the eyes and blinking is important to refine the tracking of the driver's head and compute the so-called PERCLOS, which is the percentage of time the eyes are closed over a given time interval. The latter cue is essential for noninvasive driver's alert state estimation as it has a high correlation with drowsiness. To further improve the location of the eyes under different conditions of illumination, the proposed method takes advantage of the high reflectivity of the retina to near infrared illumination employing a camera with an 850 nm wavelength filter. The paper shows that motion analysis of the salient points, in particular cluster mass centers and spatial distributions, yields better head tracking results compared to the state-of-the-art and provides measures of the driver's alert state.

Keywords

Alert state assessmentFatigue detectionDrowsiness detectionDriver assistanceIR eye trackingYawn detectionEyebrow rising detection

Information

Type
Articles
Information
Robotica , Volume 30 , Issue 5 , September 2012 , pp. 731 - 741
Copyright
Copyright © Cambridge University Press 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

1.Alvarez, S., Sotelo, M. A., Ocaña, M., Llorca, D. F., Parra, I. and Bergasa, L. M., “Perception advances in outdoor vehicle detection for automatic cruise control,” Robotica 28 (5), 765–779 (Sep. 2010).CrossRefGoogle Scholar
2.Bergasa, L. M., Nuevo, J., Sotelo, M. A., Barea, R. and López, M. E., “Real-time system for monitoring driver vigilance,” IEEE Trans. Intell. Transp. Syst. 7 (1), 63–77 (Mar. 2006).CrossRefGoogle Scholar
3.Davison, A. C. and Hinkley, D. V., Bootstrap Methods and Their Application (Cambridge University Press, Cambridge, UK, 1997).CrossRefGoogle Scholar
4.de la Escalera, A. and Armingol, J. M., “Vehicle detection and tracking for visual understanding of road environments,” Robotica 28 (6), 847–860 (Dec. 2010).CrossRefGoogle Scholar
5.Dong, W., Qu, P. and Han, J., “Driver Fatigue Detection Based on Fuzzy Fusion,” Proceedings of the Chinese Control and Decision Conference (CCDC '08), Yantai, Shandong, China (Jul. 2008) pp. 2640–2643.Google Scholar
6.Dong, W. and Wu, X., “Driver Fatigue Detection Based on the distance of Eyelid,” Proceedings of the IEEE International Workshop on VLSI Design and Video Technology, Suzhou, China (May 2005) pp. 365–368.Google Scholar
7.D'Orazio, T., Leo, M., Spagnolo, P. and Guaragnella, C., “A Neural System for Eye Detection in a Driver Vigilance Application,” Proceedings of the Seventh International IEEE Conference on Intelligent Transportation Systems, Washington, D.C., USA (Oct. 2004) pp. 320–325.Google Scholar
8.Fan, X., Yin, B.-C. and Sun, Y.-F., “Yawning Detection for Monitoring Driver Fatigue,” Proceedings of the International Conference on Machine Learning and Cybernetics, Hong Kong (Aug. 2007) vol. 2, pp. 664–668.Google Scholar
9.Fan, X., Yin, B. and Sun, Y., “Nonintrusive Driver Fatigue Detection,” Proceedings of the IEEE International Conference on Networking, Sensing and Control (ICNSC '08), Hainan, China (Apr. 2008) pp. 905–910.Google Scholar
10.Flores, M. J., Armingol, J. M. and Escalera, A., “Real-time warning system for driver drowsiness detection using visual information,” J. Intell. Robot. Syst. 59, 103–125 (Jun. 2010).CrossRefGoogle Scholar
11.Gu, H., Ji, Q. and Zhu, Z., “Active Facial Tracking for Fatigue Detection,” Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision (WACV '02), Orlando, Florida, USA (2002) pp. 137–142.Google Scholar
12.Hansen, D.W. and Ji, Q., “In the eye of the beholder: A survey of models for eyes and gaze,” IEEE Trans. Pattern Anal. Mach. Intell. 32 (3), 478–500 (Mar. 2010).CrossRefGoogle ScholarPubMed
13.Hong, T., Qin, H. and Sun, Q., “An Improved Real Time Eye State Identification System in Driver Drowsiness Detection,” Proceedings of the IEEE International Conference on Control and Automation (ICCA '07), Guangzhou, China (May 30–Jun. 1, 2007) pp. 1449–1453.Google Scholar
14.Horng, W.-B., Chen, C.-Y., Chang, Y. and Fan, C.-H., “Driver Fatigue Detection Based on Eye Tracking and Dynamic Template Matching,” Proceedings of the IEEE International Conference on Networking, Sensing and Control, Taipei, Taiwan (Mar. 2004) vol. 1, pp. 7–12.Google Scholar
15.Ito, T., Mita, S., Kozuka, K., Nakano, T. and Yamamoto, S. “Driver Blink Measurement by the Motion Picture Processing and Its Application to Drowsiness Detection,” Proceedings of the IEEE Fifth International Conference on Intelligent Transportation Systems, Singapore (2002) pp. 168–173.Google Scholar
16.Jain, A., Fundamentals of Digital Image Processing (Prentice-Hall, Englewood Cliff, NJ, USA, 1986).Google Scholar
17.Ji, Q., Zhu, Z. and Lan, P., “Real-time nonintrusive monitoring and prediction of driver fatigue,” IEEE Trans. Veh. Technol. 53 (4), 1052–1068 (Jul. 2004).CrossRefGoogle Scholar
18.Lal, S. K. L. and Craig, A., “A critical review of the psychophysiology of driver fatigue,” Biol. Psychol. 55 (3), 173–194 (2001).CrossRefGoogle ScholarPubMed
19.Lu, H., Zhang, W. and Yang, D., “Eye Detection Based on Rectangle Features and Pixel-Pattern-Based Texture Features,” Proceedings of the International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS '07), Xiamen, China (Nov. 28–Dec. 1, 2007) pp. 746–749.Google Scholar
20.May, J. F. and Baldwin, C. L., “Driver fatigue: The importance of identifying causal factors of fatigue when considering detection and countermeasure technologies,” Transp. Res. 12 (3), 218–224 (2009).Google Scholar
21.NCSDR/NHTSA Expert Panel on Driver Fatigue and Sleepiness, “Drowsy driving and automobile crashes,” Report No. DOT HS 808 707, National Center on Sleep Disorder Research, National Heart, Lung, and Blood Institute, and National Highway Traffic Safety Administration, Washington, D.C. (1998).Google Scholar
22.Olmeda, D., de la Escalera, A. and Armingol, J. M., “Far infrared pedestrian detection and tracking for night driving,” Robotica 29, pp. 495–505 (2010).CrossRefGoogle Scholar
23.Park, I., Ahn, J.-H. and Byun, H., “Efficient Measurement of Eye Blinking under Various Illumination Conditions for Drowsiness Detection Systems,” Proceedings of the 18th International Conference on Pattern Recognition (ICPR '06), Hong Kong (2006) vol. 1, pp. 383–386.Google Scholar
24.Pedan, M., Scurfield, R., Sleet, D., Mohan, D., Hyder, A. A., Jarawan, E. and Mathers, C. (eds.), World Report on Road Traffic Injury Prevention (WHO Press, Geneva, Switzerland, 2004).Google Scholar
25.Pickering, C. A., Burnham, K. J. and Richardson, M. J., “A Review of Automotive Human Machine Interface Technologies and Techniques to Reduce Driver Distraction,” Proceedings of the Second Institution of Engineering and Technology International Conference on System Safety, London, UK (Oct. 2007) pp. 223–228.Google Scholar
26.Qin, H., Gao, Y. and Gan, H., “Precise Eye Location in Driver Fatigue State surveillance System,” Proceedings of the IEEE International Conference on Vehicular Electronics and Safety (ICVES '07), Beijing, China (Dec. 2007) pp. 1–6.Google Scholar
27.Rongben, W., Lie, G., Bingliang, T. and Lisheng, J., “Monitoring Mouth Movement for Driver Fatigue or Distraction with One Camera,” Proceedings of the Seventh International IEEE Conference on Intelligent Transportation Systems, Washington, D.C., USA (Oct. 2004) pp. 314–319.Google Scholar
28.Shi, J. and Tomasi, C., “Good Features to Track,” Proceedings of the 9th IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, USA (Jun. 1994) pp. 593–600.Google Scholar
29.Sigari, M. H., “Driver Hypo-Vigilance Detection Based on Eyelid Behavior,” Proceedings of the Seventh International Conference on Advances in Pattern Recognition (ICAPR '09), Kolkata, West Bengal, India (Feb. 2009), pp. 426–429.Google Scholar
30.Singh, S. and Papanikolopoulos, N. P., “Monitoring Driver Fatigue Using Facial Analysis Techniques,” Proceedings of the IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems, Tokyo, Japan (1999) pp. 314–318.Google Scholar
31.Suzuki, M., Yamamoto, N., Yamamoto, O., Nakano, T. and Yamamoto, S., “Measurement of Driver's Consciousness by Image Processing—A Method for Presuming Driver's Drowsiness by Eye-Blinks Coping with Individual Differences,” Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC '06), Taipei, Taiwan (Oct. 2006) vol. 4, 2891–2896.Google Scholar
32.Tabrizi, P. R. and Zoroofi, R. A., “Open/Closed Eye Analysis for Drowsiness Detection,” Proceedings of the First Workshops on Image Processing Theory, Tools and Applications (IPTA '08), Sousse, Tunisia (Nov. 2008) pp. 1–7.Google Scholar
33.Viola, P. and Jones, M., “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proceedings of the 16th IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii (Jun. 2001) vol. I, pp. 511–518.Google Scholar
34.Wang, Q., Yang, W., Wang, H., Guo, Z. and Yang, J., “Eye Location in Face Images for Driver Fatigue Monitoring,” Proceedings of the Sixth International Conference on ITS Telecommunications, Chegdu, China (Jun. 2006) pp. 322–325.Google Scholar
35.Xu, C., Zheng, Y. and Wang, Z., “Efficient Eye States Detection in Real-Time for Drowsy Driving Monitoring System,” Proceedings of the International Conference on Information and Automation (ICIA '08), ZhangJiaJie, Hunan, China (Jun. 2008), pp. 170–174.Google Scholar
36.Yang, J. H., Mao, Z.-H., Tijerina, L., Pilutti, T., Coughlin, J. F. and Feron, E., “Detection of driver fatigue caused by sleep deprivation,” IEEE Trans. Syst. Man Cybern. 39 (4), 694–705 (Jul. 2009).CrossRefGoogle Scholar
37.Zhang, Z. and Zhang, J. S., “Driver Fatigue Detection Based Intelligent Vehicle Control,” Proceedings of the 18th International Conference on Pattern Recognition (ICPR '06), Hong Kong (2006) vol. 2, 1262–1265.Google Scholar