AI to monitor pilot drowsiness

Last November, an Australian pilot fell asleep while operating a passenger flight, overshooting its destination by 50 kilometers. A few months earlier, in the US, investigators found that an air disaster had been narrowly avoided in San Francisco the previous year. Once again, the danger had been brought about by a pilot’s lack of sleep.

Fortunately these incidents had no serious repercussions, but they are not isolated cases. Human fatigue is a serious issue affecting the safety of the traveling public in all modes of transportation. Nearly 20 percent of the major US Transportation Safety Board investigations completed between 2001 and 2012 identified fatigue as a probable cause, contributing factor, or a finding.

To address this issue, the HIPNOSIS consortium, led by CSEM, aims to improve the evaluation of pilot fatigue by providing innovative monitoring tools—namely, a specific vision-based system combined with a bio-physiological signal sensor.

Machine learning at the service of onboard safety

HIPNOSIS won the tender launched by the Clean Sky 2 Joint Undertaking, a European research program dedicated to aeronautics. Andrea Dunbar, head of Embedded Vision Systems at CSEM, explains its main features: “We will implement computer vision and machine learning algorithms in order to detect signs of drowsiness in pilots in real time.” These algorithms will be integrated into a specific camera developed by the French startup Innov+, which already commercializes similar solutions for the automotive industry.

“CSEM will also use its know-how in the measurement of physiological parameters to develop a wearable sensor that monitors pilots before and during a flight,” Dunbar adds. “The collected data will be fused with eye-gaze-related measures as well as head pose, observed by the vision system.” The French company SERMA Ingénierie will be in charge of integrating HIPNOSIS into a cockpit prototype for preliminary testing.

A tailor-made solution for the aviation industry

Honeywell Aerospace—Clean Sky II Core Partner and a leader in aerospace development and the manufacturer of avionics solutions—defines requirements for the technology developed by the HIPNOSIS project and integrates it into the overall pilot monitoring system. “HIPNOSIS will deliver key enablers for the introduction of pilot monitoring and for turning this technology into aviation reality,” enthuses Bohdan Blaha, project manager at Honeywell Aerospace. “HIPNOSIS provides building blocks critical to successfully demonstrating the potential of this technology and its benefit to the aviation industry.” Final results are expected in 2021.

The HIPNOSIS project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821466.