Fatigue is a major killer on the roads, with one-in-six fatal accidents involving a drowsy driver. The problem is particularly acute in the trucking industry, where drivers travel long distances for long periods.
An Australian company is dramatically reducing fatal vehicle accidents with its eye tracking technology that can detect, in real time, when a driver is drowsy or distracted.
Seeing Machines’ technology uses two cameras placed in the cabin of the truck, plane or train, which are pointed at the driver or pilot. The cameras measure the drivers’ head pose and orientation, their eyelid closures, pupil diameter and direction of their gaze. This information is analysed to determine how distracted the driver is – whether they are alert, drowsy or inattentive.
Guardian system in truck cab to detect fatigue and distraction.
If a driver is found to be drowsy or distracted, in trucking, for instance, the driver’s seat vibrates and an alarm sounds to wake them up, with the warnings increasing in stridency if they are ignored. Seeing Machines’ SafeGuard Centre in Tucson, Arizona is also alerted and if required, they contact the driver and tell them to pull over.
In the past year, trucks using Seeing Machines’ Guardian system have travelled 300 million kilometres without a single fatigue-related accident. In that time, the company has woken up drivers from micro-sleeps in moving vehicles 453,228 times.
Guardian can be found in trucks in Europe, North and South America, Asia and the Middle East and in over 4,000 off-road mining vehicles.
“The technological goal is to understand what is going on in the mind of a person,” says Timothy Edwards, Co-founder and Chief Technology Officer of Seeing Machines. “So, a machine can derive a high-level understanding of somebody’s intent, emotional state, level of fatigue or distraction.”
Most of Seeing Machines’ competitors assume this problem can be solved with a webcam and algorithms, but it’s also a problem of lighting and physics – how does the camera see through sunglasses or in poor lighting, for instance?
“Imagine the difference between the eyes of someone that is squinting, looking into the sun as they drive, and someone with their eyes closed – the difference is about one-millimetre of eyelid opening,” says Edwards. “We have to detect this difference reliably enough to tell the vehicle that the driver cannot see the road and to engage safety procedures or not. It's not something you want to get wrong.
“Improving road safety for drivers has enormous benefits for society,” Edwards adds, noting that road deaths are the major killer in the developed world. “People are basically being lulled into a sense of security when they’re driving their fantastic ‘lounge on wheels’,” he says. “It’s actually one of the most dangerous things you can do in your everyday life.”
One of the major applications of the technology will be in what are known as conditionally autonomous vehicles – cars which can drive themselves but can also be operated by the driver. For the next 10 or 15 years, so-called driverless cars still need a qualified driver to be behind the wheel and pay attention to the road, ready to take over the driving if need be.
Once the car becomes aware the ‘driver’ isn’t paying attention, it gently nudges the driver with seat vibrations or visual flashing and escalates the alerts if they’re being ignored.
Seeing Machines is in talks with vehicle manufacturers in Japan, Germany and the US about installing its system in their self-drive cars. Consumers can expect to start buying vehicles with its safety systems integrated from 2017.
Automotive eye tracking technology being used in a car.
Improving interactions between humans and machines
Edwards studied systems engineering at the Australian National University (ANU) in Canberra from 1995, and after graduation started working at CEA Technologies, an Australian company that is a world leader in radar technology, before joining Klein Bottlers where he helped develop software for realistic creation and animation of water/liquid surfaces.
After travelling overseas, he returned to Australia not knowing what he wanted to do, except to “find the most interesting group of people I could and spend time with them”.
This led him to join the Robotics Systems Laboratory at ANU.
Edwards and Seeing Machines’ three other co-founders Alex Zelinsky, Jochen Heinzmann and Sebastian Rougeaux started working on eye tracking technology and began talks with Volvo R&D in Sweden. It got them thinking that the first mass-market robots were likely to be in the most evolved piece of machinery that people already buy – cars.
Seeing Machines was founded in 2000 to commercialise the technology and they started selling eye tracking research equipment to car companies, before using the technology to develop their own safety equipment.
They then moved into the mining sector, applying their alertness detection systems to mining vehicles, which Edwards says was a chance to finally use their technology to save lives. It was a commercial breakthrough for Seeing Machines and it eventually exited the sector, licensing its intellectual property to Caterpillar, which is installing the technology in its vehicles.
Seeing Machines, which was a finalist in the 2016 Australian Export Awards, is now working with two aircraft OEMs and two major airlines on applying its technology to pilot monitoring and training. The company’s eye monitoring technology will be used to detect where trainee pilots are looking when using a flight simulator and compare this with the eye movements of an experienced pilot, to see if they are making ‘rookie errors’ and to give them feedback on the errors. It will also detect if the trainee pilots are becoming overwhelmed or confused.
Seeing Machines’ eye tracking technology is now so reliable a number of new applications have also opened up in education and health.
“It’s going to hopefully make society a little safer and smarter,” Edwards says.