Why the importance of driver behaviour and perception should not be underestimated when developing ADAS and AVs, and how a driver-in-the-loop simulator can help, by Jelle van Doornik, ADAS simulator development engineer at Cruden
The automotive world is currently consumed by autonomous vehicle (AV) development. And with good reason. In the UK for example, the government has announced that driverless cars will be tested on public roads by 2021. That’s 30 or so months to have every piece of the complex picture – vehicles, drivers, infrastructure and rules – not only worked out, but fully and reliably tested.
We see a lot of ADAS (advanced driver assistance systems) and AV technology development at OEMs and Tier 1 suppliers, where it is still heavily focussed on sensor technology, computer power and algorithms for different situations. We also see psychologist-led behavioural research studies involving various levels of autonomous driving scenarios at universities and research institutes. But we see little convergence of the two worlds, and this is worrying.
There is a fine line between drivers appreciating or rejecting an ADAS or a level 3 autonomous driving hand-over moment. For safety reasons, it is essential, but very difficult (as no two people are the same) to get this right. If an ADAS decides to take control of the car or hand the control back to the driver, it has to do so in a way the driver understands and accepts. Extensive testing with many different types of people is required, yet this is expensive and time consuming. It is also difficult and dangerous to test if this works with real people in real cars with real traffic.
The solution to all three issues is empirical research using an advanced driving simulator. Simulators, set up with real vehicle hardware such as seats, steering wheels, dashboards and pedals, allow not only automotive engineers but also behavioural scientists to define, develop and test ADAS more safely, quickly and in a more controllable and repeatable way, using actual humans in close to real life situations. How else do we expect to gather the necessary volume of data to be safe on our roads within three years?
Lessons from ADAS
The experience from the past five to ten years of ADAS testing, for driver assistance functions that are now commonly available such as adaptive cruise control, lane departure warning, distance alert and traffic sign recognition, can tell us a lot about what could be improved when it comes to the next step of autonomous vehicle development.
A recent Connecting Mobility study of 1,355 Netherlands business drivers reporting the presence of at least one ADAS on their cars, reveals the disconnect between years of technology development by automotive engineers and end-customer awareness. The drivers studied had little idea of what systems were installed in their cars, what they were for and how they should use them. We believe this reflects a wider issue of manufacturers focussing predominantly on the technology behind the ADAS system and not enough on the interaction between the driver and the ADAS.
In the study, roughly half of the drivers who were aware of and interested in the ADAS in their car, worked it out through trial and error, while driving. This kind of thing is played out in the many videos online of people experimenting with the ADAS/AD features of their car and pointing out when it fails or acts weirdly in their opinion. This is cause for concern. Do we really want distracted drivers pressing lots of buttons to activate or deactivate the ADAS while in the outside lane of the motorway? Are manufacturers sure that their ADAS are intuitive to use by all drivers in all situations? What trials of regular drivers took place? How are we going to take a different approach to educating and training drivers for autonomous vehicles?
Had driving simulators been more widely used in ADAS development to date, perhaps the controls in our cars would look, sound, feel and be located very different today. ADAS requires different user interfaces. The first lever on the steering column to operate the indicator lights was a great invention. Nowadays, you will find multiple levers on both sides of the steering column which can move in every possible direction and have buttons on all surfaces. It is time to rethink this and the best way to develop new concepts in a driving simulator.
The driving simulator
Being a complex machine with a complex vehicle model, traditionally, working with a simulator has been inaccessible to those in non-engineering HMI roles, an area that straddles psychology and engineering. This growing need has so far not been met by a suitable automotive testing solution.
Driving simulators are already used by many OEMs, Tier 1s, research institutes and universities of course. However, functional testing of ADAS systems using tools such as VIRES-VTD and TASS-PreScan alone, which focus on sensor simulation and engineering objectives, is insufficient for determining how a person would experience the ADAS feature in real life. Until now, there has been no dedicated simulator software for the final step in ADAS controller development that recognises the human element as its own separate test phase.
To support ADAS and AV development, Cruden has designed a purpose-built simulator, named AS1, that is equipped with the company’s in-house developed Panthera ADAS Toolbox software (see below). The open architecture software can also be bought separately, for use with any driver-in-the-loop simulator.
It is the first commercially available driving simulator software package of its kind, providing a new tool for the evaluation of human interactions with ADAS, suitable for use by anyone conducting ADAS experiments. The Panthera ADAS Toolbox includes a series of internal vehicle models, which means that users can focus on ADAS controller development and human behaviour, independent of the models provided by vehicle dynamics engineers.
Through sophisticated, realistic driving simulators, Cruden can offer a driver-in-the-loop simulator to test advanced driver-assistance systems (ADAS) and autonomous vehicle systems that will not only reduce the risk of human injury but reduce costs and development time. We invite car manufacturers, regulators and psychologists to get in touch to discuss how driving simulation can help the development of autonomous vehicles. We look forward to helping to shape this exciting new era of mobility.
Panthera ADAS Toolbox
With Panthera ADAS Toolbox, Cruden has created the perfect experimental environment in Matlab Simulink, meaning that engineers and psychologists can create any ADAS controller they want and introduce it to the simulator. The Matlab environment is placed between the Panthera software and the simulator, allowing engineers to manipulate the input and output/feedback though the driver, exposing the interaction between the human and the car.
In this environment, the situational awareness of the vehicle is presented to the ADAS controller as ideal sensors. This is all the input engineers need to develop and test the so-called decision-making part of the ADAS controller, which is the only aspect of the ADAS controller the driver actually experiences.
In real life, this objectification would be the result of the different sensors on the car and sensor fusion. They include information like “there is a car 15 m ahead with a speed of 80 kph, decelerating with 1.2 m/s2.” In a simulator, where the complete environment is determined by the simulation software, these ideal sensor outputs can be generated without having to model the actual sensors and sensor fusion.
The Panthera ADAS toolbox enables engineers to concentrate on the aspects of the ADAS controller that affect the driver in the loop of the simulation. There is a lot of work to be done on sensors and sensor fusion to make cars fully autonomous, but the real challenge is to create the best possible experience for the human inside.
As part of a complete driving simulator package, such as that offered by Cruden (which includes high quality content, 3D rendering, motion and actual vehicle hardware), Panthera ADAS Toolbox offers drivers the best possible level of immersion that allows subconscious interaction with the ADAS system. Panthera is compatible with many advanced external ADAS development packages like Vires-VTD and TASS-PreScan and supports Lidar-sourced scenario databases with OpenDRIVE and OpenCRG.