How PREDICTION ALGORITHMS CAN SYNCHRONIZE AND MINIMIZE LATENCY ON A DRIVING SIMULATOR
Cruden has made a breakthrough in synchronizing and minimizing driving simulator latency - the amount of time between driver input and simulator feedback - by implementing prediction algorithms. The result, zero simulator-induced latency, will allow car manufacturers and motorsport teams to use validated car models directly on the simulator without having to make modifications to compensate for simulator hardware and software delays.
This development was the subject of a joint presentation by Cruden, car manufacturer Audi and the University of Stuttgart at this year’s Driving Simulation Conference (DSC), held at the renowned engineering and research graduate school Arts et Métiers Paris Tech from September 7-9, 2016. Cruden also shared a research paper which explains in detail the theory and methodology of achieving zero simulator-induced latency.
Latency has always been a challenge in the design of driving simulators when trying to replicate exactly the handling characteristics and driving experience of a real car. Because simulators are subject to the laws of physics, there will always be some latency in the system, caused primarily by sampling delays, processing time, and data transfer. This latency means that additional time elapses between the model output and the resulting feedback from the simulator when compared to the real vehicle. Hence, lowest possible latency and fast response are needed to provide the most accurate and realistic driving experience on a simulator.
Test drivers assess the handling quality of a vehicle by the time delays between vehicle states, and these states are perceived by different senses, presented to the driver by the simulator’s various feedback channels. When driving through a curve, for example, the yaw rate is mainly perceived visually, whereas the car’s lateral acceleration is perceived in the inner ear and as force acting on the body. From this combined information, and from sensing the progression in time of yaw rate and lateral acceleration, the driver can actively control the vehicle. The goal of the simulator must be to present the driver with the same information content at the same time as would the real car.
Low latency is especially important on validated vehicle models which must precisely represent real car behaviour. Car manufacturers often have vehicle models that were validated for offline simulation, to replicate the exact behaviour, including exact latency, of the real vehicle. They want to be able to use simulators in vehicle development work without having to spend time, effort and money on a re-validation for DiL-simulator work.
Cruden has first analysed and optimized the latency of each individual feedback channel of their simulators. After optimization, the simulator features the (very low) minimum achievable latencies of 12ms for audio, 11ms for steering, 29ms for visuals and 10-18ms for motion. Cruden then set two goals: synchronizing the latency across all channels and reducing to zero the net latency added by the simulator to the pre-validated vehicle model latency.
These two key considerations, synchronization and minimization, lead Cruden to achieve zero simulator-induced latency with a two-step approach. In the first step, the visuals are predicted in order to synchronize them with the simulator’s motion platform and other feedback channels. In the second step, additional prediction is applied to all system channels.
If the visual and physical information provided to the driver by the simulator is not synchronized, it is more difficult for the driver to control the vehicle or to realistically assess its handling characteristics. Synchronizing visuals with motion is also crucial for platform tracking, so that off-platform visuals are accurately corrected for platform movement, to ensure realism and prevent simulator sickness. Predicting the visuals can be used to synchronize the visual latency with the latency in the motion feedback channel.
The visual system typically shows the biggest delay of all feedback channels in a simulator. In recent years the latency here has been reduced to just below 30 ms, but Cruden’s goal was to implement prediction algorithms to entirely eliminate simulator induced latency. This required taking account of all software and hardware delays that are still present, after synchronization, outside the validated vehicle model.
The prediction methodology used for the visuals, however, cannot be used for the motion channels. The signals for the motion channel are based on vehicle accelerations, but these cannot be predicted because live data for the derivative of acceleration (jerk) is not accessible. This means that instead of implementing prediction on the output of the vehicle model, the prediction must be performed on the input, using the available information about steering wheel position and velocity. Video measurements showed it is possible to predict the car body camera position as well as the future steering wheel position. This prediction can compensate for all delays that are not a result of the dynamics of the vehicle model, to effectively reach a state of zero simulator-induced latency.
This methodology was tested on two 6-DOF simulators: an automotive simulator with an off-board projection screen used for chassis development at Audi AG and a motorsport simulator with three screens mounted on the top platform and a Formula 3 car model. Tests proved conclusively that car manufacturers and motorsport teams will now be able to use validated car models directly on the simulator without needing to make modifications to compensate for simulator hardware and software delays. The era of driving simulators with zero simulator-induced latency has arrived.
If you would like to read the technical paper: "Implementing prediction algorithms to synchronize and minimize latency on a driving simulator" by van Doornik, Jelle, Brems, Willibald, de Vries, Edwin and Wiedemann, Jochen, please email firstname.lastname@example.org.