The racing auditory display (RAD) is an audio-based user interface that allows players who are blind to play the same types of racing games that sighted players can play with the same efficiency and sense of control that sighted players have. The RAD comprises two novel sonification techniques: the sound slider for understanding a car's speed and trajectory on a racetrack and the turn indicator system for alerting players of the direction, sharpness, length, and timing of upcoming turns. When the gamer above — who has been blind his entire life — used the RAD, he could race as well as casual sighted players, with similar lap times and driving paths.
To understand how the RAD's sound slider works, imagine being behind the car that you are controlling, so you can hear the sound of the car's engine right in front of your face. The car's sound will move left or right as it becomes more at risk of hitting the track's left or right edges, respectively. When you steer, you control the car's sound directly, so if you hear the car's sound move far toward the left, you will want to steer right to bring the sound back toward the center.
The RAD's sound slider communicates the car's relative risk of hitting the track's left or right sides if the player wanted to. Though the car's lateral position is the same between (a) and (b) and between (c) and (d), for example, the corresponding sound slider values are very different. This is because the left and right trajectories' lengths — and therefore the relative risks of hitting the left and right sides of the track — are very different in each pair of cases. The result is that the player receives many pieces of information — the car's lateral position on the track, its heading with respect to the track's, its speed, the track's width, whether the track is about to immediately turn, and more — as a single measure that is no less relevant to the process of racing than all of that information put together.
The RAD's turn indicator system works by playing a series of four beeps that trigger when the player's car crosses four corresponding and equally spaced distance markers placed ahead of the turn. The beeps' rhythm helps players anticipate when turn will begin, allowing them to cut corners by steering just before the last beep. The RAD also announces each upcoming turn's number (where "1" means the track's first turn) to help players learn the track over time.
The track that we used for our user studies (found here) is much more complex than ones from previous blind-accessible games. It features a wide variety of turns: soft, moderate, and sharp turns; a long straightaway; a series of hairpin turns (Turns 9–11) that require players to slow down; a 270° turn (Turn 16); several short kinks in the track (Turns 8, 13, 14, & 17); several series of esses (Turns 1–5 & 19–20); long and gradual turns (Turns 5b & 7a); and turns that vary in sharpness as they progress (Turns 5, 7 & 17).
Driver assistance systems work by telling drivers which way to steer at any given time, but the RAD is different. It was designed to not just tell players what to do but give them enough information to form a plan of action themselves. As a result, the RAD makes it possible for players who are blind to cut corners and race comparably to casual sighted players. Here are two sample driving paths from a player who is blind: one using the RAD and one using a state-of-the-art driver assistance system. The player oscillates constantly with the driver assistance system but drives more smoothly when using the RAD. He is also able to cut the corners in the track's final ess turn sequence (shown in green at the bottom-left) using the RAD. Our supplemental video shows the player's third lap with the RAD's audio included.
"The system, called racing auditory display or RAD, is truly amazing. It lets the visually impaired play racing games without 'seeing' the screen. Instead, the audio output tells the player when they're getting closer to an edge and can even enable to them to cut corners in tight turns."
"It doesn't simply tell players to turn, though; it gives them the information and then the plan of action is up to them."