AUTONEWS

Audio tones can help automated vehicles communicate with pedestrians

Pedestrian volume data offer valuable insights into urban activity patterns, which support planning efforts such as evaluating sidewalk improvements, assessing land use changes, and identifying areas needing investments in safety and walkability. These data also support optimizing street connectivity and accessibility.

The widespread adoption of smartphones has brought new opportunities for automated human mobility sensing, particularly through mobile GPS data. However, growing privacy concerns, particularly under frameworks like the General Data Protection Regulation (GDPR) in the European Union, have placed restrictions on using mobile location data to track individuals. In parallel, smart city initiatives have adopted the deployment of IoT-based sensors to monitor activity in urban environments. These efforts have largely focused on vision-based systems, such as computer vision and infrared cameras, although other sensing technologies have also been tested.

Urban sound offers a promising alternative. Microphones are affordable, energy-efficient, and effective in visually occluded environments. They can complement or replace cameras in contexts where installation is impractical, such as shaded areas, narrow corridors, or locations or scenarios where the costs of cameras are prohibitive. The general feasibility of using microphone recordings for the detection of pedestrians has been shown recently for a vehicle-free courtyard on a university campus.

This study addresses two key gaps in existing work. First, the generalizability of audio-based models remains unclear. Given the variability in urban soundscapes, shaped by traffic, land use, and average pedestrian activity levels, it is necessary to evaluate model performance across data collected from different settings, particularly in the presence of typical urban noise. Second, existing studies lack information on interpretability; it is unclear which sound characteristics existing models rely on for detecting pedestrians.

For a pedestrian, the decision to cross a busy road often involves a distinct moment of human connection — an exchange of subtle cues with drivers, such as eye contact or a hand wave, to judge if it is safe to proceed.

But how can vulnerable road users get this information when the driver's seat is empty?

A novel study from Virginia Tech and Zoox, the Amazon-owned autonomous ride-hailing company, provides a promising answer: through sound. The research was published at the 28th Enhanced Safety of Vehicles Conference, which was sponsored by the National Highway Traffic Safety Administration and Transport Canada.

Using custom tones, the research team explored how sound could be used to communicate vehicle intent in participant trials. Results show that sounds can be as effective as a traditional horn in increasing pedestrian awareness and influencing crossing decisions, particularly in jaywalking situations, even without prior training on the tones.

“This is a one-of-a-kind study,” said Charlie Klauer, research scientist at Virginia Tech Transportation Institute (VTTI). “We really need to understand how pedestrians are making those decisions and what information they need to make safe decisions.”

To emulate real-world road conditions, the VTTI team developed a realistic traffic environment on the Virginia Smart Roads closed-course test track and recruited 40 participants to act as pedestrians in specific scenarios — either traversing a marked crosswalk or jaywalking in front of an automated test vehicle.

In tandem, the Zoox team designed seven unique sounds to communicate either an urgent “stop, vehicle is here and moving” or “vehicle is here and waiting.” They worked with Rafael Patrick, assistant professor in the Grado Department of Industrial and Systems Engineering, and Tanner Upthegrove, media engineer at the Virginia Tech Institute for Creativity, Arts, and Technology, to ensure the audio environment was well-controlled to allow for accurate audio data capture.

“Sound is one of our most innate senses,” said Jeremy Yang, sound design lead at Zoox. “We wanted to examine the effectiveness of non-traditional automotive sounds in eliciting safer responses from pedestrians.”

Zoox team member Daniel Edillor demonstrates a crosswalk test scenario designed to measure participant response times to audio tones played by the approaching test vehicle to signal it will yield. Photo by Erem Memisyazici for Virginia Tech.

To study these purpose-built audio tones, pedestrians participated in 200 jaywalking scenarios and 160 crosswalk scenarios. Alert tones were played to discourage crossing, while waiting tones were played to indicate it was safe to cross. Participants were also surveyed to rank the sounds by qualities like “urgency,” “friendliness,” and “aggression.”

All alert tones were effective in discouraging potential jaywalkers, with results suggesting that these tones are as effective as a horn sound but perceived in a more favorable light. While the waiting tones were successful in encouraging the safe passage across a crosswalk a little less than 50% of the time, the time to cross was not significantly different from the baseline.

Pedestrians crossing...Klauer said ensuring the safety benefits of automated vehicles extends beyond riders and drivers to pedestrians and is a crucial aspect of their development. 

“All road users, pedestrians, bicyclists, mopeds, scooters, everyone counts and we're attempting to make sure that those interactions are safe,” Klauer said.

Yang said that because the Zoox robotaxi was built from the ground-up, the company has the opportunity to incorporate different features, like sound, to explore how it can be used to communicate and enhance safety.

In the study, each pedestrian crossing session involved interacting with a Zoox “test fleet” vehicle, outfitted with the standard sensors used for automated driving, as well as an external speaker system to produce the Zoox-designed sounds. The participant group was evenly distributed between those who reported normal or corrected-to-normal vision and those who reported non-correctable vision or an acuity of 20/200 or worse.

“One of the primary goals here was to make sure that we could not only communicate with people who have a lot more visual information and can see motion, but also those who can't,” Klauer said.

While the test fleet vehicles are capable of autonomous driving, they are typically operated by human drivers. To enhance the realism of interacting with an autonomous vehicle, the driver was camouflaged in a “seat suit” – a VTTI invention that provides a unique way to assess human reactions to automated vehicles while maintaining test consistency.

“From our pedestrian's perspective, all they saw was an empty seat,” Klauer said. “So, when they were making their decisions about whether they would cross or not, they fully believed it was an automated vehicle.”

Upon interacting with the vehicle, pedestrians were given methods to communicate a change in their intent to cross. Researchers measured the difference between time-to-cross or time-to-not-cross, calculating the amount of time between the sound and the decision. This data was compared with a pre-established measure of typical pedestrian reaction time to a standard car horn as a baseline. 

Developing intuitive sound...One unique challenge for this study included developing tones that are both nuanced and intuitive.

 “[The goal was] to evaluate different messages. Not just an alert, but also the equivalent of a driver waving a pedestrian along,” Klauer said.

Yang said another challenge was to identify sounds that were as effective as a horn in eliciting a reaction, but “less obnoxious and aggressive.”

The study’s most promising results occurred from the jaywalking scenarios, which the researchers said are known trouble areas for pedestrian safety. Results suggest the decision to not cross was faster for all sound conditions compared with the baseline.

“We know that a lot of fatalities happen mid-block, not necessarily at intersections, but at mid-block,” Klauer said. “In the jaywalking scenario, they responded very similarly to a horn.”

The crosswalk scenarios, which used audio tones to communicate that the vehicle would wait for the pedestrian, showed more mixed results.

“Repeated sound exposure may be needed for learning what a sound is communicating or that additional external communication may not be as necessary above that of the overall motion of the vehicle,” Yang said.

Yang, whose passion for sound stems from learning piano at a young age, said he was excited about how automated vehicles can spark the development of new audio languages and techniques that can aid in robot-human interactions.

“To use sound in a meaningful way, to improve safety, awareness, and make the world a better sounding place adds a real significance to this work,” he said.

The future of sound...While there is still more work to be done before these novel sounds will be utilized by autonomous vehicles on the road for the purposes of communicating intent to pedestrians, Klauer and Yang agreed that the results will serve as a launch pad for further development of the technology.

“This is just the beginning,” Yang said, “but it is really promising to see the impact sound can have in lieu of a driver and how it can improve our ability to communicate with external road users.”

Virginia Tech university-EUA