AUTONEWS

Worn traffic signs mislead AI vision systems, exposing autonomous driving risks

Recent research from South Korea has uncovered a critical vulnerability in artificial intelligence vision systems deployed for autonomous driving, demonstrating how naturally deteriorating traffic signs can consistently mislead machine learning models. The study, published in IEEE Transactions on Dependable and Secure Computing, was led by Associate Professor Seong Tae Kim of Kyung Hee University and Assistant Professor Hong Joo Lee of the Seoul National University of Science and Technology.

To investigate this issue, the researchers developed Adversarial Wear and Tear, or AdvWT, a generative framework that simulates realistic environmental degradation on traffic signage. Utilizing a StarGAN-v2-based image-to-image translation model, the system learns a latent damage representation capable of reproducing diverse physical deterioration while preserving the semantic content of the signs. By progressively adjusting this representation, the framework generates visually authentic but adversarially optimized signage designed to trigger classification errors in deep neural networks.

The framework was rigorously tested across two traffic sign datasets and eight distinct recognition architectures, including lightweight convolutional neural networks like ResNet-18 and MobileNet, as well as transformer-based models. AdvWT achieved near-perfect attack success rates, demonstrating exceptional transferability across different model types. Human evaluation involving thirty-two participants confirmed that the synthetic damage closely matched the perceived realism of physically worn signs. Further validation through physical experiments, which involved printing and photographing adversarial signs under varied distances, angles, and lighting conditions, confirmed that the AI misclassification persisted across real-world capture scenarios.

Beyond identifying vulnerabilities, the research offers a practical pathway for enhancing AI reliability. The bidirectional architecture of the model can simultaneously reverse simulated damage, suggesting applications in digital signage restoration. More critically, integrating AdvWT-generated deteriorated imagery into training datasets significantly improved model generalization and resilience to naturally damaged signs. The authors emphasize that ensuring AI safety in high-stakes environments requires moving beyond average performance metrics to systematically expose and patch failure modes induced by real-world environmental factors. This approach provides a scalable method for stress-testing computer vision systems ahead of deployment in autonomous navigation, industrial automation, and other safety-critical domains.

“We focused on traffic signs because they are exposed to weather and environmental damage,” Lee said, noting that unlike a temporary sticker-based attack, natural deterioration persists until the physical sign is repaired or replaced.

To build AdvWT, the team trained a generative image-to-image model, based on StarGAN-v2, to learn the visual patterns of damaged and undamaged signs. By adjusting the model’s internal representation of “damage style,” the researchers could generate signs that looked naturally worn but were more likely to be misread by AI systems. In a survey of 32 people, the generated images were rated as similarly natural-looking to real damaged signs. Tested against two traffic-sign datasets and eight recognition architectures, AdvWT achieved near-perfect success in fooling lightweight models such as ResNet-18 and MobileNet, remained effective against transformer-based models, and showed the strongest ability among the methods tested to transfer its effect across different AI architectures. When the researchers printed and photographed the altered signs under varying distances, angles and lighting, the effect held up outside the digital environment as well.

The work builds on earlier research showing computer vision can be vulnerable to physical changes people would barely notice. A 2018 study found that stickers placed on a real stop sign caused a deep-learning system to misclassify it in all captured lab images and in 84.8 percent of frames during a moving-vehicle field test. A separate USENIX study found a detector failed to recognize altered stop signs in more than 85 percent of lab frames, and in 63.5 to 72.5 percent of frames in outdoor tests using posters and stickers.

Researchers caution that a vulnerability found in one vision model does not mean a real autonomous vehicle would make the same error, since commercial systems typically combine multiple cameras and sensors, maps and other redundancies. Waymo, for instance, says its driverless system has logged more than 220 million fully autonomous miles through March 2026, with far fewer serious-injury and airbag-deployment crashes than human drivers over the same routes, though those figures are company-reported and don’t rule out individual perception weaknesses. The U.S. National Transportation Safety Board has separately warned that developmental automated-driving systems can have limitations detecting hazards and predicting how other road users will move.

The researchers say AdvWT also has a defensive use: the same model can restore naturally damaged signs, and training recognition systems on AdvWT-generated damage improved their ability to handle real-world wear. Kim said building reliable AI requires continuously finding where systems fail and using that insight to make them more robust, adding that such work could matter for other high-stakes fields such as healthcare and finance over the next five to ten years. The findings add to a broader case that autonomous-vehicle developers should test perception systems against imperfect, real-world conditions rather than relying only on clean benchmark images, since road signs routinely fade, crack, corrode or become obscured well before anyone deliberately tampers with them.

Worn, faded, or vandalized traffic signs pose a critical risk to autonomous vehicles by misleading the AI vision systems that rely on them for navigation. Because computer vision models are heavily trained on pristine dataset images, real-world degradation can cause severe misclassification or total detection failures.

Texture distortion: Rust, peeling paint, and graffiti alter the sign's surface. AI models can misinterpret these random patterns as entirely different signs (similar to an adversarial attack).

Color fading: Faded red stop signs or yellow warning signs lose their high-contrast distinctiveness, making it hard for neural networks to segment them from the background.

Shape obscuration: Dented edges or partial obstructions (like dirt or stickers) break the geometric boundaries that algorithms use for quick geometric validation.

Inappropriate speed: A faded speed limit sign might be missed entirely, causing the vehicle to travel too fast or too slow for the zone.

ntersection failures: Misinterpreting a worn "Stop" or "Yield" sign can result in dangerous, unprompted entries into oncoming traffic.

Erratic braking: Ghost classifications—where a damaged sign is misread as a hazard—can cause the vehicle to slam on its brakes unexpectedly.

Seoul National University of Science and Technology.