Research
New research shows that even subtle changes in digital images designed to confuse computer vision systems can also affect human perception
Computers and humans see the world in different ways. Our biological systems and machine-made ones may not always pay attention to the same visual signals. Neural networks trained to classify images can be completely misled by subtle perturbations in an image that a human wouldn’t even notice.
That AI systems can be fooled by such adversarial images may suggest a fundamental difference between human and machine perception, but it prompted us to investigate whether humans too might—under controlled testing conditions—reveal susceptibility to the same perturbations. In a series of experiments published in Nature Communications, we found evidence that human judgments are indeed systematically affected by adversarial perturbations.
Our discovery highlights a similarity between human and machine vision, but also demonstrates the need for further research to understand the influence that adversarial images have on humans as well as AI systems.
What is an adversarial image?
An adversarial image is one that has been slightly modified by a process that causes an artificial intelligence model to misclassify the contents of the image. This deliberate deception is known as a counter attack. Attacks can be aimed at making an AI model classify a vase as a cat, for example, or they can be designed to make the model see anything but a vase.
Left: An artificial neural network (ANN) correctly classifies the image as a vase, but when perturbed by a seemingly random pattern across the image (middle), with intensity magnified for illustrative purposes – the resulting image (right) is incorrect and confidently misclassified as a cat.
And such attacks can be subtle. In a digital image, each individual pixel in an RGB image is on a scale of 0-255 representing the intensity of individual pixels. An opponent’s attack can be effective even if no pixel is formed by more than 2 levels at this scale.
Reverse attacks on physical objects in the real world can also succeed, such as causing a stop sign to be misidentified as a speed limit sign. Indeed, security concerns have led researchers to explore ways to resist adversary attacks and mitigate their risks.
How is human perception affected by counterexamples?
Previous research has shown that people can be sensitive to large image perturbations that provide clear shape cues. However, less is understood about the effect of more varied adversary attacks. Do people dismiss noise in an image as harmless, random image noise, or can it affect human perception?
To find out, we conducted controlled behavioral experiments. First, we took a series of original images and performed two adversary attacks on each, to generate many pairs of perturbed images. In the animated example below, the original image is classified as “jar” by a model. The two images perturbed by adversary attacks on the original image are then misclassified by the model, with high confidence, as the adversary targets “cat” and “truck”, respectively.
We then showed human participants the pair of images and asked a targeted question: “Which image looks more like a cat?” Although neither picture looked like a cat, they were forced to make a choice and usually reported feeling like they were making an arbitrary choice. If brain activations are insensitive to subtle opponent attacks, we would expect people to choose each image 50% of the time on average. However, we found that the selection rate—which we refer to as perceptual bias—was reliably above chance for a wide variety of perturbed image pairs, even when no pixel was adjusted to more than 2 levels on this 0–255 scale.
From a participant’s perspective, it is like being asked to discriminate between two nearly identical images. However, the scientific literature is replete with evidence that people use weak perceptual signals to make choices, signals that are too weak to express confidence or awareness ). In our example, we might see a vase of flowers, but some activity in the brain tells us that there is a cat hint about it.
Left: Examples of pairs of opposing images. The top pair of images is subtly perturbed, at a maximum size of 2 pixel levels, so that a neural network misclassifies them as “truck” and “cat”, respectively. A human volunteer is asked “Who is more like a cat?” The bottom pair of images is most obviously manipulated, at a maximum size of 16 pixel levels, to be misclassified as “chair” and “sheep”. The question this time is “Who is more sheepish?”
We performed a series of experiments that ruled out possible artificial explanations for the phenomenon for our Nature Communications paper. In each experiment, participants reliably chose the opposite picture corresponding to the target question more than half of the time. While human vision is not as sensitive to opposite perturbations as machine vision (machines no longer recognize the original image category, but humans still see it clearly), our work shows that these perturbations can nevertheless prompt people towards decisions made by machines.
The importance of AI safety and security research
Our primary finding that human perception can be influenced—albeit subtly—by contrasting images raises critical questions for AI safety and security research, but using formal experiments to explore similarities and differences in the behavior of of visual AI systems and human perception, we can leverage the insights to create safer AI systems.
For example, our findings can inform future research that seeks to improve the robustness of computer vision models by better aligning them with human visual representations. Measuring human sensitivity to contrast perturbations could help judge this alignment for a variety of computer vision architectures.
Our work also demonstrates the need for further research to understand the wider impacts of technologies not only on machines, but also on humans. This in turn highlights the continued importance of cognitive science and neuroscience in better understanding AI systems and their potential impacts as we focus on creating safer and more secure systems.