Black and White Aren’t Opposites After All

Most people see the world in color, yet artists can conjure up whole worlds—both realistic and imaginary—by using black pigments on white paper. Our ability to understand these drawings suggests that we use variations in brightness to extract a lot of visual information from the world. As a perceptual neuroscientist, I appreciate these drawings not just aesthetically, but also as experiments that can reveal what aspects of the world we perceive well and the neural processes by which we perceive them.

In popular culture, black and white are thought of as simple opposites, but my colleague Jose-Manuel Alonso and I have been uncovering ways that we perceive black and white differently, and how our brains have evolved mechanisms that create these differences. I want to walk you through some of our experiments that have led to an interesting answer to a 400-year-old puzzle.

If you look at the left side of the two busts of Caesar paired at the top, you’ll see a three-dimensional white sculpture, because we interpret the two-dimensional variations of brightness in the image as shading caused by light reflected from an object. In the photo on the right, we perceive essentially the same three-dimensional shape, but lit from the opposite side and appearing to be made of a darker material. So not only is our judgment of the illumination different, but so is our judgment of the object’s material properties. This photo is just the contrast reversal (photo negative) of the photo with the white bust, so comparing the two illustrates how we interpret gradual variations of light versus variations of dark.

The difference in how we use black and white cues to infer the properties of an object is even more evident when the changes in brightness are more distinct. The bright streaks on the black bust above appear to be highlights, and from their sharpness and small scale relative to the object, we can estimate the glossiness of the material. If the contrast of the image is reversed (the right side of the image pair above), we see a white or even metallic bust, and the black streaks appear to be smudges or paint. In this situation, the two tones give entirely different sorts of clues about illumination and material properties.

These examples illustrate a basic asymmetry in the way we perceive white and black: We interpret difference in lights as information about illumination, while differences in darks reveal something about materials. These interpretations fit well with the physics of the world: Illuminants light up objects and are reflected less by highly absorbent materials and holes.

Other basic asymmetries have been noted for centuries. Galileo Galilei, who was as perceptive as he was creative, observed that Venus appeared larger through his telescope as a light object against the dark night sky than it appeared as a dark object against the bright day sky. Ernst Mach, for whom the speed of sound is named, demonstrated that letters are difficult to recognize if some strokes are white and others black, suggesting that the two shades may be processed separately by the brain.

We began our experiments by testing whether subjects could pick out blacks and whites on unbiased backgrounds with equal ease. We created randomly arranged background panels with an equal number of equal-sized black and white pixels, and then asked observers to count the number of larger targets presented on the background as quickly as possible. To make the targets clearly different from the background pixels, we made them nine times larger in area. The targets were either all black or all white. To our surprise, we found that people counted black targets significantly faster and with many fewer errors. You can see that black targets are easier to identify than white ones in these 12 examples:

How could we explain this discrepancy? Galileo had attributed the illusion of Venus’s size to light scatter in the eye, which would enlarge the image of a light area compared to a dark area. Herman von Helmholtz, the polymath physicist and physician, showed that there was too little light scatter in the eye for Galileo’s explanation to be complete, but the actual cause remained unclear. Helmholtz took Galileo’s observation and made a simpler, abstract version of it with equal-sized white and black squares on the opposite backgrounds. The white square appeared larger, so he called it the “irradiation illusion.” We noticed that something akin to the irradiation illusion was occurring in our backgrounds, too: Even though there were equal areas of black and white, there appeared to be more white area.

We measured the magnitude of Helmholtz’s irradiation illusion by asking people to increase the area of the black square until it appeared to be the same size as the white. Then we calculated the ratio of the physical sizes of the squares. Similarly, we measured the magnitude of the area illusion in the background by having people increase the ratio of black to white pixels until the areas appeared equal. Since the ratios required by the two corrections were approximately equal, we reasoned that both illusions must share an underlying cause. We became more convinced that our ability to see black versus white targets was influenced by the irradiation illusion when we found that if we used backgrounds with what people saw as the “balanced” ratio of black to white, targets of the two shades were equally easy for viewers to see and count.

Our next step was to search for a brain mechanism that could explain the irradiation illusion. When Keffer Hartline recorded the first electric signals from single retinal nerves responding to light stimulation in the 1970s, he found two types of nerves whose responses are shown on the left side of the figure below. One kind of neuron generated spiking electrical signals when exposed to light (bottom two rows). The other kind generated spikes in the dark, but turned off when exposed to light (top row). Since then, such cells have been found in eyes of many species as disparate as insects and mammals, separated by more than 500 million years of distinct evolutionary pressures, suggesting that this neural strategy fits something fundamental about the world, across many environmental niches.

These neurons are called ON and OFF cells, because of their behavior in response to light. Until recently, they had been generally considered to be opposite but equal. However, we found that when we measured the responses of ON cells to increasing equal increments of light on a black background, the response increased rapidly but then plateaued (red curve). On the other hand, OFF cell responses increased roughly in a straight line as light was decreased on a white background (blue curve). The more rapid initial increase in ON outputs explains why we are more sensitive to small increases of light in dark settings than to small decreases of light in bright settings. The plateau at the top of the ON response curve explains why we are more limited at distinguishing progressively lighter shades than we are darker shades. The neural explanation for both the irradiation illusion and Galileo’s observation arises from the same difference in the response curves.

Remarkably, these simple but different responses of the ON and OFF cells also explain some black-white asymmetries that seem almost paradoxical. For example, it is easier to make out black text on a white background than white text on a black background, despite the fact that we are capable of seeing much tinier white dots on black backgrounds than black dots on white backgrounds. The reason for this is that the ON response expands strokes of white letters slightly so that they become more difficult to distinguish. The same effect “expands” a white background, making small black dots seem smaller.

Whites and blacks in images of the world thus arise from different physical causes, provide information about different aspects of the world, and are processed differently by the brain. The differences in how we see shades originate in the beginnings of sensory neural processing. We have yet to figure out the neural mechanisms that allow us to make inferences about illumination and materials from different scales of lights and darks, so we may have much to learn from the strategies that artists use to depict them. If you look at black-and-white drawings not as impoverished versions of the colored world, but as pared-down illustrations of the cues we use to understand what we are looking at, you can enjoy them as intellectual puzzles, and it may change the way you look at art.