Chromatic Chutes And Ladders

Chromatic Chutes and Ladders: How the McCollough effect may involve feedback loops in the visual system.

“Vision is the art of seeing things invisible.” -- Jonathan Swift

Bryan Kennedy

Psychology 126 November 2002 TA: Christopher Cantor

2 Chromatic Chutes and Ladders: How the McCollough effect may involve feedback loops in the visual system It seems as though our eyes are always playing tricks on us: at times we see a larger moon on the horizon than high in the sky or an artistic staircase that appears to rise infinitely. It is unlikely that our visual systems, having evolved over millions of years, would engage in such petty games for the delight of misleading us. Instead, these “mistaken” perceptions are likely the result of systems that, at other times, are employed to good effect. The McCollough effect (ME) is such an illusion, where color-tinged bars are seen on a black and white screen. The results of studies linking the McCollough effect to processes involving occlusion and chromatic aberration, along with recent data from fMRI brain scans, underline the possibility that this compelling illusion involves both the lower and higher visual areas of the brain. The McCollough effect is a convincing illusion first demonstrated by Celeste McCollough in 1965. In order to experience it, one is first shown, and becomes adapted to, two alternating colored bar gratings for five minutes. The gratings might be composed of vertical black-and-green bars, and horizontal black-and-red bars, for example. When an individual subsequently views a screen with identical black and white bars, he or she will see color hazing around the edges of the black bars. Even more incredible to observe, these illusionary colors are complementary to those of the adaptation, in that the horizontal bars will now appear fringed with green, whereas they were originally red. The effects of the adaptation can be seen for days or weeks1. When the effect was first introduced by McCollough, it was proposed that the early visual area V1 played a central role. The visual cortex is generally believed to be striated into at least

1

The inexperienced and curious reader is encouraged to participate in an online demonstration of the effect produced by the author, located at http://www.electricfox.com/mccollough

3 five main functional levels, from V1 through V5, layered like a stack of pancakes. Visual information is first delivered from the retinas to the area V1, where it is then fed through the remaining levels for various stages of processing. For the purposes of this discussion, we will only concern ourselves with this first area and area V4, which is believed to be central in the perception of color. Information has since accumulated that supports the original theory that the McCollough effect involves activity in V1; for example, people who have suffered severe damage to V1 do not experience the effect. Even more convincing was the discovery that the illusionary tinge was complementary to the wavelength of the adapting stimulus, rather than the perceived color. And since V4 is thought to play a central role in the color constancy system that converts color wavelengths to perceived colors, it was supposed that the effect relied on regions before area V4 (Humphrey et al., 1999). It would seem then that the argument for V1 was all but decided, but a series of findings began to suggest that higher visual areas were involved as well. A study conducted by Watanabe (1994) supports this new theory that the McCollough effect may involve higher-level visual areas, and more specifically, those responsible for figuring out occlusion. When an object is in front of another, it is said to cover or occlude it. Our visual system is able to extract information about the covered object and conclude that it is indeed one object rather than two disconnected halves. In Watanabe’s experiment, subjects were adapted to normal McCollough adaptation gratings, but were shown test gratings that were partially occluded by rectangles. It was hypothesized that subjects would see the McCollough effect despite the occlusion, and that this would suggest that the subjects were mentally connecting the disparate bars in the test grid. The results supported this hypothesis, and it was proposed that the McCollough effect not only relied on the higher visual areas responsible for occlusion, but likely

4 involved some sort of feedback loop to V1. A subsequent study takes this assumption a step further by actually recording McCollough-effect-related brain activity in V4, the area responsible for color perception. In 1999, Humphrey et al. conducted an fMRI brain scanning experiment to locate the region responsible for McCollough adaptation. Based on earlier research evidence linking ME with the early visual pathway, the experimenters expected to find brain activity to be mostly centered in V1. The study involved six subjects and three test periods. The experimenters used three main types of slides during the tests: an adaptation grating slide, similar to the example gratings described previously in this paper, and two test slides, one with bar gratings that were congruent with the adaptation bars (which should incite the illusion), and one with noncongruent bars (which should not). The first of the three tests was a pretest, where subjects were shown only the alternating test slides without first being adapted. This pretest was intended to determine subject bias, and indeed, one subject’s results were disregarded. In the second test period, subjects were shown the adaptation gratings minus coloration, followed by the test gratings. This test was used as a control to ensure that subjects did not show activity based only on contrast adaptation. The final test period involved the actual adaptation to the McCollough gratings, followed again by the two test grids. As expected, all subjects reported experiencing the McCollough effect when viewing the congruent test grating. Scans taken during the viewing of the congruent and non-congruent test slides were compared to determine if the illusion was causing measurable brain activity. Contrary to what was originally expected, only one subject actually showed activity in V1, whereas all of the subjects showed significant activity in area V4. If nothing else, this shows that V4 plays an important role in the McCollough effect.

5 In their discussion of these unexpected results, the researchers suggested a variety of confounders that could explain the unanticipated activity in V4. One such proposal was that the activity in V4 might be associated with the perception of the color in the illusion itself, rather than any actual adaptation. However, this is a moot point – for even if V4’s activity was related to the perception of the colors of the illusion, it would still be related to the illusion, whether or not it is considered central to it. A non-fMRI study has furthered these surprising findings, by suggesting that V4 does indeed play a role in this intriguing visual illusion. In their study, Broerse et al. (1998) suggest that the McCollough effect may be related to the system in V4 that compensates for chromatic aberrations in the human eye. Chromatic aberration, a phenomenon well known to photographers, causes color irregularities to appear on an image, and is due to slight imperfections in the lens. In studies conducted using optical prisms to induce chromatic aberrations, subjects reported that the color irregularities (blue tinges on straight edges) mysteriously disappeared after extended observation. When the prisms were subsequently removed, subjects reported seeing colors that were complementary to the original chromatic irregularities. This finding suggests that the visual cortex is actively compensating for chromatic aberrations, presumably as a routine method of perfecting the visual image from an imperfect lens (Broerse et al., 1998). Broerse et al. conducted a series of five experiments to determine if the McCollough effect was related to that system. For the purposes of this discussion, only the first two will be examined. In the first experiment, the researchers found that the McCollough effect could still be induced by adapting subjects to black bars that were merely framed with fine edges of green and red. The subjects were adapted to the novel color-fringed gratings and shown the test gratings. As hypothesized, all subjects experienced the McCollough effect. The researchers theorized the

6 slight color fringes shown during adaptation were similar to that of the fringes experienced in the original prism experiment, and thus must involve similar mechanisms of correction. They then conducted a virtually identical follow-up study using the more traditional filled-color bars, to ensure that the illusion in the first experiment was indeed the McCollough effect. They found that the illusionary colors of both experiments were the same, and so concluded that the fringes were properly inducing the McCollough effect. This association, between chromatic distortion compensation and the McCollough effect, could explain evolutionarily why the effect persists for a greater amount of time than other visual aftereffects. Chromatic aberrations in the lens would be more long-term problems associated with eye growth – thus, a system designed to account for them would optimally operate slowly. A possible follow-up study might compare the differences in illusion strength and persistence between children and adults, assuming that such a system would be more active and plastic during the period of rapid eye growth in childhood. While it is obvious that further research in this area is necessary to arrive at any definitive conclusions, the evidence linking the McCollough effect to the higher visual areas is mounting. Could it be that the illusion involves many layers of the visual cortex working in parallel, like a game piece moving among the various levels of the board game “Chutes and Ladders”? Regardless of whether or not the answer is ever found, it is clear that examining illusions such as the McCollough effect will lead us to better a understanding of our visual system as a whole.

7 References Broerse J., Vladusich, T., O’Shea, R. P. (1999). Colour at edges and colour spreading in McCollough effects. Vision Research, 39(7), 1305-1320. Humphrey, G. K., James, T. W., Gati, J. S., Menon, R. S., Goodale, M. A. (1999). Perception of the McCollough Effect Correlates with Activity in Extrastriate Cortex: A Functional Magnetic Resonance Imaging Study. Psychological Science, 10(5), 444-448. Watanabe, T. (1995). Orientation and Color Processing for Partially Occluded Objects. Vision Research, 35(5), 647-655.