Recurrent Corticocortical Interactions in Neural Disease

The cerebral cortex consists of many areas, each subserving a more or less distinct function. This view has its roots in the early work of Penfield and today is reflected in functional magnetic resonance imaging literature describing the regions of the brain that are activated during particular tasks, percepts, actions, or thoughts. Moreover, the currently held view is that these areas, and their associated functions, are organized in a hierarchical fashion. Some areas are low level, performing basic operations on the sensory input. Via feed-forward corticocortical connections, this information is transferred to intermediate and high-level areas (Figure 1A) where more sophisticated processes occur, such as object recognition, multisensory integration, decision making, attention, or reasoning.

This view is now being challenged. Feed-forward connections are reciprocated by numerous feedback fibers (Figure 1B). Within areas, there are extensive horizontal connections that link neurons separated by large distances. Together, horizontal and feedback connections provide the anatomical basis for extensive recurrent processing, where low-level and high-level information interacts (Figure 1B and G).³ However, we have only recently gained some insight into how recurrent interactions work and what their function might be—providing cognitive operations ranging from perceptual organization to awareness and attention.⁴ These insights have implications for understanding neurological and neuropsychological disease.

Before identifying the role of recurrent interactions, we must understand the properties of the hierarchical feed-forward network. This has been studied in detail in the visual system. Visual information enters the cerebral cortex mainly through a single area, V1, and is then distributed to higher areas in the occipital, parietal, and temporal lobes (Figure 1A and F). This feed-forward distribution occurs very rapidly. V1 is activated within 40 milliseconds after a stimulus has been presented. Only 10 milliseconds later, many areas in the occipital and parietal lobe respond. After 10 to 20 milliseconds more, the activation includes most visual areas, including those in the temporal lobe. At 120 milliseconds, latency visual responses can be recorded from anywhere in the brain^3,4 (Figure 1F).

The information that is extracted from the visual input during this feed-forward sweep varies from area to area; generally, it grows more and more complex, going from low-level to high-level areas. In low-level areas such as V1, cells have small receptive fields (RFs); they respond only when stimuli are within a particular region of the visual field that is typically not larger than the width of a thumb at arm's length. Each cell has its own RF, so the visual field is covered by the V1 cells. In addition, each cell is tuned to specific features and only responds to stimuli of a particular nature. For example, V1 cells are selective for the orientation of contours that fall within the RF. In higher-level areas, such as the inferotemporal cortex (IT) in the temporal lobe (Figure 1A), RFs are much bigger, as large as about half the visual field. Cells are tuned to complex geometrical shapes, ranging from the abstract to the more realistic, such as faces.⁵

Tuning properties and RFs are determined by the feed-forward sweep because they are evident from the first spikes a cell fires. Therefore, feed-forward processing defines the basic functionality of cortical areas; for example, area IT is an "object recognition" area because cells are tuned to the shapes of objects, and the medial temporal area (in the extrastriate cortex) is a "motion" area because cells are tuned to the direction of motion of stimuli. What is added to this functionality by recurrent interactions? Several studies indicate that they play an important role in perceptual organization, visual awareness, and visual attention.

Consider the visual stimuli depicted in Figure 1C, D, and E, which consist of many similar elements. Clear structure can be observed, in that continuous contours emerge from Figure 1C, and a square figure, overlying a background, can be seen in Figure 1E. Apparently, our visual system automatically groups some elements together to form a coherent object that segregates from the other elements. This process is called perceptual organization and seems to follow certain rules. For example, elements not only group together when they have the same orientation, as in Figure 1E, but also when elements are similar in other ways, such as having the same direction of motion, color, or depth.⁶ Colinearity is another factor. For example, linking is much stronger (and the contour is more visible) in Figure 1C than in Figure 1D because the angles between elements are smaller. In this way, psychologists have revealed that perceptual organization follows certain rules. Besides the rules of similarity (Figure 1E) and colinearity (Figure 1C), other rules such as proximity, good form, and closure have been identified.⁶

The number of objects that can be made by randomly combining the elements of Figure 1C would outnumber the total number of brain cells. Therefore, perceptual organization must be mediated by a more flexible system than feed-forward connections. These provide a fixed set of operations on the visual input that may lead to basic categorization. Beyond that, mechanisms are necessary that are capable of adapting, as it were, to the visual input. In other words, rather than having specialized cells for each and every possible percept, it is much more likely that the brain mediates the rules according to which of these percepts is created. Recent evidence is converging on the idea that recurrent processing is doing just that.

Horizontal connections in V1 are not randomly organized; they link neurons that respond to the same orientation of line segments. The connections are even stronger when RFs are colinearly aligned.⁷ In the anatomical sense, the horizontal connections reflect the rules of similarity and colinearity. At the neurophysiological level, this causes the activity of V1 cells to "reflect" perceptual grouping phenomena. When a V1 cell is activated by a line segment, it responds selectively to its orientation and other features according to its feed-forward RF tuning properties. However, the horizontal interactions will modulate the response to this line segment when other elements are present. When the RF is surrounded by colinear line segments, the neuron will respond more strongly than when it is surrounded by randomly organized line segments (Figure 2A). This phenomenon is called contextual modulation. In fact, the factors that influence perceptual grouping (similarity and colinearity) also influence contextual modulation.⁹

Contextual modulation also reflects perceptual organization in more complex scenes. Figure 2B, C, and D show the contextual modulation evoked by the figure-ground display of Figure 1E. To obtain the plot of Figure 2C, responses from a neuron with its RF at different positions relative to the figure and the background were recorded, such that the line segments within the RF were the same for all conditions (as in Figure 2A¹). Subsequently, the response to a scene with identical line segments but containing no figure was subtracted (Figure 2B). As a result, the initial feed-forward activity (typically occurring at a latency of 40-50 milliseconds) is no longer visible. Only the modulation evoked by recurrent interactions remains. We observe positive modulation (ie, a response enhancement) from about 80 milliseconds after stimulus onset at the boundary between figure and ground. After 100 milliseconds, all line segments that belong to the figure uniformly evoke a stronger response than line segments of the background.¹⁰ It is as if a neural image of the figure is stamped out of the neural image of the background, closely reflecting the figure-ground percept (Figure 2D).

To what extent do these findings show that recurrent interactions reflect perceptual organization? The relation between the organization of horizontal connections in V1 and various contour grouping effects (Figure 1C and D) is evident. More direct evidence is that inactivation of the extrastriate areas selectively interferes with the delayed contextual modulations recorded in V1 (Figure 2B, C, and D) and not with the early feed-forward activity.^4,11 This shows that these modulations are indeed reflecting recurrent interactions between V1 and extrastriate areas. In turn, this suggests that perceptual organization is mediated by recurrent interactions between different widespread areas of the occipital and temporal lobes. How exactly this works remains a topic for further study, but most likely, the spatial and temporal extent of these interactions will depend on the complexity of the scene and on the nature of the rules that are employed in detecting objects and backgrounds.

The findings noted thus far focused on the relationship between recurrent interactions and perceptual organization. However, this is only one example. Many high-level perceptual and cognitive operations may not simply be functions of the high-level cortical areas but may also depend critically on dynamic interactions between low-level and high-level areas. In this new view of cortical functioning, feed-forward hierarchical processing of sensory input merely creates a framework upon which recurrent interactions construct complex operations, resulting in perception, cognition, and flexible behavior.

A specific example of the role of feed-forward vs recurrent processing in conscious visually guided behavior explains this theory more clearly. On the basis of recent evidence from psychophysical, neuroanatomical, neurophysiological, and neuroimaging studies,² it appears that 3 subsequent events are crucial for a conscious response to visual stimulation (Figure 1F and G). First, the feed-forward sweep extracts the elementary features and feature constellations that make up a scene. When the feed-forward sweep reaches the motor areas of the brain, this could in principle trigger a motor response, but if it did, it would be a fully unconscious or automatic visuomotor reflex. The reflex can be sophisticated, such as returning a 100-mph tennis serve, but its execution will begin before you are consciously aware of the visual input causing it (note that this does not imply that you do not become aware of the ball, and you hitting it, at some later point).

Meanwhile, approximately 100 milliseconds after stimulus onset, the visual areas engage in recurrent interactions (Figure 1G). According to findings from various experiments, these interactions are crucial for the development of a conscious percept. For example, when animals are anesthetized, figure-ground modulation is selectively suppressed, whereas feed-forward RF properties are unaffected.¹² Obviously, perception is absent in these animals. In the awake animal, if hundreds of trials of detecting figures on a background are conducted, contextual modulation fails to develop every now and then. When this happens, the monkey does not perceive the figure (Figure 2E⁸). When figure-ground stimuli are masked by subsequent stimuli, contextual modulation is suppressed at the stimulus-mask intervals that lead to reduced visibility of the figure.¹³ From these and related findings, it can be concluded that recurrent interactions most likely cause the phenomenal experience of the stimulus, ie, perception as such.

As these recurrent interactions grow more and more widespread and finally include areas in executive space, such as those in the prefrontal cortex, a behavioral response based on the conscious experience of the stimulus becomes possible (Figure 1G). Because these recurrent interactions have provided a much more elaborate and detailed evaluation of the visual stimulus and its relation to other stimuli and current needs and goals, this response can also be much more flexible than one based on the automatic feed-forward system. Therefore, visual processing proceeds from an automatic unconscious feed-forward process to a recurrent process that generates perception and ultimately may result in a conscious and deliberate behavioral response.²

A second example of a high-level function that is not localized within a single area but is distributed throughout the brain is attention. Initially, the prefrontal and parietal cortices were identified as the prime foci of attentional selection. However, more recently it has been recognized that although these nodes are often involved, attentional selection is a process that operates at all cortical levels, depending on what is selected or attended to. The effects of attention can be recorded in dorsal and parietal visual areas and in high-level areas, such as IT or V4, as well as in low-level areas up to and including V1. Which of these areas are and are not recruited to direct attention in a particular task seems to depend on factors such as sensory modality, the number of items that have to be selected from, the spatial scale at which selection takes place, or whether attention is directed at a particular location or a particular object or feature.¹⁴

When a sufficient amount of neurons are synchronously active, the added synaptic input results in a measurable electroencephalographic (EEG) or magnetoencephalographic signal from the human scalp. Figure 2F shows visually evoked potentials recorded from a subject viewing either homogenously textured displays or displays in which many textured figures, such as those in Figure 1E, were present. As with the neural activity recorded in monkeys, the initial activity (up to about 100 milliseconds) is identical for the 2 stimuli. However, a late visually evoked potential component with a peak latency of approximately 160 milliseconds selectively signals the difference between displays with squares and displays without squares. As the line elements that make up the scenes were kept identical, on average, in the 2 displays, the difference signal must come from a mechanism that is sensitive to the difference in perceptual organization.¹⁵ Equivalent dipole analysis, depth recordings,¹⁶ and subsequent single-unit studies in monkeys have confirmed that the difference signal is in fact identical to the contextual modulation that can be recorded from the V1 area in monkeys (Figure 2C). The 160-millisecond latency difference signal thus reflects the recurrent interactions involved in perceptual organization.

Synchrony between distant neurons is not only a prerequisite for a recordable EEG signal. It is also a reflection of recurrent interactions. Single-unit studies have revealed that synchronous firing between neurons reflects horizontal interactions in V1, ie, synchrony is stronger between neurons with similarly oriented and colinear RFs. Also, whether stimuli on 2 distant RFs belongs to the same object affects the amount of synchrony between the neurons. Such long distance synchrony is often associated with oscillatory EEG activity in the gamma range (frequency from 30-70 Hz).¹⁷ In human subjects, it has been found that the successful organization of scene components as recognizable objects is associated with increased synchronous EEG activity over the occipital cortex.¹⁸ This further confirms the relationship between perceptual organization and synchronous firing.

Although our knowledge about the role of recurrent interactions is relatively fresh, several neural abnormalities that could be associated with a disturbance of these interactions may already be identified. In some cases, the evidence that the dysfunction is related to a disturbance of recurrent interactions is strong (amblyopia), and, in other cases, it is compelling but still hypothetical (schizophrenia and autism).

In strabismic amblyopia, one eye has a strongly reduced visual acuity because of squinting and subsequent monocular suppression during childhood. Nevertheless, the amblyopic eye is as capable of driving cells in the visual cortex as is the normal eye. Therefore, the reduced visual acuity cannot be attributed to a failure of the feed-forward activation of neural pathways. Instead, it appears that the problem is caused by a disturbance of the dynamic interactions among visual neurons. These interactions, mediated by horizontal connections, are expressed by the synchronous firing of simultaneously recorded neurons. Synchrony between neurons that are driven by the amblyopic eye is less strong than is synchrony between neurons driven by the healthy eye.¹⁹ Thus, a reduction in the amount of horizontal interactions among neurons leads to a marked perceptual deficit.

Schizophrenia and autism are complex syndromes with many subtypes and a multitude of neuropsychological symptoms. However, in both conditions, patients often have problems with perceptual organization; they focus too much on the details and elements of a visual scene and are less capable of integrating the information into a global percept.^20,21 For example, patients with schizophrenia or autism have difficulties detecting the integrated contours of Figure 1C and D. Both diseases have a strong genetic components, but the neural abnormalities are still obscure. However, visually evoked potential studies in these patients show that the late (about 150 milliseconds) occipital activity,²² which is the activity associated with recurrent processing, is different from that in healthy subjects. Also, synchronous gamma activity seems to be reduced,²³ again suggesting that the problem may be caused by a deficit of recurrent interactions. Much more research is needed, however, to fully elucidate the possible deficits of recurrent processing in patients with these conditions.

Perceptual organization is most severely disturbed in apperceptive agnosia. Patients with this disorder cannot make sense of scenes. They perceive all the elements but are unable to group these elements into a coherent scene of objects and background. This condition is often caused by lesions to the extrastriate areas but may also result from more diffuse damage to the occipital cortex, such as in carbon monoxide poisoning.²⁴ Both conditions restrict the recurrent interactions among areas, but feed-forward activation is compromised as well. The most extreme example of a combined feed-forward and recurrent deficit is hemianopia following a V1 lesion. Whereas the feed-forward component of this condition is obviously the strongest, some arguments (reviewed in a recent article²⁵) attribute the loss of visual awareness in these patients to a disturbance of recurrent interactions.

In hemispatial neglect and extinction, patients ignore stimuli contralateral to the lesion, which is usually right-sided. The lesion typically includes regions of the parietal or parieto-occipital lobe. There is considerable controversy about the exact nature of the deficit in patients with neglect and extinction.^26,27 Do they simply not respond to contralateral stimuli, do they fail to attend to them, or do they have a perceptual deficit that causes them not to act on contralateral stimuli? The latter seems unlikely. In extinction, which is typically the milder form of neglect that remains after revalidation, contralateral stimuli are only ignored if there are ipsilateral stimuli as well. In the scheme of Figure 1F and G, this would imply that the deficit in neglect is mostly in linking the posterior recurrent interactions with frontal or executive processes. Indeed, there is substantial evidence that visual processing proceeds normally up to and including the stage of perceptual organization.²⁷ It might even be that patients have phenomenal experience of contralateral stimuli but fail to attend and respond to them. In that sense, the condition is almost orthogonal to that of patients with hemianopia, who have no phenomenal experience but nevertheless respond to contralateral stimuli.

Recurrent corticocortical interactions are central to our cognitive capabilities of conscious perception, selection, and action. Several neuropathological conditions can be identified in which these functions are compromised, probably because of the abnormal way in which recurrent interactions unfold. Whether, and in what way, recurrent interactions are affected in these conditions is still a topic of study. This pursuit will benefit from further investigations into the role of recurrent interactions in the normal condition, as well as from clinical studies. Electrophysiological recordings in patients, when combined with paradigms that selectively address the functioning of recurrent interactions (Figure 2F), will prove most helpful in further clarifying that role and in further fine-tuning diagnosis and classification.

Corresponding author: Victor A. F. Lamme, PhD, Cognitive Neuroscience Group, Room A626, Department of Psychology, University of Amsterdam, Roeterstraat 15, 1018 WB Amsterdam, the Netherlands (e-mail: vlamme@fmg.uva.nl).

Accepted for publication September 17, 2002.