Neural correlates of perception and working memory in the primary visual cortex. A, Neurons in the primary visual cortex only "see" a small part of the visual scene through their receptive field (RF). The neural response to the stimuli that fall within the RF is modulated by stimuli that fall outside the RF. In a figure-ground display, this results in an enhancement of the neural activity in the late part of the response to the line segments of the figure (blue line) compared with the response to line segments of the background (green line). This enhancement of activity is the neural correlate of the perception of the figure (red-shaded area). B, The perceptual signal continues in the absence of the figure and represents a memory trace of the figure. Note the general suppression of activity.
To study the neural mechanisms underlying mental functioning, animals are tested with delayed-response tasks whereby they have to remember the location of a briefly presented figure (dark square on the monitor). While the animal is performing the task, multiunit activity is recorded in the primary visual cortex (red-shaded brain area). The obtained neural signal (spiking activity) is full-wave rectified and low-pass filtered to obtain an envelope or amount of spiking activity over time (red line). Then the neural responses to the many trials of the task (shaded vertical bands) are selected according to the visual and behavioral condition. These data are averaged and give an estimation of the neural processing of the stimulus (green line) in relationship to the animal's behavior.
Supèr H. Working Memory in the Primary Visual Cortex. Arch Neurol. 2003;60(6):809-812. doi:10.1001/archneur.60.6.809
The primary visual cortex is the first cortical area of the visual system that receives information from the external visual world. Neurons in the primary visual cortex have small receptive fields and respond to basic elements of the visual scene. Recent findings, however, show that the primary visual cortex is also involved in cognitive processes, such as visual perception and working memory. These neural correlates are expressed in the late part of the neural response to a visual stimulus. These new observations provide a neural link between the perception of a visual object and the subsequent storage of the visual information in working memory. The inability to store the information and poor perception contribute to the impairment of visual working memory.
Working memory, or short-term memory, is a cognitive system for the maintenance of behaviorally relevant information. This retention of information is a selective process, ie, not all of the sensory information that enters the brain is maintained, and only a limited amount of information is stored into memory. In vision, the storage of information in working memory is understood in terms of complete objects rather than the individual elements of the object.1 For example, Figure 1A shows a textured figure that is formed by grouping the similarly orientated line segments and segregating them from the line segments that have an orthogonal orientation (background). Thus, it is not the individual line segments that are remembered but the entire figure as an object. At the neural level, this means that the activity that encodes the figure is retained and not the responses to the individual line segments. Therefore, to understand the neural mechanisms of working memory, one has to know the neural representation of the visual object.
The primary visual cortex is the first cortical area to receive information from the visual world surrounding us. The neurons in this area receive information from a small part of the visual environment (classical receptive field) and become active only when specific elements, like orientation, contrast, or motion of an object are presented in the receptive field location. For visual perception, the individual elements need to be grouped into a coherent and meaningful object.2 Recent findings show that in figure-ground segregation, the grouping of the receptive field elements occurs in the primary visual cortex. This grouping results in the enhancement of the neural response to the line segments of the figure compared with the response to the line segments of the background (Figure 1A). The enhanced response to the figure represents the neural correlate of the perception of the figure.3 Therefore, the primary visual cortex not only encodes low-level vision, such as individual line segments, but also higher-level vision, such as the perception of a figure. This offers the opportunity to study working memory and to investigate the link between perceptual activity and the neural mechanisms of working memory.
For the study of working memory in humans and animals, a delayed-response task is commonly used. In such a task, a stimulus is presented briefly, followed by a delay, during which the stimulus is removed. The subject must then respond using the mental representation of the stimulus. Many studies have found neurons with sustained responses during the delay period in the higher visual areas but have failed to link this memory trace with perceptual activity. In the primary visual cortex, the memory signal of the figure appears to be a continuation of the perceptual signal,4 ie, the enhanced neural response to the figure compared with the background (Figure 1B).
The maintenance of the perceptual figure-ground activity in the primary visual cortex is a selective process whereby the figure-ground signal continues when the figure is relevant to the goal of the animal, and becomes disrupted when the figure is irrelevant. This provides evidence for the existence of a neural correlate of working memory in the primary visual cortex. The maintenance of the figure-ground activity as a storage mechanism furthermore demonstrates a neural transformation of the activity that signals the perception of the figure into a memory trace of the figure.
To explore the neural implementation of cognitive functions in the brain, one needs to record the activity of neurons during a behavioral task that measures cognitive performance. By combining the results from the behavioral task with the recorded neural activity, the neural mechanisms underlying cognitive functions can be unraveled.
To study the neural mechanisms of working memory, monkeys are engaged in a delayed-response task whereby they have to remember the location of a figure. During the task, neural activity is recorded, which can be done in several ways. A direct and appropriate way to measure neural activity is to record spike activity (action potentials) of single (single-unit recordings) or small groups (multiunit recordings) of neurons using intracortical electrodes. To quantify the amount of neural activity, the number of spikes is counted or the envelope of activity, which is an estimate of the total amount of spike activity, is calculated (Figure 2). Finally, the relationship between neural activity and behavior is analyzed by selecting the neural data according to the visual event (eg, onset of figure-ground stimulus) and the performance of the animal (eg, correctly remembered trials).
The visual system consists of ˜30 cortical areas distributed throughout the cerebral cortex. All of these areas are reciprocal, connected to each other via feed-forward and feedback projections. When a visual stimulus is presented to the retina, it triggers a rapid cascade of neural responses over the entire visual system. The feed-forward axonal projections provide the neurons of the primary visual cortex with the retinal information, so that within ˜30 to 100 milliseconds, visual neurons respond to the elements that fall within their receptive field.5 The figure-ground activity that signals the perception of the figure appears around 80 to 120 milliseconds after the onset of the figure-ground stimulus (Figure 1A). This indicates that the grouping of line segments into a coherent object (eg, a figure) develops after the initial response to the visual elements.
The grouping depends on the context of the elements in the visual scene; therefore, the elements that fall outside the neuron's small classical receptive field location modulate the neural response to the visual elements that fall within the receptive field.6 These modulatory effects on the receptive field response in figure-ground segregation are mediated by the recurrent interaction among the neurons in different cortical sites and areas.7 This signifies that the neural organization of figure-ground perception has a distributed nature.
It has been put forward that the prefrontal cortex has a central role in maintaining and manipulating the processing of visual information for memory.8 This executive process comprises the enhancement of relevant information processing and the inhibition of irrelevant processing. The primary visual cortex receives feedback from extrastriatal, parietal lobe, and temporal lobe areas, but not directly from prefrontal areas.9 Therefore, if the prefrontal cortex has a central role in memory retention, the modulated neural activity in the primary visual cortex during the retention period derives indirectly from the prefrontal cortex through multiple stages of feedback processing. This suggests that the mechanism underlying working memory is not a localized phenomenon but a distributed process, covering the full extent of the cortical visual system.
The strength of neural activity is a measure of the involvement of neurons in information processing. Apart from the specific enhancement of the figure compared with the background, the electrophysiologic findings of working memory in the primary visual cortex show a strong, global suppression of neural activity during the retention period (Figure 1B). This is different from the delayed responses found in higher-order visual areas that do not show suppression.
In the study of vision, functional magnetic resonance imaging (fMRI) is a relatively new technique that indicates brain regions involved in mental processing. The fMRI technique records blood flow changes that result from neural activity. The underlying assumption is that blood flow to areas with increased neural activity increases. Thus, the brain regions that show high fMRI signals during a mental task are considered to contribute to the cognitive process. However, a role of the primary visual cortex in working memory has remained unnoticed in fMRI studies. Due to suppression of the overall activity, this location-specific memory signal may vanish by analyzing the enhancement of the more global responses. Therefore, in contrast with the delayed responses in higher-order areas, the suppression of overall activity in the primary visual cortex may result in the failure to show a role in memory. Thus, exclusively analyzing the global response enhancement may give an incomplete picture of the brain areas involved in mental processing.
The findings of neural correlates of figure-ground perception and the storage of information in the primary visual cortex aid in our understanding of the neural basis of the transformation of perception into working memory. The linkage of these 2 cognitive functions may result in correspondence between the performance of perception and memory capacity, whereby clear visible objects are better remembered than faintly perceived ones. Therefore, patients with deficits in visual perception may produce impairments in visual working memory, such as in cases where the cause of the memory impairment is due to poor perception instead of the inability to store the information. For example, schizophrenic patients, who often show impaired memory performance, may also have perceptual deficits.10
Corresponding author and reprints: Hans Supèr, PhD, Department of Visual System Analysis, AMC, University of Amsterdam, PO Box 12011, 1100 AA Amsterdam, the Netherlands (e-mail: email@example.com).
Accepted for publication November 13, 2002.