The capacity of visual short-term memory is highly limited, maintaining only three to four objects simultaneously1,2. This extreme limitation necessitates efficient mechanisms to select only the most relevant objects from the immediate environment to be represented in memory and to restrict irrelevant items from consuming capacity3,4,5. Here we report a neurophysiological measure of this memory selection mechanism in humans that gauges an individual's efficiency at excluding irrelevant items from being stored in memory. By examining the moment-by-moment contents of visual memory6, we observe that selection efficiency varies substantially across individuals and is strongly predicted by the particular memory capacity of each person. Specifically, high capacity individuals are much more efficient at representing only the relevant items than are low capacity individuals, who inefficiently encode and maintain information about the irrelevant items present in the display. These results provide evidence that under many circumstances low capacity individuals may actually store more information in memory than high capacity individuals. Indeed, this ancillary allocation of memory capacity to irrelevant objects may be a primary source of putative differences in overall storage capacity.
To examine the selection mechanism for allocating memory capacity, we recorded event-related potentials from healthy young adults while they performed a visual memory task7 in which it was necessary to remember selectively only a few relevant items from within an array. On each trial they were presented with a brief bilateral array of coloured rectangles of varying orientations and were asked to remember the orientations of only the red items in either the left or right hemifield, as indicated by an arrow (Fig. 1a). Memory for these red items was tested 1 s later with a test array that was either identical to the original memory array or differed by one orientation. Subjects reported whether the red items in the two arrays were identical or not by pressing one of two buttons. On a third of the trials, two red items were presented along with two blue items in each hemifield. On the remaining trials, arrays of either two red items or four red items alone were presented in each hemifield.
To observe directly whether the subjects could exclude the irrelevant blue items from being stored and maintained in visual memory, we measured a waveform of the event-related potential that reflects the encoding and maintenance of item representations in visual working memory6. This wave is a sustained negative voltage over the hemisphere that is contralateral to the memorized hemifield, and this activity persists throughout the memory retention interval. The amplitude of this contralateral delay activity (CDA) increases significantly as the number of representations being held in memory increases, reaching an asymptotic limit at each individual's specific memory capacity (ref. 6; A.W.M., M.G.M. and E.K.V., submitted).This limit is measured as a difference in amplitude between an array of four items and an array of two items. Low capacity individuals show a smaller difference than high capacity individuals, indicating that an array of two items consumes a larger proportion of available memory capacity for low capacity subjects.
Because of the sensitivity of this measurement to the number of items that are currently held in memory, we used the CDA as a direct neurophysiological measure of whether or not the irrelevant distractor items unnecessarily consumed memory capacity. For example, on the trials in which two red items were presented simultaneously with two blue items, if an individual was perfectly efficient at remembering only the red items and excluding the blue items from memory, then the CDA amplitude should be equivalent to that observed when two red items were presented alone. By contrast, if an individual was perfectly inefficient at excluding the blue items, all four of the items in the array (two red and two blue) would be stored in memory, resulting in an amplitude equal to that when four red items alone were presented.
Memory capacity varies considerably across individuals, ranging from 1.5 objects to about 5 objects6,7. To examine whether memory selection efficiency varies across memory capacity, we estimated each individual's memory capacity8,9 and divided the subjects into two groups: high and low capacity. These two groups differed markedly in their filtering efficiency abilities (Fig. 1b). For the high capacity group, the amplitude of the distractors-present condition was significantly smaller than that of four red items alone (P < 0.001) but was not significantly different from that of two red items alone (P > 0.20), indicating that these subjects were very efficient at excluding the distractors from consuming memory capacity. By contrast, the low capacity group had an amplitude in the distractors-present condition that was significantly larger than that in the two items alone condition (P < 0.001), but not significantly different from that in the four items alone condition (P > 0.25). These results indicate that low capacity subjects were highly inefficient at keeping the irrelevant items from being stored in memory.
We measured this relationship more formally by quantifying each subject's filtering efficiency (Methods). The scores are plotted as a function of each individual's memory capacity in Fig. 1c. These two measures were very strongly correlated (r = 0.69; P < 0.001): low capacity subjects showed low filtering efficiency scores, and high capacity subjects produced much higher efficiency scores. These results contrast with studies that have examined the neural bases of individual differences and have often reported complex relationships between the difficulty of the task and the magnitude of the neural activity10,11,12. However, the CDA is primarily modulated by the number of objects held in memory rather than the difficulty of the task6, which may explain the simple relationship observed here.
The first experiment indicated that low capacity subjects are highly inefficient at excluding information on the basis of the colour of an item. However, previous research has shown that colour-based selection tends to be very difficult and inefficient relative to other selection attributes13. Consequently, it is possible that the relationship between memory capacity and filtering efficiency is present only under challenging filtering conditions. In experiment 2, we examined whether this relationship would generalize to a task in which subjects must filter distractors on the basis of location, a selection attribute that is considerably easier than selection by colour14. Figure 2a shows an example of a distractors-present trial in which the subject is cued to remember the colours of only the items in the upper left quadrant and to exclude the items in the lower quadrant.
Figure 2b shows the CDA difference waves for the high capacity and low capacity groups for the three conditions. As in experiment 1, for the high capacity group the distractors-present condition had an amplitude that was equivalent to that in the two items alone condition and significantly smaller than in the four items alone condition (P < 0.001). By contrast, the low capacity group in the distractors-present condition had an amplitude that was significantly lower than in the four items alone condition (P < 0.01), but was significantly higher than in the two items alone condition (P < 0.01), indicating that this group was inefficiently storing information about some of the irrelevant distractors. Figure 2c shows the filtering efficiency scores plotted as a function of each subject's memory capacity. Whereas low capacity subjects were much more efficient than they were in the colour-based selection task, they were still considerably less efficient than the high capacity subjects (r = 0.62; P < 0.001), indicating that the relationship between memory capacity and filtering efficiency generalizes to both feature- and location-based selection.
It is plausible that the results of the first two experiments are due to a general inability of low capacity individuals to exert effective control over any aspect of working memory functioning, rather than to a more specific inability to exclude irrelevant items from being stored. An aspect of control over working memory is the ability to append new items into memory without overwriting existing items held in memory15,16. In experiment 3, we examined whether low capacity subjects were also limited in their ability to append successfully, or whether their limitations are primarily restricted to situations that require the exclusion of irrelevant items from memory. Subjects were instructed to remember the orientations of only the red items in the cued hemifield. On half of the trials, subjects were presented with a single memory array that consisted of either two or four red items alone in the hemifield. On the other half of trials, subjects were presented a sequence of two memory arrays separated by 500 ms. The first memory array consisted of two red items and the second array consisted of either two red items which were to be appended (append red) or two green items which were to be excluded (exclude green). After a 1-s retention interval, all four items from both memory arrays were presented together in the test array and the subjects responded whether any of the red items had changed orientation or not.
Figure 3 shows the results of experiment 3 divided across high and low memory capacity subjects. For both groups, in the append red condition CDA amplitude was initially equivalent to an array size of two. Shortly after the onset of the second array, however, the amplitude rose to the equivalent of an array of four items. That is, both high and low capacity subjects showed a perfect additivity of the two arrays in terms of the amount of memory capacity consumed, indicating that low capacity subjects were not impaired at appending the items into working memory.
By contrast, in the exclude green condition large differences between the high and low capacity subjects were observed. For the high capacity subjects, CDA amplitude was initially equivalent to an array of two items. After the onset of the second array, CDA amplitude briefly rose to almost a four-item level but then quickly returned to near its original two-item level. For the low capacity subjects, however, CDA amplitude rose to the equivalent of a four-item level that was maintained throughout the retention interval, indicating that the green items had been unnecessarily appended into working memory. These results suggest that, although both low and high capacity subjects can append items into working memory, these two groups substantially diverge in their abilities to determine selectively which items will be appended into memory.
The control processes that regulate access to working memory are crucial for keeping irrelevant information from consuming capacity. Our results show that there is systematic variability across human individuals in the ability to control what is stored in working memory at any given moment. Neurophysiological studies in monkeys have indicated that the prefrontal cortex has a crucial role in determining what information is to be maintained in memory3,17,18, and it is plausible that the individual differences reported in this study may stem from variability in a bias signal emanating from prefrontal cortex19,20,21,22. A further implication of our study is that individual differences in memory capacity may not simply reflect variability in available storage space, but may also be strongly constrained by the efficiency with which the available space is allocated. By this view, an individual's specific memory capacity does not simply reflect ‘how many’ items can be stored, but also ‘how efficient’ the individual is at excluding irrelevant information from reaching this highly limited memory system23.
Fifteen neurologically normal college students participated in each experiment (age range 19–28 yr) and gave informed consent according to procedures approved by the University of Oregon. Each of these observers performed between 200 and 240 trials per condition in each experiment. All stimulus arrays were presented within two 4° × 7.3° rectangular regions that were centred 3° to the left and right of a central fixation cross on a grey background (8.2 cd m-2). Stimulus positions were randomized on each trial, with the constraint that the distance between objects within a hemifield was at least 2° (centre to centre).
In experiment 1, each memory array consisted of two or four oriented rectangles (0.65° × 0.65°) in each hemifield selected randomly from a set of four orientations (vertical, horizontal, left 45° and right 45°). In experiment 2, each memory array consisted of either two or four coloured squares in each hemifield. Each colour was randomly selected with limited replacement from a set of seven easily distinguished colours (red, blue, green, violet, yellow, black and white). The positions of the items were randomly distributed within the upper and lower quadrants of each hemifield. In the two items alone condition, both squares were presented in either the upper or the lower quadrant. In the four items alone condition, two items were presented in each quadrant.
In experiment 3, the first memory array consisted of either two or four red oriented rectangles. On half of the trials, a second memory array was presented 500-ms later consisting of two rectangles that were either red or green and were presented at new locations in the same general region as the first memory array. The 500-ms delay enables us to establish the CDA amplitude for the first memory array before the onset of the second array and provides sufficient time to extend beyond the duration of iconic memory for the first array.
Memory capacity and filtering efficiency
We computed visual memory capacity with a standard formula8,9 that essentially assumes that if an observer can hold in memory K items from an array of S items, then the item that changed should be one of the items being held in memory on K/S trials, leading to correct performance on K/S of the trials on which an item changed. To correct for guessing, this procedure also takes into account the false alarm rate. The formula is K = S(H - F), where K is the memory capacity, S is the size of the array, H is the observed hit rate, and F is the false alarm rate. Subjects were divided into high capacity and low capacity groups using a median split of their memory capacity estimates.
We quantified each individual's filtering efficiency with a formula in which we computed the mean amplitudes of the CDA across three conditions: two items alone, four items alone, and the distractors-present condition. In essence, this efficiency score measures whether the CDA amplitude on the distractors-present condition is more similar to that on the four items condition or the two items condition, with a range of scores from 1 (efficient: identical to two items) to 0 (inefficient: identical to four items). The formula is α = (F - D)/(F–T), where α is the filtering efficiency, F is the amplitude for four items, D is the amplitude for the distractors-present condition, and T is the amplitude for the two items alone condition. It is important to note that it is mathematically possible for this formula to yield a value outside the range 0 to 1 (for example, if T > F or if D < T). Across all of the subjects in this study, however, there was no single case that met any of these conditions.
Event-related potentials (ERPs) were recorded in each experiment using our standard recording and analysis procedures24, including rejection of trials contaminated by blinks or large (> 1°) eye movements. We recorded from 22 standard electrode sites spanning the scalp. We computed contralateral waveforms by averaging the activity recorded at right hemisphere electrode sites when subjects were cued to remember the left side of the memory array with the activity recorded from the left hemisphere electrode sites when they were cued to remember the right side. Contralateral delay activity was measured at posterior parietal, lateral occipital and posterior temporal electrode sites as the difference in mean amplitude between the ipsilateral and contralateral waveforms, with a measurement window of 300–900 ms after the onset of the memory array. Mean amplitudes were compared across conditions by analysis of variance.
This work was supported by grants from the US National Institute of Mental Health and the Oregon Medical Research Foundation.