Background & Summary

Visual attention refers to the cognitive process of selectively focusing on specific elements of a visual scene while filtering out irrelevant information, enabling efficient perception and decision-making in complex environments1,2,3. Goal-directed visual search is a critical aspect of visual attention, involving the deliberate and focused scanning of a visual environment to locate specific targets based on current goals and expectations. Natural visual search involves feature-based attention, which enhances the representation of relevant stimuli across the visual field, and spatial attention, which leverages foveal visual processing while continuously scanning the periphery4,5. Behavioral studies have demonstrated that both peripheral and foveal visual fields are essential for visual search. The peripheral field aids in selection, searching, and guiding the search, while the foveal field supports object analysis, observation, and recognition6,7,8,9,10,11 (see also fMRI evidence12). However, there is limited neural evidence at the single-unit resolution encompassing both foveal and peripheral populations.

Most neuronal-level visual search studies have recorded activity from units with receptive fields (RFs) confined to either the peripheral/parafoveal4,13,14,15,16,17 or central/foveal18,19,20 visual field. Moreover, many studies employed simplified or constrained stimuli: for example, combinations of eight colors and eight shapes4, eight object images14, a letter “T” as the target and “ + ” as the distractor15, circular image patches from black-and-white photographs13,16, and letter-like figures (e.g., T, I, E, L, F, O, Z)17 (but see also19 for 70 natural and man-made objects and18,20 for 20 visual search arrays, each containing 24 items). These limitations have restricted a comprehensive understanding of visual and cognitive processes in brain regions essential for visual attention. Notably, there are no publicly available single-unit datasets encompassing a large population of both foveal and peripheral units for naturalistic visual search.

To address this gap, we present a large neuronal dataset to the research community, offering several significant advantages. First, we recorded a total number of 6871 units from V4, 8641 units from IT (including both TE and TEO), 5622 units from the OFC, and 9916 units from the LPFC. Among these, 5070 units from V4, 5051 units from IT, 1470 units from the OFC, and 2997 units from the LPFC were visually responsive units. These units exhibited RFs covering the foveal, peripheral, or both regions, as well as neural selectivity across visual categories. This diversity in neuronal properties and the large number of recorded units are critical for a comprehensive understanding and modeling of visual attention. Second, we utilized naturalistic stimuli and trained monkeys to perform a visual search based on four object categories among 160 images. To the best of our knowledge, this represents one of the few natural object category-based visual search tasks used with monkeys, enabling insights into how the primate brain performs complex visual attention tasks. Third, unlike most monkey studies that constrain gaze to a fixed, limited number of positions through behavioral controls, we employed a natural free-gaze visual search task with fewer behavioral constraints, allowing for voluntary eye movements. With these advantages, we can investigate the neural mechanisms underlying the ___domain specificity of social attention21, the properties of RFs22, and the interplay between visual feature coding and attention23. Together, this paradigm, combined with extensive neuronal recordings, advances our understanding of the neural mechanisms underlying natural visual search behavior and contributes to the development of more accurate computational models for visual search.

Methods

Animal welfare statement

Non-human primate research represents a small yet essential component of neuroscience. The scientists involved in this study acknowledge their responsibility to conduct high-quality research while minimizing any discomfort or distress experienced by the animals. Sharing the study data in a standardized, open-access format aligns with this commitment, promoting transparency, accessibility, and optimal use of the data. This approach adheres to the principles of the 3Rs and supports international efforts to enhance the reporting and accessibility of biomedical research data.

All animal procedures in this study adhered to the guidelines of the Institutional Animal Care and Use Committee (IACUC) and were approved by the Animal Care and Use Committee at the Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences, under permit number SIAT-IRB-160223-NS-ZHH-A0187-003.

In accordance with SIAT’s protocol for the care and use of laboratory animals, the psychological and veterinary welfare of the animals were carefully monitored throughout the study by veterinarians, facility staff, and researchers specializing in non-human primates. After the experiment concluded, the animals remained in their original housing with access to free drinking water and food, as well as regular health checks and quarantine.

Animals and general procedures

Two male rhesus macaques (Macaca mulatta), weighing 12 kg and 15 kg on average, were used in this study (see Table 1 for details). The monkeys were trained to perform a category-based visual search task and a visually guided saccade task. The animals were head-fixed (using a headpost) while performing the tasks. Training began with familiarizing them with sitting in a primate chair. Once acclimated, they were introduced to the laboratory for task-specific training. The training started with a simplified version of the task and gradually progressed to the full task. The training phase was considered complete when the monkeys achieved stable performance above 85%.

Table 1 Subject information.
Full size table

Under aseptic conditions, each monkey was implanted with a head post for fixation and two recording chambers, providing access to brain regions including V4, IT, OFC, and LPFC. Localization of the chambers was based on MRI scans obtained before surgery. Surgeries were performed under sterile conditions using gas anesthesia following intramuscular injection induction, with pain levels assessed throughout. Specifically, before the surgery, atropine and ketamine were administered to induce muscle relaxation and anesthesia, followed by isoflurane gas to maintain anesthesia. During the surgery, various measurements were taken using an anesthesia monitor, including heart rate (ECG), blood oxygen saturation, respiratory rate, and rectal temperature. Post-operative monitoring and care, including pain relief, fluid supplementation, and infection prevention, were provided along with clinical assessments.

Daily routine

The animals were housed in a standard facility alongside other macaque monkeys, with controlled lighting and free access to food, water, and fruits. Veterinarians and caretakers conducted daily observations of both the animals and the facilities. During a typical workday, each animal was removed from its cage, placed in a primate chair for weighing, and then brought to the laboratory for training or recording sessions. Successful task performance was rewarded with juice, and sessions concluded based on the animal’s engagement. After each session, the animal was returned to its cage, with daily water intake monitored to ensure adequacy; additional water was provided in the cage if necessary. Access to food remained unrestricted and was freely available at all times.

Experimental setup and neural recordings

The experimental setup included electrophysiological recording hardware and software, an infrared eye-tracking system, a primate chair, a reward system, and an experimental control computer. Behavioral experiments were managed using a computer running MonkeyLogic software (University of Chicago, IL, USA; https://monkeylogic.nimh.nih.gov/index.html)24, which presented stimuli, monitored eye movements, and controlled reward delivery via PCIe-6321 multifunction DAQ cards connected to a BNC-2090A breakout box (National Instruments, Austin, TX, USA). Eye movements were tracked using an infrared eye-tracking system (iViewX Hi-Speed, SensoMotoric Instruments [SMI], Teltow, Germany), with a sampling rate of 500 Hz. The reward system comprised a graduated cylinder filled with juice, connected to a solenoid valve that regulated juice delivery through a feeding tube.

Extracellular recordings were daily performed using a 128-channel recording system (Cerebus System, Blackrock Microsystems, Salt Lake City, UT, USA; Table 2) with 24- or 32-contact electrodes (V-Probe or S-Probe, Plexon Inc, Dallas, TX, USA; Table 3). The electrodes were inserted through the dura mater under the protection of a guiding tube and were advanced to the targeted brain area using an electrode manipulator (Narishige Scientific Instrument, Tokyo, Japan). Recordings were filtered between 250 Hz and 5 kHz, amplified, and digitized at 30 kHz to obtain spikes. Spikes from different units were isolated using Offline Sorter software (Plexon Inc., Dallas, TX, USA; https://plexon.com/products/offline-sorter/) after the recording was completed. Single-unit and multi-unit spikes were acquired from V4, IT, OFC, and LPFC, with activity from two or three of these areas recorded simultaneously in most sessions. The locations of recordings were verified with MRI (Fig. 1 and Table 4). Additionally, task event codes and eye position signals were sent to the Cerebus system in parallel via digital and analog inputs, respectively.

Table 2 Summary of the neural recording system.
Full size table
Table 3 Electrode information.
Full size table
Fig. 1
figure 1

MRI images of typical recording sites and all sites overlaid on the rhesus monkey brain atlas in stereotaxic coordinates28. The red arrow points to the electrodes. The red and blue lines represent the estimated spatial range of recordings in monkey S and monkey E, respectively. The numbers below indicate the rostral (+) or caudal (−) distances of the slices from the Ear Bar Zero.

Full size image
Table 4 Stereotaxic coordinates of the brain areas for recordings (according to the atlas of the rhesus monkey brain28).
Full size table

Behavioral tasks

Monkeys were trained to perform a category-based free-gaze visual search task (Fig. 2a) using stimuli consisting of 160 natural object images across four categories: 40 faces, 40 houses, 40 flowers, and 40 hands, each subtending a visual angle of approximately 2 × 2 degrees. After a 400 ms fixation on a central spot, a cue stimulus was presented at the center of the screen. The cue stimulus was followed by the central spot after a random interval of 500 to 1300 ms. Following another 500 ms fixation on the central spot, a search array with 11 items, including two target stimuli, appeared. The 11 search items appeared at 11 locations randomly selected from 20 predefined positions (Fig. 2b).

Fig. 2
figure 2

Task sequence and predefined search array locations. (a) Monkeys initiated the trial by fixating on a central point for 400 ms. A cue was then presented for 500 to 1300 ms. After a delay of 500 ms, the search array with 11 items appeared. Monkeys were required to fixate on one of the two search targets that belonged to the same category as the cue for at least 800 ms to receive a juice reward. The white trace indicates eye gazes. (b) The 20 possible stimulus locations in the visual search task and the visually guided saccade task. Peripheral receptive fields (RFs) were mapped using the visually guided saccade task, which had the same 20 possible stimulus locations as the visual search task.

Full size image

The two target stimuli and the cue stimulus belonged to the same category, although the target stimuli were always different from the cue stimulus. The cue stimulus was selected randomly from the house or face stimuli with equal probability. The other nine stimuli in the search array belonged to the other three categories. Monkeys were required to start the search within 4000 ms, find one of the two targets, and maintain fixation on the target for 800 ms to receive a juice reward. No constraints were placed on their search behavior, allowing the animals to conduct the search naturally.

Before the search array onset, monkeys were required to maintain central fixation, keeping their gaze within a circular window with a radius of 2° around the central spot/cue (referred to as the “fixation window”). The 20 predefined locations (Fig. 2b) covered the visual field with eccentricities ranging from 5.00° to 11.18° and included 18 locations symmetrically distributed in the left and right visual fields (nine on each side) and two locations along the vertical midline. It is worth noting that each ___location was surrounded by other locations at a minimum distance of 5°. Thus, during the search, when the animal fixated on an item, the nearest surrounding items were at least 5° away, facilitating the analysis of fixations following the array onset.

A visually guided saccade task was employed to map the RFs of the recorded units. Following a 400 ms central fixation, a stimulus randomly appeared in one of the 20 peripheral locations, and monkeys were required to make a saccade to the stimulus within 500 ms and maintain fixation on it for 300 ms to receive a reward.

Receptive field

The visual response to the cue and the search array in the free-gaze visual search task was assessed by comparing the firing rate during the post-stimulus period (50 to 200 ms after cue/array onset) to the corresponding baseline (−150 to 0 ms relative to cue/array onset) using a Wilcoxon rank-sum test. Based on these responses, we classified units into three categories of RFs:

  1. (i)

    Units with a focal foveal RF: These units responded solely to the cue in the foveal region (P < 0.05) but not to the search array that included items in the periphery (P > 0.05).

  2. (ii)

    Units with a broad foveal RF: These units responded to both the cue and the search array.

  3. (iii)

    Units with a peripheral RF: These units only responded to the search array (P < 0.05) but not to the cue (P > 0.05).

We implemented a visually guided saccade task to further map the peripheral RFs of units. In this task, we used the same faces and houses presented at the 20 predefined locations, identical to those in the visual search task. We compared responses within a time window of 50 to 200 ms after stimulus onset to the baseline (−150 to 0 ms relative to stimulus onset) using a two-tailed Wilcoxon rank-sum test to determine whether each unit had a significant response to stimuli in a specific peripheral RF ___location.

Category selectivity

We determined the category selectivity of each unit by comparing the response to face cues versus house cues in a time window of 50 to 200 ms after cue onset (Wilcoxon rank-sum test, P < 0.05). We further imposed a second criterion using a selectivity index similar to indices employed in previous IT studies25,26. For each unit with a foveal RF, the response to face stimuli (Rface) or house stimuli (Rhouse) was calculated using the visual search task by subtracting the mean baseline activity (−150 to 0 ms relative to the onset of the cue) from the mean response to the face or house cue (50 to 200 ms after the onset of the cue). For each unit with a peripheral RF, Rface and Rhouse were calculated using the visually guided saccade task by subtracting the mean baseline activity (−150 to 0 ms relative to the peripheral stimulus onset) from the mean response to the saccade target (50 to 200 ms after the onset of the saccade target). The selectivity index (SI) was then defined as (Rface − Rhouse) / (Rface + Rhouse). SI was set to 1 when Rface > 0 and Rhouse < 0, and to −1 when Rface < 0 and Rhouse > 0. Face-selective units were required to have an Rface at least 130% of Rhouse (i.e., the corresponding SI was greater than 0.13). Similarly, house-selective units were required to have an Rhouse at least 130% of Rface (i.e., the corresponding SI was smaller than −0.13). Units were labeled as non-category-selective if the response to face cues versus house cues was not significantly different (P > 0.05). The remaining units that did not fit into any of the aforementioned types were classified as undefined units (i.e., there was a significant difference but did not meet the second criterion).

Data Records

The dataset is publicly available on OSF (osf.io/sdgkr)27, including stimuli used in the experiments, behavioral data, events, neuronal spikes, local field potentials (LFPs), and preprocessing code. The data are organized into five folders: two folders contain data from the two monkeys, with RM006 for monkey S and RM008 for monkey E. The ‘spkLFPmatrix’ folder contains simultaneously recorded spikes and LFPs, with filenames indicating the monkey ID and brain region. The ‘imageset’ folder contains the stimuli used in the tasks, and the ‘code’ folder contains MATLAB scripts for data preprocessing (to analyze RF and category selectivity and store the information in the ‘cell list < region > ’ files, as described below) and for producing the figures (note that the scripts under ‘code for Distinct visual processing networks for foveal and peripheral visual fields’ are specifically for producing the figures in22).

The subject-specific directory (RM00 < label > ) contains the units (located in the ‘neurons’ directory) and the metadata for the units (located in the ‘cell list’ directory). Each file in the ‘neurons’ directory is stored in MATLAB ‘.mat’ format and contains variables related to trial events (‘TrlInfoMatrix’), search array stimuli (‘SearchSti’), saccade behavior (‘SearchEye’), and spike timing (‘neuron’) for all trials completed by the monkey. These files are organized by monkey, task, recording session, and unit, and can be identified by their filenames. For example, in the filename ‘m06cat003spk001a’, the first three characters represent the monkey (e.g., ‘m06’ for monkey S and ‘m08’ for monkey E), the next three characters represent the task (e.g., ‘cat’ for the category-based free-gaze visual search task and ‘det’ for the visually guided saccade task), the following three digits represent the session ID, and the final seven characters represent the unit ID. Each file in the ‘cell list’ directory is stored in ‘.txt’ format and contains metadata, including unit ID, recording site, and RF and category selectivity for each unit. These files are organized by brain areas and are named as ‘cell list < region >’, with five regions: V4, TEO, TE, OFC, and VPA (indicating LPFC). Note that we have separated IT into two subregions: TEO and TE.

The stimuli directory ‘imageset’ contains all the stimuli, with each file stored in ‘.bmp’ format and named with four digits. The first digit indicates the category of the stimulus, where 1, 2, 3, and 4 correspond to flower, house, monkey face, and monkey hand, respectively. The remaining three digits serve as unique identifiers for each image within its category.

Technical Validation

Behavior

The monkeys performed well on this task, with accuracy rates of 91.37% ± 2.93% (mean ± SD across sessions) and 92.25% ± 1.94% for finding faces and houses, respectively, for monkey S, and 86.55% ± 3.78% and 82.74% ± 3.61% for monkey E (Fig. 3a,b). The mean reaction time (RT), measured from the onset of the search array to the onset of the last fixation, was 462.04 ± 36.14 ms (mean ± SD across sessions) and 469.53 ± 38.98 ms for faces and houses, respectively, for monkey S, and 321.65 ± 24.48 ms and 368.53 ± 26.62 ms for monkey E (Fig. 3c,d).

Fig. 3
figure 3

Accuracy and reaction time. (a,b) Accuracy for face search and house search trials in two monkeys. On each box, the central mark is the median, the edges of the box are the 25th and 75th percentiles, the whiskers extend to the most extreme data points the algorithm considers to be not outliers, and the crosses denote the outliers. All box plots in this and subsequent figures follow the same convention. (c,d) Cumulative distribution of the reaction times (from the onset of the search array to the onset of the last fixation).

Full size image

Monkeys completed the search with free eye movements, either by fixating on multiple targets (Fig. 4a,b) or when they first spotted a target (Fig. 4c). The mean number of fixations when searching for faces and houses was 2.47 ± 1.46 (mean ± SD across trials) and 2.52 ± 1.50, respectively, for monkey S, and 1.80 ± 0.95 and 2.02 ± 1.05 for monkey E (Fig. 4d). It is worth noting that our search paradigm differs from many prior studies in that each trial included two targets among nine distractors. Under a serial search strategy, this configuration would typically require approximately four saccades, on average, to reach a target. However, both monkeys reached a target in about two saccades, suggesting the involvement of a parallel or partially parallel search mechanism. This finding aligns with the idea that, particularly in trained subjects, multiple items in the array may be processed simultaneously to guide efficient target selection. The mean fixation duration was 208.24 ± 153.77 ms (mean ± SD across fixations), with 216.93 ± 162.91 ms for monkey S and 184.27 ± 121.97 ms for monkey E (Fig. 4e). In trials with multiple fixations on targets, 57.91 ± 4.79% and 53.18 ± 4.94% (mean ± SD across sessions) of fixations immediately returned to the same target when searching for faces and houses, respectively, for monkey S, and 39.91 ± 7.90% and 38.85 ± 6.35% for monkey E (Fig. 4f). The percentage of fixations that eventually returned to the same target was 63.23 ± 4.29% and 59.22 ± 3.77% for faces and houses, respectively, for monkey S, and 41.89 ± 7.54% and 41.86 ± 6.26% for monkey E (Fig. 4g).

Fig. 4
figure 4

Eye movement during visual search. (a–c) Sample search trials for faces. The white trace indicates eye gazes. The red square indicates the position of the central fixation (shown during search). (d) Distribution of the number of fixations for two monkeys. (e) Cumulative distribution of fixation duration. (f) Percentage of target (T) fixations that were immediately revisited (with only one item in between; only the same target item was considered). (g) Percentage of target (T) fixations that were eventually revisited (with any number of items in between; only the same target item was considered).

Full size image

Firing rate

We recorded a total number of 6871 units from V4, 8641 units from IT (including both TE and TEO), 5622 units from the OFC, and 9916 units from the LPFC. 5070 units from V4, 5051 units from IT, 1470 units from the OFC, and 2997 units from the LPFC had a significant visually evoked response (i.e., the response to the cue or search array was significantly greater than the response to the baseline; Wilcoxon rank-sum test: P < 0.05). Among these visually responsive units, 1624 units from V4, 1419 units from IT, 888 units from the OFC, and 32 units from the LPFC had a focal foveal RF, while 781 units from V4, 268 units from IT, no units from the OFC, and 514 units from the LPFC had a localized peripheral RF.

Our trials involved presenting stimuli from different categories at various locations within the RFs, eliciting varying degrees of spiking activity—strong, weak, or none—depending on the cell’s RF and category selectivity. Consequently, the firing rate of units was expected to be higher for stimuli presented within their RFs and for stimuli belonging to their preferred category. Additionally, the response latency of units in the visual cortex was anticipated to be earlier than that in the prefrontal cortex (PFC). We plotted peri-stimulus time histograms (PSTHs) for stimuli presented at different locations and across various categories in both the visual cortex and PFC. Figure 5 and Fig. 6 illustrate these trends. A paired t-test revealed a significant difference in the average firing rate for the preferred ___location (Fig. 5; 43.37 ± 33.75 Hz vs. 28.91 ± 22.83 Hz; t(5896) = 62.4, P < 0.001). The identified foveal category-selective units from both monkeys differentiated fixations on faces and houses, showing higher firing rates for stimuli from the preferred category (Fig. 6a,b). Moreover, both foveal units (Fig. 6c,d) and peripheral units (Fig. 6e,f) distinguished target stimuli from distractors, demonstrating a feature attention effect. We characterized the RFs of peripheral units using the visually guided saccade task. As expected, the size of the peripheral RF (quantified by the number of stimuli that activated the unit; Fig. 7) in V4 (2.72 ± 2.03 [mean ± SD] stimuli) was smaller than that in IT (5.62 ± 3.07 stimuli; two-tailed two-sample t-test across units, t(1047) = 17.51, P = 1.95 × 10−60) and the LPFC (7.11 ± 2.95 stimuli; t(1293) = 31.75, P = 5.03 × 10−164).

Fig. 5
figure 5

Receptive field (RF) of units from the visual cortex and PFC. (a–d) Trials are aligned to the foveal cue onset for all stimuli. (e–h) Trials are aligned to the peripheral search array onset for all stimuli. The shaded area denotes ± SEM across units. The spike rate of each unit was normalized to its maximum spike rate across conditions.

Full size image
Fig. 6
figure 6

Category selectivity and feature attention effect in units with focal foveal or peripheral receptive fields (RFs). (a,b) Category selectivity in two monkeys. (c–f) Feature attention effect. The spike rate of each unit was normalized to its maximum spike rate across conditions.

Full size image
Fig. 7
figure 7

The RFs of peripheral units were mapped using the visually guided saccade task. (a,b) Histogram of the number of stimulus locations that activated the peripheral RF in two monkeys. (c,d) The aggregated tuning regions of all units with a peripheral RF. Color bars show the number of units with tuning regions in a given stimulus ___location. Recordings were made from the right side of the brain for both monkeys.

Full size image

Usage Notes

Our data are primarily organized by animal, with each monkey having its own dedicated folder. The code for data preprocessing and figure production can be found in the corresponding ‘code’ directory. For example, to define RFs, refer to ‘/code/data_preprocessing/RF*.m’; to define category selectivity, refer to ‘/code/category_selectivity*.m’. The files ‘cell list < region > .txt’ contain key information about each unit.