Abstract
Working memory (WM) is constructive in nature. Instead of passively retaining information, WM reorganizes complex sequences into hierarchically embedded chunks to overcome capacity limits and facilitate flexible behaviour. Here, to investigate the neural mechanisms underlying hierarchical reorganization in WM, we performed two electroencephalography and one magnetoencephalography experiments, wherein humans retain in WM a temporal sequence of items, that is, syllables, which are organized into chunks, that is, multisyllabic words. We demonstrate that the one-dimensional sequence is represented by two-dimensional neural representational geometry in WM arising from left prefrontal and temporoparietal regions, with separate dimensions encoding item position within a chunk and chunk position in the sequence. Critically, this two-dimensional geometry is observed consistently in different experimental settings, even during tasks not encouraging hierarchical reorganization in WM and correlates with WM behaviour. Overall, these findings strongly support that complex sequences are reorganized into factorized multidimensional neural representational geometry in WM, which also speaks to general structure-based organizational principles given WM’s involvement in many cognitive functions.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
27,99 € / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
118,99 € per year
only 9,92 € per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Data supporting main findings of the study are available at https://osf.io/drzuy/#.
Code availability
The code illustrating key analyses of the study can be found here https://osf.io/drzuy/#.
References
Schacter, D. L., Norman, K. A. & Koutstaal, W. The cognitive neuroscience of constructive memory. Annu. Rev. Psychol. 49, 289–318 (1998).
Brady, T. F. & Alvarez, G. A. Hierarchical encoding in visual working memory: ensemble statistics bias memory for individual items. Psychol. Sci. 22, 384–392 (2011).
Prabhakaran, V., Narayanan, K., Zhao, Z. & Gabrieli, J. D. E. Integration of diverse information in working memory within the frontal lobe. Nat. Neurosci. 3, 85–90 (2000).
van Ede, F. & Nobre, A. C. Turning attention inside out: how working memory serves behavior. Annu. Rev. Psychol. 74, 137–165 (2023).
Kwak, Y. & Curtis, C. E. Unveiling the abstract format of mnemonic representations. Neuron 110, 1822–1828 (2022).
Lee, S.-H., Kravitz, D. J. & Baker, C. I. Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat. Neurosci. 16, 997–999 (2013).
Ehrlich, D. B. & Murray, J. D. Geometry of neural computation unifies working memory and planning. Proc. Natl Acad. Sci. USA 119, e2115610119 (2022).
Gelastopoulos, A., Whittington, M. A. & Kopell, N. J. Parietal low beta rhythm provides a dynamical substrate for a working memory buffer. Proc. Natl Acad. Sci. USA 116, 16613–16620 (2019).
Caucheteux, C., Gramfort, A. & King, J.-R. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nat. Hum. Behav. 7, 430–441 (2023).
de Saussure, F. Course in General Linguistics (Columbia Univ. Press, 2011).
Georgopoulos, A. P. Higher order motor control. Annu. Rev. Neurosci. 14, 361–377 (1991).
Sarafyazd, M. & Jazayeri, M. Hierarchical reasoning by neural circuits in the frontal cortex. Science 364, eaav8911 (2019).
Balaguer, J., Spiers, H., Hassabis, D. & Summerfield, C. Neural mechanisms of hierarchical planning in a virtual subway network. Neuron 90, 893–903 (2016).
Koechlin, E. & Jubault, T. Broca’s area and the hierarchical organization of human behavior. Neuron 50, 963–974 (2006).
Thalmann, M., Souza, A. S. & Oberauer, K. How does chunking help working memory? J. Exp. Psychol. Learn. Mem. Cogn. 45, 37–55 (2019).
Farrell, S. Temporal clustering and sequencing in short-term memory and episodic memory. Psychol. Rev. 119, 223–271 (2012).
Brady, T. F., Konkle, T. & Alvarez, G. A. A review of visual memory capacity: beyond individual items and toward structured representations. J. Vis. 11, 4 (2011).
Geddes, C. E., Li, H. & Jin, X. Optogenetic editing reveals the hierarchical organization of learned action sequences. Cell 174, 32–43.e15 (2018).
Tomov, M. S., Yagati, S., Kumar, A., Yang, W. & Gershman, S. J. Discovery of hierarchical representations for efficient planning. PLoS Comput. Biol. 16, e1007594 (2020).
Wu, S., Élteto, N., Dasgupta, I. & Schulz, E. Learning structure from the ground up—hierarchical representation learning by chunking. Adv. Neural Inf. Process. Syst. 35, 36706–36721 (2022).
Ng, H. L. H. & Maybery, M. T. Grouping in short-term verbal memory: is position coded temporally? Q. J. Exp. Psychol. A 55, 391–424 (2002).
Ryan, J. Grouping and short-term memory: different means and patterns of grouping. Q. J. Exp. Psychol. 21, 137–147 (1969).
Ryan, J. Temporal grouping, rehearsal and short-term memory. Q. J. Exp. Psychol. 21, 148–155 (1969).
Brown, G. D. A., Preece, T. & Hulme, C. Oscillator-based memory for serial order. Psychol. Rev. 107, 127–181 (2000).
Hartley, T., Hurlstone, M. J. & Hitch, G. J. Effects of rhythm on memory for spoken sequences: a model and tests of its stimulus-driven mechanism. Cogn. Psychol. 87, 135–178 (2016).
Henson, R. N. A. Short-term memory for serial order: the start-end model. Cogn. Psychol. 36, 73–137 (1998).
Lee, C. L. & Estes, W. K. Item and order information in short-term memory: evidence for multilevel perturbation processes. J. Exp. Psychol. Hum. Learn. Mem. 7, 149–169 (1981).
Hurlstone, M. J. Functional similarities and differences between the coding of positional information in verbal and spatial short-term order memory. Memory 27, 147–162 (2019).
Behrens, T. E. J. et al. What is a cognitive map? Organizing knowledge for flexible behavior. Neuron 100, 490–509 (2018).
Whittington, J. C. R. et al. The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell 183, 1249–1263.e23 (2020).
Hitch, G. J. Temporal grouping effects in immediate recall: a working memory analysis. Q. J. Exp. Psychol. A 49, 116–139 (1996).
Burgess, N. & Hitch, G. J. A revised model of short-term memory and long-term learning of verbal sequences. J. Mem. Lang. 55, 627–652 (2006).
Liu, Y., Dolan, R. J., Kurth-Nelson, Z. & Behrens, T. E. J. Human replay spontaneously reorganizes experience. Cell 178, 640–652.e14 (2019).
Kornysheva, K. et al. Neural competitive queuing of ordinal structure underlies skilled sequential action. Neuron 101, 1166–1180 (2019).
Guidali, G., Pisoni, A., Bolognini, N. & Papagno, C. Keeping order in the brain: the supramarginal gyrus and serial order in short-term memory. Cortex 119, 89–99 (2019).
Zhou, J. et al. Evolving schema representations in orbitofrontal ensembles during learning. Nature 590, 606–611 (2021).
Xie, Y. et al. Geometry of sequence working memory in macaque prefrontal cortex. Science 375, 632–639 (2022).
Fan, Y., Han, Q., Guo, S. & Luo, H. Distinct neural representations of content and ordinal structure in auditory sequence memory. J. Neurosci. 41, 6290–6303 (2021).
Fan, Y. & Luo, H. Reactivating ordinal position information from auditory sequence memory in human brains. Cereb. Cortex 33, 5924–5936 (2023).
Gwilliams, L., King, J.-R., Marantz, A. & Poeppel, D. Neural dynamics of phoneme sequences reveal position-invariant code for content and order. Nat. Commun. 13, 6606 (2022).
Goldstein, A. et al. Shared computational principles for language processing in humans and deep language models. Nat. Neurosci. 25, 369–380 (2022).
Panichello, M. F. & Buschman, T. J. Shared mechanisms underlie the control of working memory and attention. Nature 592, 601–605 (2021).
Libby, A. & Buschman, T. J. Rotational dynamics reduce interference between sensory and memory representations. Nat. Neurosci. 24, 715–726 (2021).
Weber, J. et al. Subspace partitioning in the human prefrontal cortex resolves cognitive interference. Proc. Natl Acad. Sci. USA 120, e2220523120 (2023).
Navon, D. Forest before trees: the precedence of global features in visual perception. Cogn. Psychol. 9, 353–383 (1977).
Sanders, L. D. & Poeppel, D. Local and global auditory processing: behavioral and ERP evidence. Neuropsychologia 45, 1172–1186 (2007).
Fitch, W. T. & Martins, M. D. Hierarchical processing in music, language, and action: Lashley revisited. Ann. N. Y. Acad. Sci. 1316, 87–104 (2014).
Hasson, U., Chen, J. & Honey, C. J. Hierarchical process memory: memory as an integral component of information processing. Trends Cogn. Sci. 19, 304–313 (2015).
Khanna, A. R. et al. Single-neuronal elements of speech production in humans. Nature 626, 603–610 (2024).
Ding, N., Melloni, L., Zhang, H., Tian, X. & Poeppel, D. Cortical tracking of hierarchical linguistic structures in connected speech. Nat. Neurosci. 19, 158–164 (2016).
Henin, S. et al. Learning hierarchical sequence representations across human cortex and hippocampus. Sci. Adv. 7, eabc4530 (2021).
Badre, D. Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes. Trends Cogn. Sci. 12, 193–200 (2008).
Planton, S. et al. A theory of memory for binary sequences: evidence for a mental compression algorithm in humans. PLoS Comput. Biol. 17, e1008598 (2021).
D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol. 66, 115–142 (2015).
Frankish, C. Modality-specific grouping effects in short-term memory. J. Mem. Lang. 24, 200–209 (1985).
Frankish, C. Perceptual organization and precategorical acoustic storage. J. Exp. Psychol. Learn. Mem. Cogn. 15, 469–479 (1989).
Henson, R. N. A. Positional information in short-term memory: relative or absolute? Mem. Cogn. 27, 915–927 (1999).
Yokoi, A. & Diedrichsen, J. Neural organization of hierarchical motor sequence representations in the human neocortex. Neuron 103, 1178–1190 (2019).
Kikumoto, A. & Mayr, U. Decoding hierarchical control of sequential behavior in oscillatory EEG activity. eLife 7, e38550 (2018).
Flesch, T., Juechems, K., Dumbalska, T., Saxe, A. & Summerfield, C. Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron 110, 1258–1270 (2022).
Riggall, A. C. & Postle, B. R. The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging. J. Neurosci. 32, 12990–12998 (2012).
Ólafsdóttir, H. F., Bush, D. & Barry, C. The role of hippocampal replay in memory and planning. Curr. Biol. 28, R37–R50 (2018).
Grosmark, A. D., Sparks, F. T., Davis, M. J. & Losonczy, A. Reactivation predicts the consolidation of unbiased long-term cognitive maps. Nat. Neurosci. 24, 1574–1585 (2021).
Liu, Y., Nour, M. M., Schuck, N. W., Behrens, T. E. J. & Dolan, R. J. Decoding cognition from spontaneous neural activity. Nat. Rev. Neurosci. 23, 204–214 (2022).
Son, J.-Y., Vives, M.-L., Bhandari, A. & FeldmanHall, O. Replay shapes abstract cognitive maps for efficient social navigation. Nat. Hum. Behav. https://doi.org/10.1038/s41562-024-01990-w (2024).
Nassar, M. R., Helmers, J. C. & Frank, M. J. Chunking as a rational strategy for lossy data compression in visual working memory. Psychol. Rev. 125, 486–511 (2018).
Graybiel, A. M. The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70, 119–136 (1998).
Servan-Schreiber, E. & Anderson, J. R. Learning artificial grammars with competitive chunking. J. Exp. Psychol. Learn. Mem. Cogn. 16, 592–608 (1990).
Norris, D. & Kalm, K. Chunking and data compression in verbal short-term memory. Cognition 208, 104534 (2021).
Chase, W. G. & Simon, H. A. Perception in chess. Cogn. Psychol. 4, 55–81 (1973).
Miller, G. A. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63, 81–97 (1956).
Heusser, A. C., Poeppel, D., Ezzyat, Y. & Davachi, L. Episodic sequence memory is supported by a theta–gamma phase code. Nat. Neurosci. 19, 1374–1380 (2016).
Oberauer, K. Removing irrelevant information from working memory: a cognitive aging study with the modified Sternberg task. J. Exp. Psychol. Learn. Mem. Cogn. 27, 948–957 (2001).
Oberauer, K. Access to information in working memory: exploring the focus of attention. J. Exp. Psychol. Learn. Mem. Cogn. 28, 411–421 (2002).
Stokes, M. G. ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework. Trends Cogn. Sci. 19, 394–405 (2015).
Kandemir, G. & Akyürek, E. G. Impulse perturbation reveals cross-modal access to sensory working memory through learned associations. NeuroImage 274, 120156 (2023).
Wolff, M. J., Jochim, J., Akyürek, E. G. & Stokes, M. G. Dynamic hidden states underlying working-memory-guided behavior. Nat. Neurosci. 20, 864–871 (2017).
Sreenivasan, K. K., Curtis, C. E. & D’Esposito, M. Revisiting the role of persistent neural activity during working memory. Trends Cogn. Sci. 18, 82–89 (2014).
Lundqvist, M., Herman, P. & Miller, E. K. Working memory: delay activity, yes! Persistent activity? Maybe not. J. Neurosci. 38, 7013–7019 (2018).
Nelli, S., Braun, L., Dumbalska, T., Saxe, A. & Summerfield, C. Neural knowledge assembly in humans and neural networks. Neuron 111, 1504–1516 (2023).
Howard, M. W. & Kahana, M. J. A distributed representation of temporal context. J. Math. Psychol. 46, 269–299 (2002).
Polyn, S. M. & Kahana, M. J. Memory search and the neural representation of context. Trends Cogn. Sci. 12, 24–30 (2008).
Averbeck, B. B., Chafee, M. V., Crowe, D. A. & Georgopoulos, A. P. Parallel processing of serial movements in prefrontal cortex. Proc. Natl Acad. Sci. USA 99, 13172–13177 (2002).
Watanabe, K. & Funahashi, S. Prefrontal delay-period activity reflects the decision process of a saccade direction during a free-choice ODR task. Cereb. Cortex 17, i88–i100 (2007).
Van Ede, F., Chekroud, S. R., Stokes, M. G. & Nobre, A. C. Concurrent visual and motor selection during visual working memory guided action. Nat. Neurosci. 22, 477–483 (2019).
Kleiner, M., Brainard, D. & Pelli, D. What’s new in Psychtoolbox-3? Perception 36, 1–16 (2007).
Oostenveld, R., Fries, P., Maris, E. & Schoffelen, J.-M. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 156869 (2011).
Gramfort, A. et al. MEG and EEG data analysis with MNE-Python. Front. Neurosci. 7, 267 (2013).
Grootswagers, T., Wardle, S. G. & Carlson, T. A. Decoding dynamic brain patterns from evoked responses: a tutorial on multivariate pattern analysis applied to time series neuroimaging data. J. Cogn. Neurosci. 29, 677–697 (2017).
Wolff, M. J., Kandemir, G., Stokes, M. G. & Akyürek, E. G. Unimodal and bimodal access to sensory working memories by auditory and visual impulses. J. Neurosci. 40, 671–681 (2020).
Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164, 177–190 (2007).
Schaefer, A. et al. Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity MRI. Cereb. Cortex 28, 3095–3114 (2018).
Acknowledgements
This work was supported by the National Science and Technology Innovation 2030 Major Project 2021ZD0204100 (2021ZD0204103 to H.L. and 2021ZD0204105 to N.D.), the National Natural Science Foundation of China (31930052 to H.L.), the Science Fund for Creative Research Groups of the National Natural Science Foundation of China (T2421004 to F.F.), the China Postdoctoral Science Foundation (2023M740124 to Y.F.), the National Natural Science Foundation of China (32222035 to N.D.), the National Science and Technology Innovation 2030 Major Project (2022ZD0204802 to F.F.) and the National Natural Science Foundation of China (31930053 to F.F.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the paper. We thank the National Center for Protein Sciences and Center for MRI Research at Peking University in Beijing, China, for assistance with data acquisition. We would like to thank D. Liu, Q. Han and J. Gao for their help during the data collection, and M. Luo and J. Li for helpful support on data analysis.
Author information
Authors and Affiliations
Contributions
Y.F. and H.L. originally conceived and designed the experiments. Y.F. performed the experiments. Y.F. and M.W. analysed the data. F.F. and N.D. contributed to the experimental materials. Y.F., F.F., N.D. and H.L. wrote the paper.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Human Behaviour thanks Atsushi Kikumoto, Freek van Ede and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Fan, Y., Wang, M., Fang, F. et al. Two-dimensional neural geometry underpins hierarchical organization of sequence in human working memory. Nat Hum Behav 9, 360–375 (2025). https://doi.org/10.1038/s41562-024-02047-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-024-02047-8
