Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Mental graphs structure the storage and retrieval of visuomotor associations

Nature Human Behaviour (2025)Cite this article

Subjects

Abstract

Much of human memory takes the form of cognitive graphs that allow us to relate and generalize knowledge. The influence of structured memory in the motor system is less clear. Here we examine how structured memory representations influence action selection when responses are retrieved from newly learned, hierarchical visuomotor maps. Human participants (N = 182) learned visuomotor mappings with (or without) an imposed latent structure that linked visual stimulus features (for example, colour or shape) to intuitive motor distinctions, such as hands and pairs of fingers. In participants who learned structured visuomotor mappings, transitional response times indicated that retrieving the correct response from memory invoked the ‘traversal’ of a structured mental graph. Forced-response experiments revealed similar computations within individual trials. Moreover, graph-like representations persisted even after multiple days of practice with the visuomotor mappings. Our results point to direct links between internal computations over structured memory representations and the preparation of movements.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Task design and baseline correction.
Fig. 2: Theoretical models.
Fig. 3: Results for Experiments 1–3.
Fig. 4: Forced-response design and results.
Fig. 5: Experiment 6 results.

Similar content being viewed by others

Data availability

The data that support the findings of this study are available via GitHub at https://github.com/jetrach/StructuredActionPrepVMDM.

Code availability

The code used is available via GitHub at https://github.com/jetrach/StructuredActionPrepVMDM. The task code is available upon request. Please refer to the Methods for details on the software used in this project.

References

  1. Trach, J. E., McKim, T. H. & Desrochers, T. M. Abstract sequential task control is facilitated by practice and embedded motor sequences. J. Exp. Psychol. Learn. Mem. Cogn. 47, 1638–1659 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  2. Peer, M., Brunec, I. K., Newcombe, N. S. & Epstein, R. A. Structuring knowledge with cognitive maps and cognitive graphs. Trends Cogn. Sci. 25, 37–54 (2021).

    Article  PubMed  Google Scholar 

  3. Schapiro, A. C., Rogers, T. T., Cordova, N. I., Turk-Browne, N. B. & Botvinick, M. M. Neural representations of events arise from temporal community structure. Nat. Neurosci. 16, 486–492 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Schuck, N. W., Cai, M. B., Wilson, R. C. & Niv, Y. Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91, 1402–1412 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Kahn, A. E., Karuza, E. A., Vettel, J. M. & Bassett, D. S. Network constraints on learnability of probabilistic motor sequences. Nat. Hum. Behav. 2, 936–947 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Rosenbaum, D. A., Kenny, S. B. & Derr, M. A. Hierarchical control of rapid movement sequences. J. Exp. Psychol. Hum. Percept. Perform. 9, 86–102 (1983).

    Article  CAS  PubMed  Google Scholar 

  7. Cisek, P. & Pastor-Bernier, A. On the challenges and mechanisms of embodied decisions. Phil. Trans. R. Soc. B 369, 20130479 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Gallivan, J. P., Chapman, C. S., Wolpert, D. M. & Flanagan, J. R. Decision-making in sensorimotor control. Nat. Rev. Neurosci. 19, 519–534 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Gold, J. I. & Shadlen, M. N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).

    Article  CAS  PubMed  Google Scholar 

  10. Selen, L. P. J., Shadlen, M. N. & Wolpert, D. M. Deliberation in the motor system: reflex gains track evolving evidence leading to a decision. J. Neurosci. 32, 2276–2286 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Shadlen, M. N. & Kiani, R. Decision making as a window on cognition. Neuron 80, 791–806 (2013).

    Article  CAS  PubMed  Google Scholar 

  12. Song, J.-H. & Nakayama, K. Target selection in visual search as revealed by movement trajectories. Vis. Res. 48, 853–861 (2008).

    Article  PubMed  Google Scholar 

  13. Song, J.-H. & Nakayama, K. Numeric comparison in a visually-guided manual reaching task. Cognition 106, 994–1003 (2008).

    Article  PubMed  Google Scholar 

  14. Song, J.-H. & Nakayama, K. Hidden cognitive states revealed in choice reaching tasks. Trends Cogn. Sci. 13, 360–366 (2009).

    Article  PubMed  Google Scholar 

  15. Spivey, M. J. & Dale, R. Continuous dynamics in real-time cognition. Curr. Dir. Psychol. Sci. 15, 207–211 (2006).

    Article  Google Scholar 

  16. Spivey, M. J., Grosjean, M. & Knoblich, G. Continuous attraction toward phonological competitors. Proc. Natl Acad. Sci. USA 102, 10393–10398 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Ghez, C. et al. Discrete and continuous planning of hand movements and isometric force trajectories. Exp. Brain Res. 115, 217–233 (1997).

    Article  CAS  PubMed  Google Scholar 

  18. Hardwick, R. M., Forrence, A. D., Krakauer, J. W. & Haith, A. M. Time-dependent competition between goal-directed and habitual response preparation. Nat. Hum. Behav. 3, 1252–1262 (2019).

    Article  PubMed  Google Scholar 

  19. Mcdougle, S. D. & Taylor, J. A. Dissociable cognitive strategies for sensorimotor learning. Nat. Commun. 10, 40 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Constantinescu, A. O., O’Reilly, J. X. & Behrens, T. E. J. Organizing conceptual knowledge in humans with a gridlike code. Science 352, 1464–1467 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Musslick, S. & Bizyaeva, A. Examining cognitive flexibility and stability through the lens of dynamical systems. Curr. Opin. Behav. Sci. 57, 101375 (2024).

    Article  Google Scholar 

  22. Collins, A. G. E. & Frank, M. J. Motor demands constrain cognitive rule structures. PLoS Comput. Biol. https://doi.org/10.1371/journal.pcbi.1004785 (2016).

  23. Dykstra, T., Smith, D. M., Schumacher, E. H. & Hazeltine, E. Measuring task structure with transitional response times: task representations are more than task sets. Psychon. Bull. Rev. https://doi.org/10.3758/s13423-021-02035-3 (2022).

  24. Reitman, J. S. & Rueter, H. H. Organization revealed by recall orders and confirmed by pauses. Cogn. Psychol. 12, 554–581 (1980).

    Article  CAS  PubMed  Google Scholar 

  25. Maechler, M. R., Rousseeuw, P., Struyf, A., Hubert, M. & Hornik, K. cluster: Cluster analysis basics and extensions. R package version 2.1.3 (2022).

  26. Kassambara, A. & Mundt, F. factoextra: Extract and visualize the results of multivariate data analyses. R package version 1.0.7 (2020).

  27. Brown, T. I. et al. Prospective representation of navigational goals in the human hippocampus. Science 352, 1323–1326 (2016).

    Article  CAS  PubMed  Google Scholar 

  28. Rmus, M., Ritz, H., Hunter, L. E., Bornstein, A. M. & Shenhav, A. Humans can navigate complex graph structures acquired during latent learning. Cognition 225, 105103 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Tavares, R. M. et al. A map for social navigation in the human brain. Neuron 87, 231–243 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Karuza, E. A., Kahn, A. E., Thompson-Schill, S. L. & Bassett, D. S. Process reveals structure: how a network is traversed mediates expectations about its architecture. Sci. Rep. 7, 12733 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Rosenbaum, D. A. Human movement initiation: specification of arm, direction, and extent. J. Exp. Psychol. Gen. 109, 444–474 (1980).

    Article  CAS  PubMed  Google Scholar 

  32. Jaffe, P. I., Poldrack, R. A., Schafer, R. J. & Bissett, P. G. Modelling human behaviour in cognitive tasks with latent dynamical systems. Nat. Hum. Behav. https://doi.org/10.1038/s41562-022-01510-8 (2023).

  33. Balaguer, J., Spiers, H., Hassabis, D. & Summerfield, C. Neural mechanisms of hierarchical planning in a virtual subway network. Neuron 90, 893–903 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Ekstrom, A. D. & Ranganath, C. Space, time, and episodic memory: the hippocampus is all over the cognitive map. Hippocampus 28, 680–687 (2018).

    Article  PubMed  Google Scholar 

  35. Theves, S., Neville, D. A., Fernández, G. & Doeller, C. F. Learning and representation of hierarchical concepts in hippocampus and prefrontal cortex. J. Neurosci. 41, 7675–7686 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Whittington, J. C. R. et al. The Tolman–Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell 183, 1249–1263.e23 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Wu, C. M., Schulz, E., Garvert, M. M., Meder, B. & Schuck, N. W. Similarities and differences in spatial and non-spatial cognitive maps. PLoS Comput. Biol. 16, e1008149 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Zhou, J. et al. Complementary task structure representations in hippocampus and orbitofrontal cortex during an odor sequence task. Curr. Biol. 29, 3402–3409.e3 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Burk, D., Ingram, J. N., Franklin, D. W., Shadlen, M. N. & Wolpert, D. M. Motor effort alters changes of mind in sensorimotor decision making. PLoS ONE 9, e92681 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Cisek, P. & Kalaska, J. F. Simultaneous encoding of multiple potential reach directions in dorsal premotor cortex. J. Neurophysiol. 87, 1149–1154 (2002).

    Article  PubMed  Google Scholar 

  41. Hagura, N., Haggard, P. & Diedrichsen, J. Perceptual decisions are biased by the cost to act. eLife 6, e18422 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  42. Moher, J. & Song, J.-H. Perceptual decision processes flexibly adapt to avoid change-of-mind motor costs. J. Vis. 14, 1 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  43. Selen, L. P. J., Corneil, B. D. & Medendorp, W. P. Single-trial dynamics of competing reach plans in the human motor periphery. J. Neurosci. 43, 2782–2793 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Thura, D. & Cisek, P. Deliberation and commitment in the premotor and primary motor cortex during dynamic decision making. Neuron 81, 1401–1416 (2014).

    Article  CAS  PubMed  Google Scholar 

  45. Wong, A. L. & Haith, A. M. Motor planning flexibly optimizes performance under uncertainty about task goals. Nat. Commun. 8, 14624 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Freeman, J. B., Nakayama, K. & Ambady, N. Finger in flight reveals parallel categorization across multiple social dimensions. Soc. Cogn. 31, 792–805 (2013).

    Article  Google Scholar 

  47. Magnuson, J. S. Moving hand reveals dynamics of thought. Proc. Natl Acad. Sci. USA 102, 9995–9996 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Alhussein, L. & Smith, M. A. Motor planning under uncertainty. eLife 10, e67019 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Haith, A. M., Huberdeau, D. M. & Krakauer, J. W. Hedging your bets: intermediate movements as optimal behavior in the context of an incomplete decision. PLoS Comput. Biol. 11, e1004171 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  50. Miller, J. Discrete versus continuous stage models of human information processing: in search of partial output. J. Exp. Psychol. Hum. Percept. Perform. 8, 273–296 (1982).

    Article  CAS  PubMed  Google Scholar 

  51. Cellier, D., Petersen, I. T. & Hwang, K. Dynamics of hierarchical task representations. J. Neurosci. https://doi.org/10.1523/JNEUROSCI.0233-22.2022 (2022).

  52. Ranti, C., Chatham, C. H. & Badre, D. Parallel temporal dynamics in hierarchical cognitive control. Cognition 142, 205–229 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  53. Badre, D. & D’Esposito, M. Is the rostro-caudal axis of the frontal lobe hierarchical? Nat. Rev. Neurosci. 10, 659–669 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Badre, D. & Nee, D. E. Frontal cortex and the hierarchical control of behavior. Trends Cogn. Sci. 22, 170–188 (2018).

    Article  PubMed  Google Scholar 

  55. Logan, G. D. Toward an instance theory of automatization. Psychol. Rev. 95, 492–527 (1988).

    Article  Google Scholar 

  56. Logan, G. D. Automatic control: how experts act without thinking. Psychol. Rev. 125, 453–485 (2018).

    Article  PubMed  Google Scholar 

  57. Broadbent, D. E. & Broadbent, M. H. P. From detection to identification: response to multiple targets in rapid serial visual presentation. Percept. Psychophys. 42, 105–113 (1987).

    Article  CAS  PubMed  Google Scholar 

  58. McLean, J. P., Broadbent, D. E. & Broadbent, M. H. P. Combining attributes in rapid serial visual presentation tasks. Q. J. Exp. Psychol. https://doi.org/10.1080/14640748308402123 (1983).

  59. Treisman, A. Focused attention in the perception and retrieval of multidimensional stimuli. Percept. Psychophys. 22, 1–11 (1977).

    Article  Google Scholar 

  60. Behrens, T. E. J. et al. What is a cognitive map? Organizing knowledge for flexible behavior. Neuron 100, 490–509 (2018).

    Article  CAS  PubMed  Google Scholar 

  61. Chrastil, E. R. & Warren, W. H. From cognitive maps to cognitive graphs. PLoS ONE 9, e112544 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  62. de Leeuw, J. R., Gilbert, R. A. & Luchterhandt, B. jsPsych: Enabling an open-source collaborative ecosystem of behavioral experiments. J. Open Source Softw. 8, 5351 (2023).

    Article  Google Scholar 

  63. Eaton, J. W. GNU Octave and reproducible research. J. Process Control 22, 1433–1438 (2012).

    Article  CAS  Google Scholar 

  64. Brainard, D. H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).

    Article  CAS  PubMed  Google Scholar 

  65. R Core Team. R: A language and environment for statistical computing (R Foundation for Statistical Computing, 2022); https://www.R-project.org/

  66. MATLAB version 9.13.0 (R2022a) (The MathWorks Inc., 2022); https://www.mathworks.com

  67. Torchiano, M. effsize: Efficient effect size computation. R package version 0.8.1 (2020).

  68. McDougle, S. D. Post-error slowing during instrumental learning is shaped by working memory-based choice strategies. Neuroscience 486, 37–45 (2022).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank A. Forrence and T. Adanri in particular for their help with these projects. In addition, we acknowledge M. Ingram, J. Burde, K. Chou, S. Hu, L. Pandolpho and O. Pilkinton for essential support in collecting data. We also thank T. Desrochers and J. Taylor for feedback on early drafts of the manuscript, and the members of the ACT lab and Turk-Browne Lab at Yale University for productive conversations about this work throughout. J.E.T. is supported by the NSF GRFP. S.D.M. is supported by NIH grant no. R01 NS132926.

Author information

Authors and Affiliations

  1. Department of Psychology, Yale University, New Haven, CT, USA

    Juliana E. Trach & Samuel D. McDougle

  2. Wu Tsai Institute, Yale University, New Haven, CT, USA

    Samuel D. McDougle

Authors
  1. Juliana E. Trach
    View author publications

    Search author on:PubMed Google Scholar

  2. Samuel D. McDougle
    View author publications

    Search author on:PubMed Google Scholar

Contributions

J.E.T. and S.D.M. designed the paradigm. J.E.T. collected the data. J.E.T. and S.D.M. analysed the data. J.E.T. and S.D.M. prepared the figures and drafted, edited and revised the manuscript.

Corresponding author

Correspondence to Juliana E. Trach.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Aymar de Rugy, Elliot Hazeltine and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–10, Table 1 and Results.

Reporting Summary

Peer Review File

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Trach, J.E., McDougle, S.D. Mental graphs structure the storage and retrieval of visuomotor associations. Nat Hum Behav (2025). https://doi.org/10.1038/s41562-025-02217-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41562-025-02217-2

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing