Fig. 1: A model for the stepwise learning of behavioral sequences.

a Schematic of an example scenario where the reward is predicted by a simple sequence consisting of three behavioral events A-B-C. For example, A could be a light stimulus, B the rightward choice, and C the approach behavior to obtain reward. A1/A2/B1/B2 indicate alternative behavioral elements for A/B that could be combined to generate other behavioral sequences. Among all possible sequences, only the A-B-C sequence is rewarded. b The three distinct steps when learning the A-B-C sequence using the stepwise strategy illustrate how reward-seeking behaviors are sequentially refined. For simplicity, alternative behavioral events are denoted as non-A and non-B. Three possible behavioral sequences are listed. Behavioral events that have been learned as reward-predictors are colored in red, while the rewarded sequence (A-B-C) is underlined in red. b1 The first step of learning involves the event closest to the reward, C. Animals would engage in all three behavioral sequences because they all contain this reward-predicting event. b2 The second step of learning involves the next-to-last event, B. Behavioral sequences that contain the reward-predicting events B-C are preserved, while the incompatible sequence is eliminated from the behavioral repertoire (gray). b3 The third step of learning involves the earliest event, A. Only the A-B-C sequence contains all the reward-predicting events A-B-C and therefore preserved, while another incompatible sequence is eliminated from the behavioral repertoire.