Animals interact with a complex and unstable world and, by trial and error, learn to behave in ways that raise their chances for survival. Apparently, animals distil their experience into flexible rules that generalise to new circumstances. Advanced mammals and primates are particularly quick to learn flexible rules for behaviour and this allows them to extend their behavioural repertoire to unfamiliar tasks and conditions. Current artificial systems do not possess anything like this ability.
An important factor in this behavioural flexibility is context-dependent learning, which lets the animal adjust to changing task situations without lengthy re-learning (synonymous terms are 'model-based reversal learning', 'context-dependent conditioning', and 'occasion setting', Daw et al., , 2004; Schmajuk, Holland, 2002). Neurophysiological studies of context-dependent learning in behaving primates focus on prefrontal cortex, which combines information about sensory stimuli, motor actions, rewards received or expected, and current goals [Miller & Cohen, Annu Rev Neurosci 24: 167-202, 2001]. Imaging studies of context-dependent learning in humans confirm the importance of prefrontal cortex [Crone et al., Cereb Cortex 16: 475-86, 2005; Brass et al., Trends Cogn Sci 9: 314-6, 2005; Koechlin et al., Science 302: 1181-5].
Associative learning with biologically realistic networks can been modelled with the theory of dynamical stochastic system [Amit & Brunel, Cereb Cortex 7: 237-52, 1997; Wang, J Neurosci 19: 9587-603, 1999; Compte et al., Neuron 38: 473-485]. The attractor framework also provides a plausible scenario for the formation context-dependent associations [Deco & Rolls, Cereb Cortex 15: 15-30, 2005; Machens et al., Science 307: 1121-4, 2005; Fusi, Rev Neurosci 14: 73-84, 2003; Fusi et al., Neuron 45: 599-611, 2005]. Biologically realistic attractor models (i.e., those with spike-timedependent plasticity) make a strong qualitative prediction for context-dependent learning: the learning of associations encompasses not only current inputs but also reverberant delay activity. This implies that learned associations will include the temporal sequence of input events, whether task-relevant or not. Indeed, learning of task-irrelevant sequence information is observed in behaving primates [Miyashita, Nature 335: 817-20, 1988; Yakovlev et al, Nat Neurosci 1: 310-7, 1998]. Temporal sequence is of surprising consequence also in visual procedural learning, where task-irrelevant sequence information guides attention, accelerates learning, and facilitates recall [Chun, Trends Cogn Sci, 4: 170-78, 2000; Jiang, Leung, Psychon. Bull Rev, 12: 100-6, 2005; Kuai et al., Nat Neurosci 8: 1497-9, 2006].
We hypothesize that temporal sequence plays a fundamental, and hitherto unrecognized, role in context-dependent learning. The incidental learning of temporal sequence information does not constitute an optimal decision strategy, as it gives unwarranted weight to irrelevant cues. However, incidental learning of consistent sequence information may represent a heuristic strategy suitable for natural learning scenarios, in which the relevance of environmental cues may change and hitherto irrelevant cues may suddenly become vitally important.