THE LEARNING SIMULATOR
FUNDAMENTAL RESEARCH
We use the simulator in fundamental research to better understand learning phenomena in animals and humans. We investigated the question if we can see animal behaviours where animals learn beyond associative learning. What we find, is that associative learning is a very powerful way to acquire behavioural sequences. On this page, you will find our published work on these topics, including summaries, videos and links to papers.
November 3, 2016
THE POWER OF ASSOCIATIVE LEARNING AND THE ONTOGONY OF OPTIMAL BEHAVIOUR
Behaving efficiently (optimally or near-optimally) is central to animals’ adaptation to their environment. Much evolutionary biology assumes, implicitly or explicitly, that optimal behavioural strategies are genetically inherited, yet the behaviour of many animals depends crucially on learning. The question of how learning contributes to optimal behaviour is largely open. Here we propose an associative learning model that can learn optimal behaviour in a wide variety of ecologically relevant circumstances. The model learns through chaining, a term introduced by Skinner to indicate learning of behaviour sequences by linking together shorter sequences or single behaviours. Our model formalizes the concept of conditioned reinforcement (the learning process that underlies chaining) and is closely related to optimization algorithms from machine learning. Our analysis dispels the common belief that associative learning is too limited to produce ‘intelligent’ behaviour such as tool use, social learning, self- control or expectations of the future. Furthermore, the model readily accounts for both instinctual and learned aspects of behaviour, clarifying how genetic evolution and individual learning complement each other, and bridging a long-standing divide between ethology and psychology. We conclude that associative learning, supported by genetic predispositions and including the oft-neglected phenomenon of conditioned reinforcement, may suffice to explain the ontogeny of optimal behaviour in most, if not all, non-human animals. Our results establish associative learning as a more powerful optimizing mechanism than acknowledged by current opinion.
Enquist, M., Lind, J., & Ghirlanda, S. (2016). The power of associative learning and the ontogeny of optimal behaviour. Royal Society Open Science, 3(11), 160734.
Accepted for Publication: April 29, 2020
A-LEARNING: A NEW FORMULATION OF ASSOCIATIVE LEARNING THEORY
We present a new mathematical formulation of associative learning, focused on non-human animals and )called A-learning. Building on animal learning theory and machine learning, A-learning comprises two learning equations, one for stimulus-response values and one for stimulus values (conditioned reinforcement), plus a decision-making equation based on the matching law. We show that A-learning can reproduce: instrumental acquisition, including the effects of signalled and unsignaled non-contingent reinforcement; Pavlovian acquisition, including omission training, and differences between conditioned and unconditioned responses; acquisition and extinction of instrumental chains and Pavlovian higher-order conditioning; the partial reinforcement effect; Pavlovian-to-instrumental transfer; outcome revaluation effects, including insight into why these effects vary with training procedures and with the proximity of a response to the reinforcer. We discuss the differences between A-learning and current theory and compare it with similar models from machine learning, such as Q-learning and the actor-critic model. A-learning may offer a more convenient view of associative learning that unifies instrumental and Pavlovian learning.
Stefano Ghirlanda, Johan Lind & Magnus Enquist (2020): A-learning: A new formulation of associative learning theory. Psychonomic Bulletin & Review, 1-29.
November 28, 2018
WHAT CAN ASSOCIATIVE LEARNING DO FOR PLANNING?
There is a new associative learning paradox. The power of associative learning for producing flexible behaviour in non-human animals is downplayed or ignored by researchers in animal cognition, whereas artificial intelligence research shows that associative learning models can beat humans in chess. One phenomenon in which associative learning often is ruled out as an explanation for animal behaviour is flexible planning. However, planning studies have been criticized and questions have been raised regarding both methodological validity and interpretations of results. Due to the power of associative learning and the uncertainty of what causes planning behaviour in non-human animals, I explored what associative learning can do for planning. A previously published sequence learning model which combines Pavlovian and instrumental conditioning was used to simulate two planning studies, namely Mulcahy & Call 2006 ‘Apes save tools for future use.’ Science312, 1038–1040 and Kabadayi & Osvath 2017 ‘Ravens parallel great apes in flexible planning for tool-use and bartering.’ Science357, 202–204. Simulations show that behaviour matching current definitions of flexible planning can emerge through associative learning. Through conditioned reinforcement, the learning model gives rise to planning behaviour by learning that a behaviour towards a current stimulus will produce high value food at a later stage; it can make decisions about future states not within current sensory scope. The simulations tracked key patterns both between and within studies. It is concluded that one cannot rule out that these studies of flexible planning in apes and corvids can be completely accounted for by associative learning. Future empirical studies of flexible planning in non-human animals can benefit from theoretical developments within artificial intelligence and animal learning.
Lind, J. (2018). What can associative learning do for planning? Royal Society Open Science, 5(180778).
December 21, 2018
VIDEO: GREAT APES AND RAVENS PLAN WITHOUT THINKING
Planning and self-control in animals do not require human-like mental capacities, according to a study from Stockholm University. Newly developed learning models, similar to models within artificial intelligence research, show how planning in ravens and great apes can develop through prior experiences without any need of thinking.
March 13, 2019
SOCIAL LEARNING THROUGH ASSOCIATIVE PROCESSES: A COMPUTATIONAL THEORY
Associative mechanisms can result in transfer of information and behaviour from experienced to naive individuals. A new model and results from the Centre for Cultural Evolution show that associative processes supported by genetic predisposition can account for most types of social learning processes found in nonhuman animals. Social transmission of information is a key phenomenon in the evolution of behavior and in the establishment of traditions and culture. To this day, a plethora of social learning processes have been described. Here, we used a computational formulation of associative learning, taking instinctual aspects of behavior into account, to analyze social learning in nonhuman animals. Our results show that associative processes supported by genetic predisposition can account for most types of social learning processes found in nonhuman animals. We conclude that combining associative learning with instinctual aspects of behavior provides a unitary framework for the study of social learning in nonhuman animals.
Johan Lind, Stefano Ghirlanda & Magnus Enquist (2019): Social learning through associative processes: a computational theory. Royal Society Open Science (6), 181777.