Reinforcement learning when your life depends on it: A neuro-economic theory of learning

Jiang, Jiamu; Foyardg, Emilie; van Rossumg, Mark C. W.

Abstract

Synaptic plasticity enables animals to adapt to their environment, but memory formation can require a substantial amount of metabolic energy, potentially impairing survival. Hence, a neuro-economic dilemma arises whether learning is a profitable investment or not, and the brain must therefore judiciously regulate learning. Indeed, in experiments it was observed that during starvation, Drosophila suppress formation of energy-intensive aversive memories. Here we include energy considerations in a reinforcement learning framework. Simulated flies learned to avoid noxious stimuli through synaptic plasticity in either the energy expensive long-term memory (LTM) pathway, or the decaying anesthesia-resistant memory (ARM) pathway. The objective of the flies is to maximize their lifespan, which is calculated with a hazard function. We find that strategies that switch between the LTM and ARM pathways, based on energy reserve and reward prediction error, prolong lifespan. Our study highlights the significance of energy-regulation of memory pathways and dopaminergic control for adaptive learning and survival. It might also benefit engineering applications of reinforcement learning under resources constraints.

Citation

Jiang, J., Foyardg, E., & van Rossumg, M. C. W. (in press). Reinforcement learning when your life depends on it: A neuro-economic theory of learning. PLoS Computational Biology, 20(10), e1012554. https://doi.org/10.1371/journal.pcbi.1012554

• Journal Article Type: Article
• Acceptance Date: Oct 14, 2024
• Online Publication Date: Oct 28, 2024
• Deposit Date: Mar 17, 2025
• Publicly Available Date: Mar 19, 2025
• Journal: PLOS Computational Biology
• Print ISSN: 1553-734X
• Electronic ISSN: 1553-7358
• Publisher: Public Library of Science
• Peer Reviewed: Peer Reviewed
• Volume: 20
• Issue: 10
• Pages: e1012554
• DOI: https://doi.org/10.1371/journal.pcbi.1012554
• Public URL: https://nottingham-repository.worktribe.com/output/41369549
• Publisher URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012554