Skip to main content

Research Repository

Advanced Search

Reinforcement learning when your life depends on it: a neuro-economic theory of learning

Jiang, Jiamu; Foyard, Emilie; van Rossum, Mark C.W.

Reinforcement learning when your life depends on it: a neuro-economic theory of learning Thumbnail


Authors

Jiamu Jiang

Emilie Foyard



Abstract

Synaptic plasticity enables animals to adapt to their environment, but memory formation can consume a substantial amount of metabolic energy, potentially impairing survival. Hence, a neuro-economic dilemma arises whether learning is a profitable investment or not, and the brain must therefore judiciously regulate learning. Indeed, in experiments it was observed that during starvation, Drosophila suppress formation of energy-intensive aversive memories. Here we include energy considerations in a reinforcement learning framework. Simulated flies learned to avoid noxious stimuli through synaptic plasticity in either the energy expensive long-term memory (LTM) pathway, or the decaying anesthesia-resistant memory (ARM) pathway. The objective of the flies is to maximize their lifespan, which is calculated with a hazard function. We find that strategies that switch between the LTM and ARM pathways based on energy reserve and reward prediction error, prolong lifespan. Our study highlights the significance of energy-regulation of memory pathways and dopaminergic control for adaptive learning and survival. It might also benefit engineering applications of reinforcement learning under resources constraints.

Citation

Jiang, J., Foyard, E., & van Rossum, M. C. Reinforcement learning when your life depends on it: a neuro-economic theory of learning

Working Paper Type Working Paper
Deposit Date May 18, 2024
Publicly Available Date May 21, 2024
Public URL https://nottingham-repository.worktribe.com/output/34633846
Publisher URL https://www.biorxiv.org/content/10.1101/2024.05.08.593165v1

Files





You might also like



Downloadable Citations