Jiamu Jiang
Reinforcement learning when your life depends on it: a neuro-economic theory of learning
Jiang, Jiamu; Foyard, Emilie; van Rossum, Mark C.W.
Authors
Emilie Foyard
Professor MARK VAN ROSSUM Mark.VanRossum@nottingham.ac.uk
CHAIR AND DIRECTOR/NEURAL COMPUTATION RESEARCH GROUP
Abstract
Synaptic plasticity enables animals to adapt to their environment, but memory formation can consume a substantial amount of metabolic energy, potentially impairing survival. Hence, a neuro-economic dilemma arises whether learning is a profitable investment or not, and the brain must therefore judiciously regulate learning. Indeed, in experiments it was observed that during starvation, Drosophila suppress formation of energy-intensive aversive memories. Here we include energy considerations in a reinforcement learning framework. Simulated flies learned to avoid noxious stimuli through synaptic plasticity in either the energy expensive long-term memory (LTM) pathway, or the decaying anesthesia-resistant memory (ARM) pathway. The objective of the flies is to maximize their lifespan, which is calculated with a hazard function. We find that strategies that switch between the LTM and ARM pathways based on energy reserve and reward prediction error, prolong lifespan. Our study highlights the significance of energy-regulation of memory pathways and dopaminergic control for adaptive learning and survival. It might also benefit engineering applications of reinforcement learning under resources constraints.
Citation
Jiang, J., Foyard, E., & van Rossum, M. C. Reinforcement learning when your life depends on it: a neuro-economic theory of learning
Working Paper Type | Working Paper |
---|---|
Deposit Date | May 18, 2024 |
Publicly Available Date | May 21, 2024 |
Public URL | https://nottingham-repository.worktribe.com/output/34633846 |
Publisher URL | https://www.biorxiv.org/content/10.1101/2024.05.08.593165v1 |
Files
Reinforcement learning when your life depends on it
(1.1 Mb)
PDF
Publisher Licence URL
https://creativecommons.org/licenses/by-nc-nd/4.0/
You might also like
Energetically efficient learning in neuronal networks
(2023)
Journal Article
Competitive plasticity to reduce the energetic costs of learning
(2023)
Preprint / Working Paper
Lazy learning: a biologically-inspired plasticity rule for fast and energy efficient synaptic plasticity
(2023)
Preprint / Working Paper
Rule Abstraction Is Facilitated by Auditory Cuing in REM Sleep
(2023)
Journal Article
Estimating the energy requirements for long term memory formation
(2023)
Preprint / Working Paper
Downloadable Citations
About Repository@Nottingham
Administrator e-mail: discovery-access-systems@nottingham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search