FiDRL: Flexible Invocation-Based Deep Reinforcement Learning for DVFS Scheduling in Embedded Systems

Li, Jingjin; Jiang, Weixiong; He, Yuting; Yang, Qingyu; Gao, Anqi; Ha, Yajun; Özcan, Ender; Bai, Ruibin; Cui, Tianxiang; Yu, Heng

Abstract

Deep Reinforcement Learning (DRL)-based Dynamic Voltage Frequency Scaling (DVFS) has shown great promise for energy conservation in embedded systems. While many works were devoted to validating its efficacy or improving its performance, few discuss the feasibility of the DRL agent deployment for embedded computing. State-of-the-art approaches focus on the miniaturization of agents' inferential networks, such as pruning and quantization, to minimize their energy and resource consumption. However, this spatial-based paradigm still proves inadequate for resource-stringent systems. In this paper, we address the feasibility from a temporal perspective, where FiDRL, a flexible invocation-based DRL model is proposed to judiciously invoke itself to minimize the overall system energy consumption, given that the DRL agent incurs non-negligible energy overhead during invocations. Our approach is threefold: (1) FiDRL that extends DRL by incorporating the agent's invocation interval into the action space to achieve invocation flexibility; (2) a FiDRL-based DVFS approach for both inter-and intra-task scheduling that minimizes the overall execution energy consumption; and (3) a FiDRL-based DVFS platform design and an on/off-chip hybrid algorithm specialized for training the DRL agent for embedded systems. Experiment results show that FiDRL achieves 55.1% agent invocation cost reduction, under 23.3% overall energy reduction, compared to state-of-the-art approaches.

Citation

Li, J., Jiang, W., He, Y., Yang, Q., Gao, A., Ha, Y., Özcan, E., Bai, R., Cui, T., & Yu, H. (2024). FiDRL: Flexible Invocation-Based Deep Reinforcement Learning for DVFS Scheduling in Embedded Systems. IEEE Transactions on Computers, https://doi.org/10.1109/TC.2024.3465933

• Journal Article Type: Article
• Acceptance Date: Sep 8, 2024
• Online Publication Date: Sep 23, 2024
• Publication Date: Sep 23, 2024
• Deposit Date: Oct 9, 2024
• Journal: IEEE Transactions on Computers
• Print ISSN: 0018-9340
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1109/TC.2024.3465933
• Public URL: https://nottingham-repository.worktribe.com/output/40290418
• Publisher URL: https://ieeexplore.ieee.org/document/10689358