RLMan: An Energy Manager Based on Reinforcement Learning for Energy Harvesting Wireless Sensor Networks
IEEE Transactions on Green Communications and NetworkingPublished 2 February 2018Open access
Fayçal Ait Aoudia, Matthieu Gautier, Olivier Berder
Citations116
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Exhaustive simulations using real measurements of indoor light and outdoor wind show that RLMan outperforms current state-of-the-art approaches, by enabling almost 70% gain regarding the average packet rate and is more robust to variability of the node energy consumption.
Abstract
International audience
Keywords
Engineering
Policy Gradient Methods for Reinforcement Learning with Function Approximation
4,982 Citations1999Richard S. Sutton, David McAllester +2 more
This paper proves for the first time that a version of policy iteration with arbitrary differentiable function approximation is convergent to a locally optimal policy.
Machine LearningLearning to Predict by the Methods of Temporal Differences
3,894 Citations1988Richard S. Sutton
This article introduces a class of incremental learning procedures specialized for prediction-that is, for using past experience with an incompletely known system to predict its future behavior, and proves their convergence and optimality for special cases and relate them to supervised-learning methods.
Machine LearningLearning to predict by the methods of temporal differences
2,758 Citations1988Richard S. Sutton
ACM Transactions on Embedded Computing SystemsPower management in energy harvesting sensor networks
1,537 Citations2007Aman Kansal, Jason C. Hsu +2 more
IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews)A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
1,005 Citations2012I. Grondman, Lucian Buşoniu +2 more
The workings of the natural gradient is described, which has made its way into many actor-critic algorithms over the past few years, and a review of several standard and natural actor-critic algorithms is given.
IEEE Solid-State Circuits MagazineEnergy Harvesting for Autonomous Wireless Sensor Networks
611 Citations2010Rudd Vullers, R. van Schaijk +3 more
The typical power requirements of some current portable devices, including a body sensor network, are shown in Figure 1.
Adaptive Control of Duty Cycling in Energy-Harvesting Wireless Sensor Networks
426 Citations2007Christopher M. Vigorito, Deepak Ganesan +1 more
A new technique for solving the problem of adaptive duty-cycling based on results from adaptive control theory is introduced and it is shown that it achieves better performance than previous approaches on a broader class of energy source data sets.
IEEE Transactions on Systems Man and Cybernetics SystemsMultiobjective Reinforcement Learning: A Comprehensive Overview
359 Citations2014Chunming Liu, Xin Xu +1 more
The basic architecture, research topics, and naïve solutions of MORL are introduced at first and several representative MORL approaches and some important directions of recent research are comprehensively reviewed.
Networking low-power energy harvesting devices: Measurements and algorithms
129 Citations2011Maria Gorlatova, Aya Wallwater +1 more
IEEE Transactions on Green Communications and NetworkingReinforcement Learning for Energy Harvesting Decode-and-Forward Two-Hop Communications
59 Citations2017Andrea Ortiz, Hussein Al-Shatri +3 more
The proposed approach has only a small degradation as compared to the offline optimum case and with the use of the proposed feature functions a better performance is achieved compared to standard approximation techniques.
IEEE Transactions on Emerging Topics in ComputingA Reinforcement Learning-Based ToD Provisioning Dynamic Power Management for Sustainable Operation of Energy Harvesting Wireless Sensor Node
56 Citations2014Roy Chaoming Hsu, Cheng-Ting Liu +1 more
Experimental results demonstrate that the achieved demanded throughput is improved 10.7% for the most stringent ToD requirement, while the residual battery energy of the RLTDPM is improved 7.4% compared with an existing DPM algorithm for EHWSN with image sensing purpose.
EURASIP Journal on Embedded SystemsA framework for modeling and simulating energy harvesting WSN nodes with efficient power management policies
55 Citations2012Andréa Castagnetti, Alain Pégatoquet +2 more
This article proposes a framework that permits to describe and simulate an energy harvesting sensor node by using a high level modeling approach based on power consumption and energy harvesting, and shows that the throughput of a sensor node can be improved up to 50% when compared to a state of the art power management algorithm for solar harvesting WSN.
ACM Transactions on Sensor NetworksEnergy-Harvesting Active Networked Tags (EnHANTs)
53 Citations2015Robert Margolies, Maria Gorlatova +10 more
A small-scale testbed is described that uniquely allows evaluating different algorithms with trace-based light energy inputs and experimentally evaluates the performance of different energy-harvesting adaptive policies with organic solar cells and UWB-IR transceivers.
IEEE Sensors JournalEnergy-Efficient Power Manager and MAC Protocol for Multi-Hop Wireless Sensor Networks Powered by Periodic Energy Harvesting Sources
48 Citations2015Trong Nhan Le, Alain Pégatoquet +2 more
A wake-up variation reduction PM is proposed, which not only follows the ENO condition, but also reduces thewake-up interval variations of WSN nodes, and an energy-efficient protocol, named synchronized wake- up interval MAC, is also proposed.
Ad Hoc NetworksPrediction free energy neutral power management for energy harvesting wireless sensor nodes
40 Citations2013Shuai Peng, Chor Ping Low
A fundamental framework to efficiently manage the harvested energy in a prediction free manner is proposed and theoretically derive a set of Budget Assigning Principles (BAPs) to maximize the amount of harvested energy that can be utilized by a sensor in the presence of battery energy storage inefficiencies.
Fuzzy power management for energy harvesting Wireless Sensor Nodes
33 Citations2016Fayçal Ait Aoudia, Matthieu Gautier +1 more
This work presents Fuzzyman, a novel PM based on fuzzy control theory that achieves more efficient utilization of the harvested energy and is evaluated by comparing it to a state of the art approach via extensive trace-driven network simulations.
Balancing Multiple Sources of Reward in Reinforcement Learning
31 Citations2006Christian R. Shelton
The multiple reward source problem is described and an new algorithm for finding a solution is presented and results on simulated environments are presented.
GRAPMAN: Gradual power manager for consistent throughput of energy harvesting wireless sensor nodes
14 Citations2015Fayçal Ait Aoudia, Matthieu Gautier +1 more
GRAPMAN (GRAdual Power MANager) is proposed, a novel PM for Energy-Harvesting WSN (EH-WSN) powered by pseudo-periodic energy sources that outperforms state of the art PMs in both average throughput and throughput consistency.