A Self-Adaptive Sleep/Wake-Up Scheduling Approach for Wireless Sensor Networks
IEEE Transactions on CyberneticsPublished 3 March 2017
Dayong Ye, Minjie Zhang
Citations132
SJR quartileQ1
SJR score4.45
SNIP3.22
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The proposed approach, based on the reinforcement learning technique, enables each node to autonomously decide its own operation mode (sleep, listen, or transmission) in each time slot in a decentralized manner.
Abstract
Sleep/wake-up scheduling is one of the fundamental problems in wireless sensor networks, since the energy of sensor nodes is limited and they are usually unrechargeable. The purpose of sleep/wake-up scheduling is to save the energy of each node by keeping nodes in sleep mode as long as possible (without sacrificing packet delivery efficiency) and thereby maximizing their lifetime. In this paper, a self-adaptive sleep/wake-up scheduling approach is proposed. Unlike most existing studies that use the duty cycling technique, which incurs a tradeoff between packet delivery delay and energy saving, the proposed approach, which does not us duty cycling, avoids such a tradeoff. The proposed approach, based on the reinforcement learning technique, enables each node to autonomously decide its own operation mode (sleep, listen, or transmission) in each time slot in a decentralized manner. Simulation results demonstrate the good performance of the proposed approach in various circumstances.
Keywords
Computer ScienceEngineering
Machine LearningQ-learning
8,970 Citations1992Christopher J. Watkins, Peter Dayan
This paper presents and proves in detail a convergence theorem forQ-learning based on that outlined in Watkins (1989), showing that Q-learning converges to the optimum action-values with probability 1 so long as all actions are repeatedly sampled in all states and the action- values are represented discretely.
Journal of Artificial Intelligence ResearchReinforcement Learning: A Survey
8,831 Citations1996Leslie Pack Kaelbling, Michael L. Littman +1 more
Central issues of reinforcement learning are discussed, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state.
Versatile low power media access for wireless sensor networks
3,356 Citations2004Joseph Polastre, Jason Hill +1 more
B-MAC's flexibility results in better packet delivery rates, throughput, latency, and energy consumption than S-MAC, and the need for flexible protocols to effectively realize energy efficient sensor network applications is illustrated.
IEEE/ACM Transactions on NetworkingMedium Access Control With Coordinated Adaptive Sleeping for Wireless Sensor Networks
2,737 Citations2004W. Ye, John Heidemann +1 more
This paper proposes S-MAC, a medium access control (MAC) protocol designed for wireless sensor networks that enables low-duty-cycle operation in a multihop network and reveals fundamental tradeoffs on energy, latency and throughput.
An adaptive energy-efficient MAC protocol for wireless sensor networks
2,664 Citations2003Tijs van Dam, Koen Langendoen
T-MAC, a contention-based Medium Access Control protocol for wireless sensor networks, introduces an adaptive duty cycle in a novel way: by dynamically ending the active part of it to handle load variations in time and location.
NetworksA survey of gossiping and broadcasting in communication networks
1,245 Citations1988Sandra M. Hedetniemi, Stephen T. Hedetniemi +1 more
Gossiping and broadcasting are two problems of information dissemination described for a group of individuals connected by a communication network that one individual has an item of information which needs to be communicated to everyone else.
An adaptive energy-efficient MAC protocol for wireless sensor networks
897 Citations2003Tijs van Dam, Koen Langendoen
Artificial IntelligenceMultiagent learning using a variable learning rate
786 Citations2002Michael Bowling, Manuela Veloso
This article introduces the WoLF principle, “Win or Learn Fast”, for varying the learning rate, and examines this technique theoretically, proving convergence in self-play on a restricted class of iterated matrix games.
RI-MAC
692 Citations2008Yanjun Sun, Omer Gurewitz +1 more
This paper presents a new asynchronous duty cycle MAC protocol, called Receiver-Initiated MAC (RI-MAC), that uses receiver-initiated data transmission in order to efficiently and effectively operate over a wide range of traffic loads.
Machine LearningConvergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms
613 Citations2000Satinder Singh, Tommi Jaakkola +2 more
This paper examines the convergence of single-step on-policy RL algorithms for control with both decaying exploration and persistent exploration and provides examples of exploration strategies that result in convergence to both optimal values and optimal policies.
Proceedings of the IEEEA Survey on Ambient Intelligence in Healthcare
609 Citations2013Giovanni Acampora, Diane J. Cook +2 more
The state-of-the-art artificial intelligence (AI) methodologies used for developing AmI system in the healthcare domain are summarized, including various learning techniques (for learning from user interaction), reasoning techniques ( for reasoning about users' goals and intensions), and planning techniques (For planning activities and interactions).
Ultra-low duty cycle MAC with scheduled channel polling
593 Citations2006Wei Ye, Fábio Silva +1 more
This work establishes optimal configurations for both LPL and SCP under fixed conditions, developing a lower bound of energy consumption and shows how SCP adapts to heavy traffic and streams data in multi-hop networks, reducing latency by 85% and energy by 95% at 9 hops.
Asynchronous wakeup for ad hoc networks
437 Citations2003Rong Zheng, Jennifer C. Hou +1 more
Simulation studies indicate that the proposed asynchronous wakeup protocol is quite effective under various traffic characteristics and loads: energy saving can be as high as 70%, while the packet delivery ratio is comparable to that without power management.
Applied Soft ComputingDistributed evolutionary algorithms and their models: A survey of the state-of-the-art
376 Citations2015Yue‐Jiao Gong, Wei–Neng Chen +5 more
A comprehensive survey of the state-of-the-art distributed evolutionary algorithms and models, which have been classified into two groups according to their task division mechanism, and insights into the models are presented and discussed.
IEEE/ACM Transactions on NetworkingEDAL: An Energy-Efficient, Delay-Aware, and Lifetime-Balancing Data Collection Protocol for Heterogeneous Wireless Sensor Networks
348 Citations2014Yanjun Yao, Qing Cao +1 more
This work develops one data collection protocol called EDAL, which stands for Energy-efficient Delay-aware Lifetime-balancing data collection, and proposes both a centralized heuristic to reduce its computational overhead and a distributed heuristics to make the algorithm scalable for large-scale network operations.
Journal of Artificial Intelligence ResearchEvolutionary Algorithms for Reinforcement Learning
343 Citations1999David E. Moriarty, Anna Charlotte Schultz +1 more
Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications.
Proceedings of the IEEEA Survey on Topology Control in Wireless Sensor Networks: Taxonomy, Comparative Study, and Open Issues
333 Citations2013Mo Li, Zhenjiang Li +1 more
IEEE Transactions on Mobile ComputingA MAC protocol to reduce sensor network energy consumption using a wakeup radio
331 Citations2005Matthew J. Miller, Nitin H. Vaidya
Information Processing in Sensor NetworksGradient clock synchronization in wireless sensor networks
312 Citations2009Philipp Sommer, Roger Wattenhofer
The Gradient Time Synchronization Protocol (GTSP) is proposed which is designed to provide accurately synchronized clocks between neighbors and works in a completely decentralized fashion: Every node periodically broadcasts its time information.
PW-MAC: An energy-efficient predictive-wakeup MAC protocol for wireless sensor networks
306 Citations2011Lei Tang, Yanjun Sun +2 more
Predictive-Wakeup MAC is presented, a new energy-efficient MAC protocol based on asynchronous duty cycling that minimizes sensor node energy consumption by enabling senders to predict receiver wakeup times and introduces an on-demand prediction error correction mechanism that effectively addresses timing challenges such as unpredictable hardware and operating system delays and clock drift.
DW-MAC
293 Citations2008Yanjun Sun, Shu Du +2 more
This paper presents a new MAC protocol, called Demand Wakeup MAC (DW-MAC), that introduces a new low-overhead scheduling algorithm that allows nodes to wake up on demand during the Sleep period of an operational cycle and ensures that data transmissions do not collide at their intended receivers.
Wakeup scheduling in wireless sensor networks
292 Citations2006A. Keshavarzian, Huang Lee +1 more
This work considers the design of efficient wakeup scheduling schemes for energy constrained sensor nodes that adhere to the bidirectional end-to-end delay constraints posed by such applications, and proposes novel scheduling methods that outperform existing ones.
IEEE Wireless CommunicationsWireless sensor networks with energy harvesting technologies: a game-theoretic approach to optimal energy management
258 Citations2007Dusit Niyato, Ekram Hossain +2 more
This article presents an optimal energy management policy for a solar-powered sensor node that uses a sleep and wakeup strategy for energy conservation and uses the Nash equilibrium as the solution of this game.
Nash convergence of gradient dynamics in general-sum games
253 Citations2000Satinder Singh, Michael Kearns +1 more
This work analyzes the behavior of agents that incrementally adapt their strategy through gradient ascent on expected payoff, in the simple setting of two-player, two-action, iterated general-sum games, and shows that either the agents will converge to a Nash equilibrium, or if the strategies themselves do not converge, then their average payoffs will nevertheless converge to the payoffs of a Nashilibrium.
AutomaticaDistributed finite-time tracking of multiple non-identical second-order nonlinear systems with settling time estimation
252 Citations2015Yu Zhao, Zhisheng Duan +2 more
By computing the value of the Lyapunov function at the initial point, the finite settling time can be theoretically estimated for second-order multi-agent systems with the proposed control protocols.
Computer CommunicationsPrediction-based data aggregation in wireless sensor networks: Combining grey model and Kalman Filter
214 Citations2010Guiyi Wei, Yun Ling +3 more
A novel prediction-based data collection protocol is proposed, in which a double-queue mechanism is designed to synchronize the prediction data series of the sensor node and the sink node, and therefore, the cumulative error of continuous predictions is reduced.
Towards optimal sleep scheduling in sensor networks for rare-event detection
213 Citations2005Q. Cao, Tarek Abdelzaher +2 more
A protocol for node sleep scheduling that guarantees a bounded-delay sensing coverage while maximizing network lifetime is proposed that is optimized for rare event detection and allows favorable compromises to be achieved between event detection delay and lifetime without sacrificing (eventual) coverage.
EM-MAC
185 Citations2011Lei Tang, Yanjun Sun +2 more
The design and evaluation of the EM-MAC (Efficient Multichannel MAC) protocol, which addresses challenges through the introduction of novel mechanisms for adaptive receiver-initiated multichannel rendezvous and predictive wake-up scheduling, substantially outperformed other MAC protocols studied.
Radio-triggered wake-up capability for sensor networks
184 Citations2004Lin Gu, John A. Stankovic
This work presents power management schemes that eliminate wake-up periods unless the node indeed needs to wake up, enabled by a new radio-triggered hardware component, and evaluates the potential power saving in terms of the lifespan of a sensor network application, using experiment data and SPICE circuit simulations.
IEEE Transactions on CyberneticsDistributed Optimal Consensus Filter for Target Tracking in Heterogeneous Sensor Networks
181 Citations2013Shanying Zhu, Cailian Chen +3 more
It is proven that convergence property is guaranteed for the proposed consensus filter in the presence of process and measurement noise, and it is demonstrated that the heterogeneous sensor networks with the proposed filter outperform the homogenous counterparts in light of reduction in the network cost, with slight degradation of estimation performance.
IEEE/ACM Transactions on NetworkingMinimizing Delay and Maximizing Lifetime for Wireless Sensor Networks With Anycast
165 Citations2009J. Kim, Xiaojun Lin +2 more
The proposed solution to the joint control problem of how to optimally control the system parameters of the sleep-wake scheduling protocol and the anycast packet-forwarding protocol to maximize the network lifetime can outperform prior heuristic solutions in the literature.
IEEE Transactions on Mobile ComputingTight Performance Bounds of Multihop Fair Access for MAC Protocols in Wireless Sensor Networks and Underwater Sensor Networks
115 Citations2011Yang Xiao, Miao Peng +4 more
A tight upper bound on network utilization is derived and a significant fact is demonstrated that the utilization in networks with propagation delay is larger than that in Networks with no propagation delay.
Adaptive Agents and Multi-Agents SystemsResource Allocation in the Grid Using Reinforcement Learning
114 Citations2004Aram Galstyan, Karl Czajkowski +1 more
The results of the experiments suggest that reinforcement learning can be used to improve the quality of resource allocation in large scale heterogenous system.
Multiagent reinforcement learning and self-organization in a network of agents
107 Citations2007Sherief Abdallah, Victor Lesser
This work develops a novel self-organization mechanism that not only allows agents toSelf-organize the underlying network during the learning process, but also uses information from learning to guide the self- Organization process.
IEEE Transactions on Parallel and Distributed SystemsSleep Scheduling for Critical Event Monitoring in Wireless Sensor Networks
101 Citations2011Peng Guo, Tao Jiang +2 more
IEEE Transactions on CyberneticsSemi-Flocking Algorithm for Motion Control of Mobile Sensors in Large-Scale Surveillance Systems
98 Citations2014Samaneh Hosseini Semnani, Otman Basir
The Semi-Flocking algorithm approaches the problem by assigning a small flock of sensors to each target, while at the same time leaving some sensors free to explore the environment, allowing the algorithm to strike balance between robust area coverage and target coverage.
IEEE Transactions on ComputersHeterogenous Quorum-Based Wake-Up Scheduling in Wireless Sensor Networks
97 Citations2010Shouwen Lai, Binoy Ravindran +1 more
This work proposes two heterogenous quorum-based asynchronous wake-up scheduling schemes for wireless sensor networks and proves that any two grid quorum systems will automatically form a gqs-pair, and analytically-established performance trade-off of both designs.
IEEE Transactions on CyberneticsA Deployment Strategy for Multiple Types of Requirements in Wireless Sensor Networks
96 Citations2015Xuxun Liu
This paper proposes a deployment strategy for multiple types of requirements to solve the problem of deterministic and grid-based deployment, which consists of three deployment algorithms, which are for different deployment objectives.
Dynamic analysis of multiagent <i>Q</i> -learning with ε-greedy exploration
89 Citations2009Eduardo Rodrigues Gomes, Ryszard Kowalczyk
This paper presents a framework to model the behaviour of Q-learning agents using the ε-greedy exploration mechanism and model the problem as a system of difference equations which is used to theoretically analyse the expected behaviour of the agents.
IEEE Transactions on CyberneticsA Hybrid Memetic Framework for Coverage Optimization in Wireless Sensor Networks
78 Citations2014Chia-Pang Chen, Subhas Chandra Mukhopadhyay +5 more
A hybrid memetic framework for coverage optimization (Hy-MFCO) is presented to cope with the hybrid problem using two major components: a memetic algorithm (MA)-based scheduling strategy and a heuristic recursive algorithm (HRA).
Learning the task allocation game
73 Citations2006Sherief Abdallah, Victor Lesser
This work introduces a new gradient ascent learning algorithm that outperforms state of the art multiagent learners on the distributed task allocation problem and provides an informal proof that WPL has the same convergence guarantee as the best known algorithm, GIGA-WoLF.
Computer NetworksAn asynchronous scheduled MAC protocol for wireless sensor networks
71 Citations2012Beakcheol Jang, Jun Bum Lim +1 more
An energy efficient MAC protocol for WSNs is presented that avoids overhearing and reduces contention and delay by asynchronously scheduling the wakeup time of neighboring nodes and an energy consumption analysis for multi-hop networks is provided.
International Journal of Robust and Nonlinear ControlDistributed average computation for multiple time‐varying signals with output measurements
59 Citations2015Yu Zhao, Yongfang Liu +2 more
IEEE Transactions on ComputersJoint Design of Asynchronous Sleep-Wake Scheduling and Opportunistic Routing in Wireless Sensor Networks
50 Citations2012Chih‐Cheng Hsu, Ming-Shing Kuo +2 more
Simulation results show that ASSORT effectively achieves network lifetime extension compared with other routing schemes, and is an effective mechanism to reduce energy consumption by appropriately arranging sensor nodes to sleep.
A multi-agent learning approach to online distributed resource allocation
47 Citations2009Chongjie Zhang, Victor Lesser +1 more
A MAL algorithm is proposed and applied for optimizing online resource allocation in cluster networks, distributed to each cluster, using local information only and without access to the global system reward.
International Journal of Communication Networks and Distributed SystemsDecentralised reinforcement learning for energy-efficient scheduling in wireless sensor networks
31 Citations2012Mihail Mihaylov, Yann Aël Le Borgne +2 more
This work presents a self-organising reinforcement learning (RL) approach for scheduling the wake-up cycles of nodes in a wireless sensor network, and illustrates how sensor nodes arranged in line, mesh and grid topologies autonomously uncover schedules that favour the successful delivery of messages along a routing tree while avoiding interferences.
IEEE Transactions on CyberneticsBio-inspired Group Modeling and Analysis for Intruder Detection in Mobile Sensor/Robotic Networks
23 Citations2014Bo Fu, Yang Xiao +2 more
A novel bio-inspired approach to determine group size by researching and simulating primate society and comparing large and small groups through simulations and theoretical results are presented.
Learning the IPA Market with Individual and Social Rewards
10 Citations2007Eduardo Rodrigues Gomes, Ryszard Kowalczyk
A Reinforcement Learning Algorithm for Market Participants in FTR Auctions
9 Citations2007N.P. Ziogos, Athina C. Tellidou +2 more
A Q-Learning algorithm for the development of bidding strategies for market participants in FTR auctions is presented, represented by an autonomous adaptive agent capable of developing its own bidding behavior based on a Q-learning algorithm.
IEEE/WIC/ACM International Conference on Intelligent Agent TechnologyLearning the IPA Market with Individual and Social Rewards
7 Citations2007Eduardo Rodrigues Gomes, Ryszard Kowalczyk
This paper extends the use of Reinforcement Learning to let the agents learn the demand functions given the utility functions by applying a new reward function, based on the social welfare of the allocation, and by considering more clients in the market.