Adaptive Action Selection in Autonomic Software Using Reinforcement Learning

Evaluation results show that this approach can learn to select appropriate actions in a highly dynamic environment and compare it with another technique from the literature, and the results suggest that it can achieve similar performance in spite of no expert involvement.

Abstract

The planning process in autonomic software aims at selecting an action from a finite set of alternatives for adaptation. This is an abstruse problem due to the fact that software behaviour is usually very complex with numerous number of control variables. This research work focuses on proposing a planning process and specifically an action selection technique based on "Reinforcement Learning" (RL). We argue why, how, and when RL can be beneficial for an autonomic software system. The proposed approach is applied to a simulated model of a news web application. Evaluation results show that this approach can learn to select appropriate actions in a highly dynamic environment. Furthermore, we compare this approach with another technique from the literature, and the results suggest that it can achieve similar performance in spite of no expert involvement.