Comparing Reinforcement Learning algorithms applied to crisp and fuzzy learning classifier systems

The experiments show that the more complex credit assignment algorithms generally have better performances than the more basic ones also when applied to LCSs, and fuzzy LCSs seem to require a larger computational effort, but also show more robustness.

Abstract

We have implemented a tool to compare different modules of Reinforcement Learning algorithms applied to Learning Classifier Systems (LCS). We focus on three main classes of modules: credit assignment modules, exploration policies, and evolutionary strategies. For each class we have implemented many of the proposals we can find in literature and also some new algorithms that we have designed. In this paper, we present the results of the application of our tool to both fuzzy and crisp LCSs that learn behaviors for simulated autonomous agents. Fuzzy LCSs can be considered a successful approach to cope with real-valued input and output in a real environment. A lot of investigations can be done with this tool in this experimental setting. This paper is focused on the comparison among different credit assignment algorithms and on their performance in learning both crisp and fuzzy models. Our experiments show that the more complex credit assignment algorithms (such as, for instance, the TD(λ) generally have better performances than the more basic (such as Q-learning or Bucket Brigade) also when applied to LCSs. Moreover, fuzzy LCSs seem to require a larger computational effort, but also show more robustness.