Principled Option Learning in Markov Decision Processes

Roy Fox*, Michal Moshkovitz*, and Naftali Tishby

13th European Workshop on Reinforcement Learning (EWRL), 2016

BibTeX

@conference{ Fox2016Options,
  Title = "Principled Option Learning in {M}arkov Decision Processes",
  Author = "Roy Fox and Michal Moshkovitz and Naftali Tishby",
  Booktitle = "13th European Workshop on Reinforcement Learning (EWRL)",
  Year = "2016"
}

It is well known that options can make planning more efficient, among their many benefits. Thus far, algorithms for autonomously discovering a set of useful options were heuristic. Naturally, a principled way of finding a set of useful options may be more promising and insightful. In this paper we suggest a mathematical characterization of good sets of options using tools from information theory. This characterization enables us to find conditions for a set of options to be optimal and an algorithm that outputs a useful set of options and illustrate the proposed algorithm in simulation.