Principled Option Learning in Markov Decision Processes

Roy Fox*, Michal Moshkovitz*, and Naftali Tishby

13th European Workshop on Reinforcement Learning (EWRL), 2016

It is well known that options can make planning more efficient, among their many benefits. Thus far, algorithms for autonomously discovering a set of useful options were heuristic. Naturally, a principled way of finding a set of useful options may be more promising and insightful. In this paper we suggest a mathematical characterization of good sets of options using tools from information theory. This characterization enables us to find conditions for a set of options to be optimal and an algorithm that outputs a useful set of options and illustrate the proposed algorithm in simulation.