RLlib: Abstractions for Distributed Reinforcement Learning

Eric Liang*, Richard Liaw*, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael Jordan, and Ion Stoica

35th International Conference on Machine Learning (ICML), 2018

BibTeX

@conference{ Liang2018RLlib,
  Title = "{RLlib}: Abstractions for Distributed Reinforcement Learning",
  Author = "Eric Liang and Richard Liaw and Robert Nishihara and Philipp Moritz and Roy Fox and Ken Goldberg and Joseph Gonzalez and Michael Jordan and Ion Stoica",
  Booktitle = "35th International Conference on Machine Learning (ICML)",
  Year = "2018"
}

Reinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation. We argue for distributing RL components in a composable way by adapting algorithms for top-down hierarchical control, thereby encapsulating parallelism and resource requirements within short-running compute tasks. We demonstrate the benefits of this principle through RLlib: a library that provides scalable software primitives for RL. These primitives enable a broad range of algorithms to be implemented with high performance, scalability, and substantial code reuse. RLlib is available as part of the open source Ray project.