Neural Inference of API Functions from Input–Output Examples

Rohan Bavishi, Caroline Lemieux, Neel Kant, Roy Fox, Koushik Sen, and Ion Stoica

Machine Learning for Systems workshop (ML for Sys @ NeurIPS), 2018

BibTeX

@conference{ Bavishi2018ProgSyn,
  Title = "Neural Inference of {API} Functions from Input–Output Examples",
  Author = "Rohan Bavishi and Caroline Lemieux and Neel Kant and Roy Fox and Koushik Sen and Ion Stoica",
  Booktitle = "Machine Learning for Systems workshop (ML for Sys @ NeurIPS)",
  Year = "2018"
}

Because of the prevalence of APIs in modern software development, an automated interactive code discovery system to help developers use these APIs would be extremely valuable. Program synthesis is a promising method to build such a system, but existing approaches focus on programs in domain-specific languages with much fewer functions than typically provided by an API. In this paper we focus on 112 functions from the Python pandas library for DataFrame manipulation, an order of magnitude more than considered in prior approaches. To assess the viability of program synthesis in this domain, our first goal is a system that reliably synthesizes programs with a single library function. We introduce an encoding of structured input–output examples as graphs that can be fed to existing graph-based neural networks to infer the library function. We evaluate the effectiveness of this approach on synthesized and real-world I/O examples, finding programs matching the I/O examples for 97% of both our validation set and cleaned test set.