Apache Livy edit   discuss  

A service that allows Spark jobs (pre-compiled JARs) or code snippets (Scala or Python) to be executed by remote systems over a REST API or via clients for Java, Scala and Python. Supports re-use of Spark Contexts (and caching and sharing of RDDs across jobs and clients), multiple concurrent clients, secure authenticated communications and batch job submissions. Started in November 2015 based on code from Hue, with a formal announcement and first release in June 2016 based on development led by Cloudera, Hortonworks and Microsoft, before being donated to the Apache Foundation in June 2017. Hasn't yet graduated, but under active development, and used by tools such as Hue and Zeppelin.

Technology Information

Other NamesLivy
VendorsThe Apache Software Foundation
TypeOpen Source - Active
Last UpdatedJune 2017 - v0.3

Related Technologies

Is packaged byHortonworks Data Platform, Amazon EMR

News

Blog Posts