Data transformation engine based on Directed Acyclic Graph (DAG) flows configured through a Java API or via JSON, with a stated focus on performance, code re-use, testability and ease of operations. Runs over YARN and HDFS with native support for both micro-batch streaming and batch uses cases, and includes a range of standard operators and connectors (called Apex Malhar). An Apache project, graduating in April 2016, having been originally donated in August 2015 by DataTorrent from their DataTorrent RTS product which launched in June 2014. Java based, with development lead by DataTorrent who distribute it as DataTorrent RTS in two editions - a Community Edition (which also includes a basic management GUI and a tool for configuring Apex for data ingestion), and an Enterprise Edition (which further includes a graphical transformation editor, a self service dashboard, security integration and commercial support, and is also available as a cloud offering).
Other Names Apex, DataTorrent RTS Vendors The Apache Software Foundation, DataTorrent Type Commercial Open Source Last Updated May 2018 - v3.7 (Apex Core), v3.8 (Apex Malhar), v3.8 (DataTorrent RTS)
Is packaged by Apache Bigtop
version release date release links release comment 3.7 (Apex Malhar) 2017-03-31 announcement; changelist 3.8 (DataTorrent RTS) 2017-04-18 blog post 3.6 (Apex Core) 2017-05-04 changelist 3.8 (Apex Malhar) 2017-11-12 changelist 3.7 (Apex Core) 2018-04-28 changelist