Amazon EMR edit  

Service for dynamically provisioning Hadoop clusters on Amazon EC2 infrastructure, with the ability to select one of more Hadoop based services to be pre-installed and configured. Supports selection of EC2 instance types, EC2 spot and reserved instances, programmatic execution of service jobs (steps), persistent or transient (terminate after pre-defined steps have been executed) clusters, automatic or manual scaling of live clusters, cloning of clusters, HDFS on local (EBS) node storage, an HDFS compatible filesystem (EMR File System - EMRFS) for accessing Amazon S3 storage (that supports consistency using DynamoDB for metadata), automatic configuration of Hadoop clusters and firewalls, integration with AWS CloudWatch and AWS Identity and Access Management, Hadoop encryption and Kerberos authentication, persistent storage of Hive metadata in AWS Glue Data Catalog, and bootstrap actions for custom configuration or installation of other services (with a GitHub repo of open source bootstrap action extensions). Manageable via the AWS Management Console, the AWS CLI, a REST API and a range of SDKs. Priced at an hourly rate (charged per second) based on the EC2 instance types being used, which is in addition to any EC2 or EBS charges.

Technology Information

Other NamesEMR; Elastic Map Reduce
Last UpdatedSeptember 2019 - v5.26

Related Technologies

PackagesApache Flink, Apache Hadoop, Apache HBase, Apache Hive, HCatalog, Apache Livy, Apache Mahout, Apache Oozie, Apache Phoenix, Apache Pig, Apache Spark, Apache Sqoop, Apache Tez, Apache Zeppelin, Apache ZooKeeper, Hue, Presto, Ganglia, JupyterHub, MXNet, TensorFlow

Release History

versionrelease daterelease linksrelease comment
5.182018-10-25announcementFlink 1.6.0, Zeppelin 0.8.0, and S3 Select with Hive and Presto
5.202018-12-14announcementSpark 2.4, Hue 4.3
5.212019-02announcementFlink 1.7; Recondiguring Apps on Running Clusters
5.222019-03announcementOozie 5.1
5.232019-04 Multiple Master Nodes
5.242019-06announcementNew versions of Flink, Presto, and Hue and Spark performance improvements
6.0 (beta)2019-09announcement 

See release notes for further details


Blog Posts