It is our goal at Monitis to make the lives of web developers and system administrators easy. We have reviewed the 5 leading hosted hadoop-based applications and given a short analysis of them in this post to help guide you in finding a solution that best suits your needs.

The article covers: Amazon Elastic MapReduce; Cloud Era CDH; InfoSphere BigInsights; MapR M3 and M5 and Hortonworks Data Platform.

Amazon Elastic MapReduce (http://aws.amazon.com/elasticmapreduce/)

Introduced by Amazon in 2009, Elastic MapReduce automates the process of various Hadoop cluster processes and transfers between Amazon’s EC2 and S3 products. For a minimal fee, Amazon will provide its clients with the ability to launch a preconfigured Hadoop cluster to run a client’s MapReduce Program.

Pros

• Very easy to setup a job flow
• There’s an enormous amount of documentation available to help new users
• Example applications are provided, giving an option to test drive the application before putting it to use.
• Entire application system can be powered by a command line interface, compared to a web-based management console.
• Ability to conduct several jobs simultaneously and parallel.
• No hardware is needed and costs can be very limited, which is great for small businesses seeking to be more cost efficient.

Cons

• Need an account with Amazon Web Services (AWS)
• Service is only available in the United States
• Requires the use of Amazon’s S3 service, which adds extra costs to an overall project (data transfer, security etc.)

Cloudera CDH (www.cloudera.com)

Founded in March 2009, Cloudera was previously considered to be the Red Hat of the Hadoop World. With a large customer base of over 400 (including paid and free downloads), the company’s offerings include the  Cloudera Enterprise products and Training & Support Services. Formed by a number of key executives from various technology giants (Oracle, Yahoo, Google and Facebook), Cloudera is considered the pioneer in the Hadoop community, having a head start in the industry compared to its competitors.

Pros

• Free application that can be easily downloaded
• Installed internally within an organization which allows the company to have full control of all processes, jobs etc.
• Technical support is superior and the knowledgebase is an essential resource to anyone starting out with Hadoop
• Used by a large number of companies worldwide, and has been proven as a leading choice in Hadoop applications.
• Application includes additional resources and components (e.g. Pig, Hive, Flume, HBase, Zookeeper, Mahout, Whirr, Hue, Sqoop and Oozie)
• Cloudera conducts quarterly updates: eliminating the need to conduct a big scale annual upgrade.

Cons

• Requires companies to obtain the necessary hardware in order to install the application, adding additional costs.
• Additional costs are added to support and maintain the application, increasing the company’s operating costs.

IBM InfoSphere BigInsights (www.ibm.com/software/data/infosphere/biginsights)

A new product introduced in May 2011, the product is geared towards handling extremely large volumes of streaming data using a Hadoop-based analytics framework. IBM states that the IBM InfoSphere Biginsights will be able to handle “tens-of-petabytes” of data, and will retain a sub-millisecond response time. The company also plans to launch 20 new service offerings, including numerous analytical tools for business and IT.

Pros

• Superior product support and long standing company reputation established from many years of servicing the IT community.
• Comes standard with a number of essential components including; PIG programming, IBM DB2 and IBM BigSheets.
• Offers two replication models that provide log-based replication working independently (queue-based and SQL-based).
• Lots of documentation and step-by-step training is available from the IBM website.
• Superior product for analysing big data in motion that needs to be continuously analyzed in real time.

Cons

• New to the marketplace and has not been around long enough to ensure a solid reputation.
• An expensive solution for small/medium size organizations seeking to utilize a more cost effective application.

MapR M3 and M5 (www.mapr.com)

With headquarters in San Jose, CA, MapR markets its proprietary applications with a focus on providing a number of key features and capabilities for the use with MapReduce and Hadoop.

Pros

• Offers superior monitoring that can provide a better understanding of data distribution and processing – essential for achieving increased performance.
• A free version is offered, which includes everything except management tools which are only offered in its M5 series products.
• Excellent technical support and vast quantities of documentation available

Cons

• New to the marketplace so has a limited reputation
• An expensive solution for small/medium size organizations
• 24×7 support is only available on the paid version of the application
• Requires an enormous amount of disk space to install (25GB), compared to similar products.

Hortonworks Data Platform (http://hortonworks.com/)

Hortonworks was formed in June 2011 by a number of key architects and Hadoop committers formerly employed within the Yahoo Hadoop Software department. The company’s offerings include; HDP (Hadoop Data Platform) and Training Support Services. The company currently serves 2 customers – Yahoo and Microsoft.

Pros

• A spin-off Yahoo product, so it’s been tested in the marketplace.
• Lots of documentation and support available from the knowledgebase community.
• The company is continuously working with Yahoo to develop its future products
• Scalable to meet the demands of specific projects.
• Offers variations and expanded product offerings from partnerships with a number of specialized companies.

Cons

• Product is similar in nature to Cloudera, and provides similar features.

1 YEAR WEBSITE TRAFFIC COMPARISON (from Compete.com)