Welcome!

Open Source Authors: Maureen O'Gara, Jeremy Geelan, Liz McMillan, Reuven Cohen, Lavenya Dilip

Related Topics: Cloud Expo, Open Source

Cloud Expo: Article

Yahoo Sends Its Version of Hadoop into the Wild

It’s not, however, commercializing or supporting it
By Maureen O'Gara
Article Rating:
June 10, 2009 06:45 PM EDT
Reads:
 1,227

At the Hadoop Summit in California Wednesday Yahoo released the source code to its version of Hadoop, its Google-inspired distributed file system and parallel execution environment for sifting through Brobdingnagian-size data sets, for anyone to use.

It's not, however, commercializing it or supporting it. It's leaving that to the likes of Cloudera and Amazon.

Yahoo says it's releasing it because it's been asked so often; and the wider Hadoop's adoption the more it's asked. Anyway, it figures to get added development out of the exercise.

In a statement Yahoo's senior VP of cloud computing Shelton Shugar said, "By making the Yahoo Distribution of Hadoop generally available, we are contributing back to the Apache Hadoop community so that the ecosystem can benefit from Yahoo's quality and scale investments."

Its stuff runs the largest Hadoop clusters in the world and is extensively tested. Yahoo has also employed Hadoop founder Doug Cutting since 2006 when it began pouring money into the widgetry.

Hadoop now underpins Yahoo properties such as Yahoo Search, which is the world's largest Hadoop application, as well as Yahoo Mail and various content and advertising services. It runs on more than 25,000 Yahoo servers and, by Yahoo's count, analyzes tens of billions of web pages, multiple petabytes of storage and billions of new records a day.

The Yahoo Distribution of Hadoop is based entirely on code available from Apache Hadoop, the open source project at the Apache Software Foundation.

Yahoo pioneered much of the Apache Hadoop technology and is now the primary contributor to Apache Hadoop.

Cloudera figures incorporating Yahoo's refinements into its commercial Hadoop distribution will make it more robust.

The first release is Hadoop v20 in alpha test at Yahoo. See http://developer.yahoo.com/.

Published June 10, 2009 – Reads 1,227
Copyright © 2009 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.

More Stories By Maureen O'Gara

Maureen O'Gara the most read technology reporter for the past 20 years, is the Cloud Computing and Virtualization News Desk editor of SYS-CON Media. She is the publisher of famous "Billygrams" and the editor-in-chief of "Client/Server News" for more than a decade. One of the most respected technology reporters in the business, Maureen can be reached by email at maureen(at)sys-con.com or paperboy(at)g2news.com, and by phone at 516 759-7025.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.