| By Maureen O'Gara | Article Rating: |
|
| March 19, 2009 08:00 AM EDT | Reads: |
3,380 |
Greenplum, the high-end open source database house and friend of Sun – so it’s gotta be nibbling on its nails wondering whether it’s lost a channel – has some sexy new technology to accelerate data loading for companies stressing under exponential data growth.
It’s called MPP Scatter/Gather Streaming or SG Streaming for short and it’s supposed to eliminate those darn bottlenecks usually associated with mainstream data loading.
Greenplum claims to have created the lightening-fast flow of data into its database for large-scale analytics and data warehousing. It says it’s getting loading speeds of over 4TB an hour in production with a negligible impact on concurrent database operations.
Let’s pause for a moment to consider that one terabyte is equal to a goose bump-provoking million books and that Greenplum’s claimed rates move the industry closer to real-time data warehousing.
Greenplum uses a shower head-like parallel-everywhere approach to loading in which the data flows from hundreds or thousands of parallel streams to every node of the database without any sequential choke points, quite different from the usual drip-drip-drip bulk-loading single-source stream like Oracle.
And the widgetry scales. The more nodes, the faster the loading rate so it can theoretically support better than 4TB an hour.
Data can be transformed and processed in-flight for extremely high-performance ELT (extract-load-transform) and ETLT (extract-transform-load-transform) loading pipelines.
The company says its precedent-setting approach avoids the need for a “loader” tier of servers (think MPP) that adds complexity and cost.
The widgetry supports both large batch and continuous near-real-time loading. Final gathering and storage of data takes place on all nodes simultaneously with data – compression is an option – automatically partitioned across nodes.
Greenplum says other parallel databases are as limited as traditional databases. Netezza, for instance, forces data to enter the system via a single node.
Published March 19, 2009 Reads 3,380
Copyright © 2009 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Maureen O'Gara
Maureen O'Gara the most read technology reporter for the past 20 years, is the Cloud Computing and Virtualization News Desk editor of SYS-CON Media. She is the publisher of famous "Billygrams" and the editor-in-chief of "Client/Server News" for more than a decade. One of the most respected technology reporters in the business, Maureen can be reached by email at maureen(at)sys-con.com or paperboy(at)g2news.com, and by phone at 516 759-7025. Twitter: @MaureenOGara
- Microsoft Tries Hadoop on Azure
- OpenXava 4.3: Rapid Java Web Development
- Asynchronous Logging Using Spring
- StorSimple Supports OpenStack
- What to Expect in 2012: Cloud Computing and Open Source Software
- Will PaaS Finally Bring Open Source Love to the Enterprise?
- AT&T Joins OpenStack, Floats Cloud Architect
- Linux Virtualization and Tired Open Source Myths
- Red Hat Sets Up GlusterFS Advisory Board
- OpenOffice.com Lives
- Selecting a Business Intelligence Solution
- Cloud Computing: A Platform-First Approach
- Adobe Sends Flex to the Apache Foundation
- i-Technology in 2012: Five Industry Predictions
- Microsoft Tries Hadoop on Azure
- OpenXava 4.3: Rapid Java Web Development
- Asynchronous Logging Using Spring
- StorSimple Supports OpenStack
- What to Expect in 2012: Cloud Computing and Open Source Software
- Will PaaS Finally Bring Open Source Love to the Enterprise?
- AT&T Joins OpenStack, Floats Cloud Architect
- More Use Cases for Big Data Analytics
- Linux Virtualization and Tired Open Source Myths
- Red Hat Sets Up GlusterFS Advisory Board
- After Ubuntu, Windows Looks Increasingly Bad, Increasingly Archaic, Increasingly Unfriendly
- SCO CEO Posts Open Letter to the Open Source Community
- Simula Labs Launches Hosted Delivery Platform To Enable Enterprise Open Source Adoption
- Where Are RIA Technologies Headed in 2008?
- Source Claims SCO Will Sue Google
- How Open Is "Open"? – Industry Luminaries Join the Debate
- Latest SCO News is Plain Weird
- SCO Claims Linux Lifted ELF
- IBM Tells SCO Court It Can't Find AIX-on-Power Code
- Flashback: Investing in 'Professional Open Source' - Exclusive 2004 Interview with David Skok, Matrix Partners
- Developing an Application Using the Eclipse BIRT Report Engine API
- HP Starts Pushing Desktop Linux




















