Open Source Cloud Authors: Pat Romanski, Liz McMillan, Yeshim Deniz, Elizabeth White, Zakia Bouachraoui

Related Topics: Containers Expo Blog, Open Source Cloud, @CloudExpo

Containers Expo Blog: Blog Feed Post

SSP Failure to Cloud Storage Success

What a Difference a Decade Makes

Storability 10 year Reunion

Years ago, a few buddies and I started one of the first cloud storage providers. Of course, we didn’t call it cloud storage back then, but we merry band of brithers (and sisters), we first generation Storage Service Providers (1gSSPs) were cloud storage way before the cloud was cool.

All the 1gSSPs – StorageNetworks, ScaleEight, StorageWay, Sanrise, and others – failed. The core problem was and still is that renting raw capacity over the network is a lousy business model.

  • 1gSSPs couldn’t sustainably buy their storage cheaper than their retail customers (although over a beer I can share some great stories of how the early 1gSSP robber-barron’s ‘negotiated’ with the storage vendors during the boom).
  • SSPs couldn’t sustainably offer broad enough management efficiencies to generate profits.
  • SSPs couldn’t overcome a host of logistic and cultural issues (network performance/cost, stigma/liability of releasing core data, etc).

After the bust, and the 911 attacks, the entire business simply collapsed. Some of us – my company, Storability and others like Arsenal Digital - managed to flip over to providing managed storage services – running NOCs, and doing backups and restores for our customers. Wasn’t a great business, but we survived long enough to eventually be sold off.

For those interested in an unbiased history of the 1gSSP market, there is a thorough and thoughtful analysis from the National Center for Supercomputing Applications (NCSA), University of Illinois at Urbana-Champaign (UIUC) posted here.

Ten years later, things look different, and the same.

A whole host of storage service providers – nee’ cloud storage providers – has arisen, not so much from the ashes of the 1gSSPs, but certainly with their dust in the new CSP DNA. These folks have it a little easier than we did back then, and I think more than a few of them are going to make an honest living this time.

In addition to the obvious improvements in network connectivity, bandwidth, and reliability, I see three critical changes that I believe will mark the difference between the past failure of 1gSSPs and the future success of today’s Cloud Storage Providers – file systems, file virtualization, and file storage gateways.

File Systems

Data used to be stored as long strings of 1’s and 0’s, actually millions and billions of 1’s and 0’s we called megabytes, gigabytes, petabytes, etc.  Back in the 1gSSP days, applications like database management systems untangled those 1’s and 0’s and formed them into useful information like bank account records and social security numbers.

Today, data is still made up of 1's and 0's, but the fastest growing forms of data, from the pictures you upload with your cell phone, to the books you download to your Kindle, come packaged in a convenient format called a file.

Files matter – files have digital labels that convey information about the file package itself.  Raw strings of 1’s and 0’s don’t.  More importantly, files have business and human context - 1's and 0's not so much.

Context matters - with it, we can make decisions about how to treat data. With files, we can look at the metadata (the data about the data contained in the label or header attached to the file itself) and learn who created the file, how old it is, and even gain hints about its actual content (does the file contain a song or a spreadsheet?). With this information, we can make intelligent decisions about where to put the file, how many copies we should make, how often we should back it up for safekeeping, etc.

With raw megabytes – no context - we have no way of discerning what’s what, so we have to treat the entire string of 1’s and 0’s the same – in most cases that means treating it all as if it’s all vitally important.

1gSSPs got sort of a raw deal trying to build a business storing all that raw data.

  • They had to treat it all the same – backing it all up every night, for instance.
  • They had to connect it directly to live applications. The banking app needs instant anytime access to the entire database – no telling when you might make an ATM withdrawal – and apps don’t like to wait for data, so the connection has to be very high speed (laws of physics and economics apply here).
  • They had to have it all – because they couldn’t discern one cluster of 1’s and 0’s from another, customers had to trust the SSP with all their raw data.

Files make life easier for today's wannabe Cloud Storage Provider.

  • Customers can decide and control what files go to the cloud
  • CSPs can offer differentiated services for files based on metadata
  • Applications are not as dependent on instant and constant access to files - they've learned to be patient waiting for downloads, just like the rest of us.
  • Files can be uploaded and downloaded between users and CSPs with ease, so variability in persistence and performance of the connection is better tolerated

File Virtualization

So, the ability to decide and control file location is critical for the success of cloud storage, but it's not enough.

If we know that Sally’s MP3 file of Andrea Boccelli’s “Silent Night” is non-business-critical (albeit absolutely amazing and worth downloading today), we can decide to push Sally’s file to a cheap storage device, and not back it up, saving us money and effort. We might even decide to upload Sally's file to a Cloud Service Provider that offers essentially free storage capacity, and really save the company some dough.

BUT…how will Sally know where it is when she goes to download it next Christmastime? Whoops.

Important point - moving files and treating them differently based on metadata is great, but users and applications cannot be expected to keep track of constantly changing file locations. So cloud storage won’t fly as a business model if Sally or her apps need to keep track of what’s where in the cloud. 

Enter file virtualization, a technology which masks the file's physical location.

File virtualization matters – with a virtualized file structure, regardless of where it physically resides, Sally and Sally’s applications are tricked into thinking Sally’s file is on her network drive at G:/Sally/Music/SilentNight.MP3.  She never realizes, and does not need to know, that it’s been moved, thus the Cloud Storage business model becomes viable.

File Storage Gateways

OK, so now we can decide, move, and eliminate the disruption of moving. So far so good, but the Cloud Storage Business needs one more piece of connecting tissue to reach the tipping point.

If all we care about is Sally and her music, the cloud storage business is pretty simple and in fact a bunch of free or almost free services abound that do just that. Though I admit it is obviously possible (duh, Facebook) I don’t know how to make money off ‘free’ so I am leaving that model alone.

In order to have a successful enterprise oriented (paying customer) cloud storage business, CSPs need the rough equivalent of a set-top box they can provide to the customer. Today, most CSPs offer a programmatic interface to upload and download files, which is kludgy at best, and isn’t going to scale in a commercial environment.

  • No customer is going to want to be locked into a single CSP, or be forced to adapt their infrastructure or modify their applications to support one vendor's cloud model.
  • Latency is an issue - no matter what we do to reduce the performance imperative, we are eventually going to have to accept the logic that some subset of cloud resident files must reside at least temporarily at the customer premises (sort of like the difference between downloading and streaming movies).

File storage gateways matter – with a gateway in place the customer can treat the cloud just like another storage device. Sure, the vast majority of spinning disks are now located at the CSP, but to the customer the CSP (through the File Storage Gateway) appears to be just another NAS box - albeit a cheap one, that never fills up, and never needs to be backed up.

Up until recently, there have been a few FSG startups poking about, which has been useful for vetting and growing the concept.  Fortunately, for commercial CSPs, serious and trusted vendors are now releasing FSGs.

So now, I believe we finally have the necessary infrastructure and technology for Cloud Storage success.  It's now possible to decide what data can, and control what data will, be safely stored in the cloud.  Once separated and moved, it's possible to decide and control how data is treated when it gets to the cloud.  It's now possible to do all this without disrupting users and applications. Moving from one CSP to another is now simple and non-disruptive. The performance and persistence issues that plagued 1gSSPs are under control. Modifications to the files, user behavior, and application intelligence are no longer necessary to achieve the benefits of cloud storage.

To my mind, the combination of these three major changes in the storage landscape – massive reliance on file systems, commercialization of file virtualization, and emergence of viable file storage gateways have now combined to eliminate the barriers and challenges we faced in the 1gSSP days, and together provide the technical and process infrastructure necessary for cloud storage to finally reach its full potential.

With the technical and logistical hurdles out of the way, it will be up to the skill of the players to decide who wins.

All best wishes go out to the next generation of cloud storage entrepreneurs – as we brithers say, ladies and gentlemen, the ice is yours. Good curling!

Read the original blog entry...

More Stories By Kirby Wadsworth

Kirby is widely recognized throughout the storage industry for his expertise in marketing and business strategy.

A veteran of both startups and established storage vendors, Wadsworth was a founder of Storability and served as vice president of marketing prior to its sale to StorageTek. Earlier, as vice president and general manager of Compaq's Network Storage Services Business Unit, he envisioned and introduced Compaq's Enterprise Network Storage Architecture (ENSA) which is still widely recognized today.

As vice president of marketing for Digital's Storage Business Unit, Wadsworth launched Digital's StorageWorks product line into the open systems marketplace, and led the creation and introduction of the Enterprise Storage Array product family.

IoT & Smart Cities Stories
A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to great conferences, helping you discover new conferences and increase your return on investment.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
DXWorldEXPO LLC announced today that ICOHOLDER named "Media Sponsor" of Miami Blockchain Event by FinTechEXPO. ICOHOLDER gives detailed information and help the community to invest in the trusty projects. Miami Blockchain Event by FinTechEXPO has opened its Call for Papers. The two-day event will present 20 top Blockchain experts. All speaking inquiries which covers the following information can be submitted by email to [email protected] Miami Blockchain Event by FinTechEXPOalso offers sp...
SYS-CON Events announced today that IoT Global Network has been named “Media Sponsor” of SYS-CON's @ThingsExpo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. The IoT Global Network is a platform where you can connect with industry experts and network across the IoT community to build the successful IoT business of the future.
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
Disruption, Innovation, Artificial Intelligence and Machine Learning, Leadership and Management hear these words all day every day... lofty goals but how do we make it real? Add to that, that simply put, people don't like change. But what if we could implement and utilize these enterprise tools in a fast and "Non-Disruptive" way, enabling us to glean insights about our business, identify and reduce exposure, risk and liability, and secure business continuity?