Everyone in IT knows Moore's Law, which states that the number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years (sometimes 18 months). Named after Intel co-founder Gordon Moore, who first noted the trend in 1965, Moore's Law is testament to the tremendous increase in processing power and the consequent changes in business and society we have seen over the last 47 (and counting) years. While Moore predicted the continuation of this trend "for at least 10 years," current estimates are that this rate of progress will continue to at least 2020. Until, of course, we have the first operational quantum computers, but that's another story.
Processing and Data Go Hand in Hand
With so much processing power at hand, what is it used for? While it's obvious that computers process raw data to produce information, how much data do computers need to process? As it turns out, a lot. IBM says, "Every day, we create 2.5 quintillion bytes of data - so much that 90 percent of the data in the world today has been created in the last two years alone." (See Bringing big data to the enterprise.) Considering that 60 hours of videos are added to YouTube every minute (see: Exclusive: YouTube hits 4 billion daily video views), this doesn't seem to be far off the mark. While it's obvious that those "cute" videos of cats dressed in hats have little business value, a lot of data that is generated everyday does add value to business.
Have you ever wondered how insurance companies that send you promotional mailers seem to know so much about you, your family, your spending habits and your requirements? That's because they store and process data that you give freely through your buying behavior and even public records like births and deaths.
So ubiquitous is this gathering of data and processing it into actionable information that sometimes it can get out of hand. Earlier this year, The New York Times ran an article describing how Target was able to predict pregnancy even before one expectant mother's family got the news. The retailer sent her coupons for baby products, tipping off others in her household to her condition. (See: How Companies Learn Your Secrets.) Data is being collected everywhere. As a Target statistician quoted in the above article said, "If you use a credit card or a coupon, or fill out a survey, or mail in a refund, or call the customer help line, or open an e-mail we've sent you or visit our Web site, we'll record it and link it to your Guest ID (a unique identifier for each customer)."
The computational techniques that allow this can be collectively termed Business Intelligence (BI). This field includes reporting, online analytical processing, analytics, data mining, process mining, complex event processing, business performance management, benchmarking and predictive analytics. In addition to analyzing and predicting human behavior, business intelligence is used in fields as diverse as disaster management, astronomy, oil and gas exploration, and consumer electronics. However, as stated earlier, all these fields have one thing in common - data, lots and lots of it.
A Rare Breed, the Storage Administrator
Who is in charge of all this valuable data? Storage administrators.
Storage administrators are IT specialists that have been safeguarding corporate data for decades. Storage admins have one of the most demanding jobs in IT today. From securing data against unauthorized access and malicious code to granting access to authorized personnel, allocating storage space to approved users, releasing unused space, monitoring operations, and dealing with outages, the storage administrator has a lot on his or her plate. IT staff at large companies field approximately 100,000 storage-related requests every year, spending approximately half an hour or more on each of those tasks. That means a minimum of 30,000 storage administrator staff hours per year spent on storage requests.
Combine this with the present data explosion and the limited IT budgets in today's post-recession economy, and the problem seems insurmountable. Especially when you consider that unlike the amount of data, which is expected to grow by 20 times by 2020 according to Gartner research, the number of storage administrators is expected to increase by only 50 percent in the same period. With a limited supply and exorbitant demand, enterprises will face a tough time hanging onto existing talent, never mind hiring additional storage administrators. Even if they could, salaries in excess of $60,000 are no small change.
According to IDC research, by the year 2020, IT departments will be tasked with managing 50 times the current amount of information with only 1.5 times the number of IT professionals they have today. IDC estimates staff will also be responsible for 75 times the number of files they manage now and 10 times the number of virtual and physical servers. In other words, it is unreasonable to expect storage admins to function optimally under such circumstances.
The Solution - Storage Automation
Dictionary.com defines "automation" as "the technique, method or system of operating or controlling a process by highly automatic means, as by electronic devices, reducing human intervention to a minimum." In every field of human endeavor as man progressed, he has looked toward automation to make life easier, and ideally, storage administration should be no different.
Storage automation cannot only address this talent gap plaguing enterprise IT departments, but it can also make life for storage admins significantly easier. In fact, here's a list of functionalities that storage automation solutions might provide:
- Full automation - The solution delivers full automation of services without the need for manual intervention. The service portfolio includes:
♦ Provisioning - Allocation of storage space dynamically as per requirement
♦ Reclamation - Optimum utilization of available space by reclaiming unused fragments
♦ Remediation - Disaster recovery in case of outage without affecting normal performance
- Self-service portal - A portal equipped with proper authentication protocols that allows users to submit and track requests for storage services
- Storage services catalog - A catalog of fully automated storage services that can be fired up in a few steps without having to go through lengthy authorization procedures
- Storage classification - The capability to classify storage and storage services, thereby allowing differential service depending on classification
- Policy-driven services - Ability to define policies that would be applicable to storage services
- Fully audited services - Tracking of all operations by keeping a record in a secure audit log
- Charge back/show back - Ability to associate charge back or show back information to both the storage provisioned as well as the storage services performed, synchronized with the accounting system
- Universal compatibility - Compatibility with multiple hardware vendors such as HP, Dell, Hitachi, etc.
- Interoperability with other clouds - Interoperability with other clouds, both public and private, using standard application programming interface (API)
Bridging the Storage Admin Gap
Regardless of whether you view the current explosion of data as a blessing, curse, or somewhere in between, everyone can agree that it's a trend that will be with us for the foreseeable future. Our ability to create and store new data will continue to exceed our ability to train professionals to manage that data. Storage automation technology is key for enabling enterprise organizations to scale their storage admin staff to meet the needs of the current and near term data management needs.