Are You Prepared for the Exploding ‘Digital Universe’?

Are You Prepared for the Exploding ‘Digital Universe’?
used with the permission from Symantec

“A perpetual tsunami.”

That’s how IDC describes the current growth of digital information throughout the world.¹

It may be an understatement.

IDC estimates that last year, despite the global recession, the so-called “Digital Universe” grew by 62% to nearly 800,000 petabytes. (A petabyte is a million gigabytes.) This year, the Digital Universe is expected to reach 1.2 million petabytes.

Putting this explosive growth in context, IDC expects that by 2020, the amount of digital information will be 44 times as big as it was in 2009.

And here’s another statistic to keep in mind: While the amount of information in the Digital Universe will grow by a factor of 44 by 2020, IDC estimates the number of IT professionals in the world will grow by a factor of just 1.4.

Clearly, big changes are coming. This Tech Brief looks at some of the biggest ones for IT.

Protecting all that information
IDC estimates that by 2020 almost 50% of the information in the Digital Universe will require a level of IT-based security beyond virus protection and physical protection, up from about 30% this year. And the amount requiring the highest level of security is expected to grow by a factor of 100.

Expect our understanding of the degree of security required in the Digital Universe to profoundly change, IDC says.

“A YouTube video of a cat doing tricks would seem to need less protection against hacking or corruption than a home-banking customer’s account balances. But each YouTube video is associated with an IP address and end-user profile, and of course, that video might not be of a cat but of something not fit for public viewing. Even worse, that seemingly innocuous cat video may actually be an effective delivery mechanism for the new variants of malware being created by the criminal underground.”

Then there’s the issue that what a consumer or company wants protected may change from day to day as circumstances change. (IDC notes that Sara Palin’s email account became a lot more interesting to hackers the day she was named a Vice Presidential candidate.)

The issues for IT professionals are clear. They’ll need access to tools and expertise to protect all that data as never before. Getting the full attention of management will be vital as well.

The future of storage
Not surprisingly, these burgeoning quantities of information are having a huge impact on storage. In fact, IDC forecast in its initial Digital Universe study in 2007 that, for the first time, the amount of digital information created would exceed the amount of available storage. That gap continues to widen. While much of the digital content we create isn’t important enough to be stored, the amount of data that “requires permanent or longer-term preservation for a multitude of reasons is increasing exponentially,” IDC says.

According to IDC, nearly 75% of our digital information today is a copy – that is, only 25% is unique. That’s why data deduplication technology will be key going forward. IDC expects deduplication will soon take place on first-tier storage, which will significantly reduce post-process deduplication. The “cloud” (as in cloud computing) in particular will be an attractive place to reduce redundancy, given its one-to-many model.

Despite the overwhelming growth of digital information, IDC expects companies will continue to create, copy, and store information mostly on traditional storage technologies such as hard disk drives.

The decreasing cost of managing information
According to IDC, $4 trillion was spent in 2009 on hardware, software, services, networks, and IT staff to manage the Digital Universe. That spending is expected to grow modestly between now and 2020, meaning the cost of managing each byte will drop steadily.

IDC expects that as the cost per byte drops, so too does the investment in IT staff per byte. That means the tools for managing it all will have to change. IDC expects the increased complexity of managing digital information will be an incentive to migrate to cloud services.

In addition, there will be continued pressure inside data centers for automation, consolidation, and virtualization. Expect this do-more-with-less situation to put ever-increasing stress on IT organizations.

A call to action
Clearly, as the Digital Universe grows by an order of magnitude, we will have to fundamentally adjust the way we deal with information. Looking ahead, IDC prioritizes IT’s top issues as follows:

• Develop tools for search and discovery of information as the Digital Universe expands, including finding ways to add structure to unstructured data.
  
• Deploy tools for new levels of information management and prioritized storage.
  
• Deploy tools and expertise for security and privacy protection for a growing portion of the Digital Universe in hybrid physical/virtual environments.
  
• Get ready for some level of conversion to cloud-based services and start equipping IT staff with the new skills required for providing IT as a service.
• Obtain support from top management for setting policies on social media, training end users on information security, and classifying information in an effort to set storage priorities.

---

¹ “A Digital Universe Decade – Are You Ready?” IDC, 4/26/2010