Information Overload

Information Overload
Written by: David Tan, CTO
Courtesy of CHIPS

Experts estimate that as of 1999, there was a total of 9 exabytes of electronically created data in the world. To put that in perspective, 1 exabyte is 1000 petabytes; a petabyte is 1000 terabytes; a terabyte is 1000 gigabytes – you get the idea. In practical terms, if you were to digitize the 17 million books in the Library of Congress, with full formatting, it would be approximately 136 terabytes. 9 exabytes would be roughly equivalent in size to the information contained in approximately 70,000 libraries the size of the Library of Congress!

That’s sounds like an incredible amount of data, but frankly that’s nothing. Since 1999, it is estimated that 12 NEW exabytes of digital data has been created. More than double all that existed prior to 2000. Even more amazing? More than 15 exabytes of new electronic data are now created annually!

Numbers like this are mind-boggling, and have little trickle-down impact on the average business. Even a large enterprise measures the data they store in terabytes at best, and never approaches petabytes or exabytes. So it’s important not to think in this large scale, but to think on a level that has more meaning to you and your business. Every thing we do in every aspect of our business today has an electronic data element. The world may be producing 15 exabytes of new data, and that may be overwhelming to think about, but what about the 500 gigabytes of data you are producing every year?

Clearly this is becoming a case of information overload. Dreams of the paperless office, or streamlined electronic communications come with a price. We need to store, index, archive and retrieve all that data. Not only that, we need the infrastructure to move that data around our networks, which includes local and wide area networks. And perhaps the worst of all is the regulatory and compliance restraints being put on businesses of all types and sizes. Not only do you need access to the data, you often have to prove the security and validity of the data, and provide an audit-trail of access and changes. Something has got to give.

Think about the frustration you have when you can’t find that proposal you wrote 6 months ago, or the email you received last week. How about when it takes 3 minutes to open a document stored in the main office. Or when you have 4 versions of a report and you have no idea which one is the most current. Frustration and aggravation are one thing, but what about the cost of a lawsuit where you have the burden of providing historical documents related to the case. Sure there’s a tremendous cost associated with retrieving those documents, but that cost pales in comparison to the cost of losing the lawsuit, or facing a fine for failing to meet regulatory requirements, or the cost of bad publicity in every case. The massive influx of data has caused businesses of all sizes to face IT challenges they were not prepared for. These challenges are particularly difficult in small and mid-sized companies who already have tight IT budgets, and often lack the in-house expertise necessary to make it work.

It’s not all gloom and doom, however. The cost of storage is falling drastically. More and more companies are designing solutions to address this for smaller companies with smaller budgets. Whether is a SAN for the SMB, a company intranet built on SharePoint, or an inexpensive data backup and archiving appliance, the products you need are starting to make their way to market. The key is not to wait until you get overwhelmed by a pile of 1’s and 0’s. Get out in front of it while you still have a handle on your network. Work with an IT partner that has experience with these problems and you just may be able to keep your data under control and get some sleep at night.