The world of digital forensics is in the midst of dramatic changes that will send current techniques, approaches, and technologies to the sidelines as new solutions are required to meet the analytical challenges resulting from massive amounts of electronic data being created and stored by organizations and individuals.

We are all familiar with the abundance of statistics discussing the growing amounts of digital information as well as the anecdotal stories that demonstrate current investigative techniques are being overrun with too much data. The forensic market is beginning to look to industries like national intelligence and bioinformatics that are using information discovery and high performance computing techniques to develop new approaches to analyze digital data.

Currently forensic professionals are challenged with educating decision makers on the too much data problem and helping management better articulate the digital future and build the business cases required to find, invent, and deploy new technologies. In this article I am going to discuss some big picture concepts that, hopefully, will help this process. In addition, I will introduce a new generation of server-based solutions that allow investigators to bring massive, data center computing capability into the field—solutions that could remarkably change the ability to get the job done.

In the forensic community everyone uses analogies to describe the business, probably the most favorite being, “forensics discovery is like finding a needle in a haystack.”My favorite analogy is a little different and comes from growing up on the north coast of California and spending most of my summer days at the beach playing in the sand. If you have ever lost a small object, like a ring, at the beach—well you get the point. So while writing this article a question came to my mind that I thought could help illustrate the challenges facing the forensic community “are there more grains of sand on earth than there are bytes of data?” What do you think? Here is one of the best answers I found:

“How many grains of sand are there in the world? You could start off by trying to guess how many grains of sand there are in a spoonful of sand.Use a magnifying glass to count how many grains fit in a small section. Then, count how many of those sections fit in your spoon. Multiply the two numbers together to get an estimate.Usng this same principle, plus some additional information,mathematicians at the University of Hawaii tried to guess how many grains of sand are on the world’s beaches. They came up with 7,500,000,000,000,000,000, or seven quintillion five quadrillion grains of sand.”

That’s a lot of sand! It has to be more than the number of bytes of data we have generated, right? Well, if one grain of sand is equivalent to 1 byte of data, it is 25,000 times more than all the data stored in the Library of Congress (which is estimated at 300 terabytes of data). In fact, there are approximately 7.5 exabytes (or exagrains if you like) of sand in the world. However, according to a recent Digital Britain Report 1 exabyte of data is transferred every 3 minutes across our global networks. So in about 21 minutes, about the length of a 30 minute sitcom(sans commercials) we send and receive about 7 exabytes (or exagrains) of data. Oops, every half hour we are creating the equivalent of all the sand in the world.

When trying to bring this scale down to something akin to a forensic exercise, I thought, how many grains of sand in a typical professional beach volleyball court—I won’t bore you with the math but it’s about 512 billion grains (or about 512 gigagrains), or, in our analogy, about the size of a typical laptop hard drive. There you have it—digital forensics is like finding 10,000 grains of sand scattered in a professional beach volleyball court.

So that’s where we are today—a low-end PC typically has anywhere from a 320 to 500 gigabyte hard drive with the option to upgrade to a 750 gigabtye or even 1 terabyte hard drive. My son is a junior in college and this past Christmas I got him a 1 TB external hard drive for his HP laptop because the 250 gigabyte hard drive I got him three years ago was full. This is hard for me to grasp because when I started in this industry my first computer used 640 kilobyte floppy drives and my first Compaq had a whopping 5 megabyte hard drive. However, I get it, we are creating more and more data. In fact, if there is one truism in the post-internet age it is that a distinctly unique characteristic of humans is that we like to create information, share it with everyone, and provide our opinions on what we have read.