Information Overload

The world produces a lot of new information per year-nearly two exabytes, or 2,000,000,000,000,000,000 bytes, to be exact, say UC Berkeley researchers.

According to a recent project headed by UC Berkeley School of Information Management and Systems professors Peter Lyman and Hal Varian, people in the world generate an estimated two exabytes, or two billion gigabytes, of information each year.

The massive amounts of information represent an aggregate total of new printed, film, optical, magnetic, and Internet media.

Divided amongst the world's 6 billion people, the estimation averages to approximately 250 megabytes of new data per person per year.

If all of this information were placed onto standard floppy disks, the stack would reach 2 million miles high.

Researchers say that one of the most surprising components of their study is not the pure amount of information, but rather the way it is distributed-notably in digital format.

According to their study, 93 percent of new information produced each year is stored digitally.

Varian, dean of the School of Information Management and Systems says he is not surprised by the numbers themselves. The digitization of much of the information, however, makes an interesting point of discussion, he says.

"It was surprising how dominant digital information is," Varian says. "Digital storage that is born digital is quite phenomenal."

Despite the researchers' nonchalant attitude about the bulk of information, they acknowledge that the numbers show that there exists much more information than people know what to do with.

Varian says that the study illustrates what he calls "information obesity."

"The world produces two exabytes per year in new data and about half of that shows up in my (e-mail) inbox every morning," he says jokingly.

In light of the study, researchers say it is important to think about methods to maximize utility from all information that is produced.

"It is clear that we are all drowning in a sea of information," researchers reported. "The challenge is to learn to swim in that sea, rather than drown in it. Better understanding and better tools are desperately needed if we are to take full advantage of the ever-increasing supply of information described in this report."

Aleksey Strygin, a UC Berkeley senior majoring in economics, who researched the Internet component of the project, says he agrees with the importance of being able to successfully sift through such a large cornucopia of data.

"Even as we were working, the amount of information was tremendous," Strygin says. "The problem is that the information is out there, and you need to find the right information for you."

Strygin's work showed that the fraction of information that people actually do use barely scratches the surface of the Internet.

"We separate the Web into the 'Surface' Web and the 'Deep' Web," he says. "The Deep Web (includes) databases-the size of the Deep Web is several times the size of the Surface. Pretty much everything that we use everyday is just one percent of the Web. The NASA database alone is 30 percent of the rest of the Web."

Lyman, associate dean of the School of Information Management and Systems, also points out the Internet's role in creating what the study referred to as the 'democratization of data,' or the ease of individual publication of information.

"Most of the last 500 years, publishing has been controlled by large entities," he says. "For the first time ever, we have a truly democratic publishing system. So much information is being made by individuals, not institutions."

However, Lyman also warns of the "Digital Divide."

"You have a real issue of who has access (to the information) and who doesn't," he says.

Varian says he agrees with Strygin's concern of being "over-informed."

"What is important is for people to be able to navigate and to use the information," he says. "Anyone with a computer has access to vast amounts of data that they didn't have before. You have to have people who are trained to manage the information. How do you deal with larger units of information?"

Lyman also notes the vast amount of information and the importance to be able to utilize it effectively, but mentions the inherent human ability to filter information.

"I think the human capacity to ignore information is very robust," says Lyman. "I'm able to go through my e-mail ruthlessly, but it won't be my primary source of information. It's all about the art of choosing. We should have a balance between face-to-face, computers and book information. But, Control-D (a keyboard command for delete) takes care of all my problems, except for messages from the Dean, which I read thoroughly."

Despite whatever value might be obtained by studying all of the information in all forms of media, printed, digitized or online, Varian recalls that the concept of "meaningful information" varies depending on context.

"There is more 'information' in an encyclopedia than in supermarket data," he says. "It's like in family photos. The meaning is in the eye of the beholder. One person's frivolity is another person's gold mine."

Tags: