How information mining helps impose order on chaos, for a while

Travelling to Traverse City, Michigan, in cold, rainy October, serendipity struck in funny ways. In a bookshop, I picked up a 2010 book by the Serbian physicist Vlatko Vedral, Decoding Reality, which sees the world through the lens of quantum information. I read it in Boston, in the midst of Hurricane Sandy as it tore along the US East Coast, demonstrating the frightening power of nature.

At the MIT bookshop I picked up more books on information theory. It blew my mind how modern quantum physics has begun to integrate the world in a new theory of information, almost as a theory of everything.

The ancient library of Alexandria had perhaps 500,000 scrolls. Its existence spread the knowledge of maths from the Arabs to the Greeks and the Romans, and this knowledge was used by the elite to control empires. If the Industrial Revolution was a revolution created by science and technology, the New Industrial Revolution is a revolution of information. The internet made knowledge accessible to all, and smartphones in the hands of millions created network power, so recently demonstrated by social media's impact on the Arab spring revolutions.

Physicists like Vedral are now pushing the envelope by explaining our perception of reality in terms of quantum information theory, since, today we process images of reality in terms of digitised information. All theories are really compressions of large amounts of uncertain and complex information into simple laws or principles, with which we try to predict the future.

Reality is today translated into digitised information of bits and bytes, which computers then process into knowledge using complex maths. A simple picture can be reduced to 30 kilobytes of information, but a top-rated camera is now capable of taking the same picture with 20 megapixels. That has information content of 30 or more megabytes, or 1,000 times more information. It means that if you have a top-range camera, you don't need an optical zoom (telescope) to see a distant object, you simply blow up your picture with a digital zoom.

In 1997, one of the pioneers of the internet, the late MIT professor Michael Dertouzos, conceived the web as an "information marketplace", where people and computers trade information and information services. The information marketplace actually describes all markets, because, today, almost all information is stored and processed by computers, human and machine.

As all bureaucrats know, information is power. Businesses also understand that having good information is not only very powerful, but very valuable. Social media use means we can check up on almost anyone by simply googling for information. It's now possible to measure how a supermarket is doing by using Google Maps to see how many cars are parked outside. And since smartphones like the iPhone can be tracked, you can map your movements on a daily basis. So traffic movements can be mapped and predictions made about jams by simply tracking the movement of phones and tablets. A group of researchers using Google Trends data was able to predict gross domestic product trends by the amount of searches for information using Google.