Humans 'losing digital control' warn experts after system crashes

Experts warn complexity, demand and greed are combining to crash computer systems of governments and businesses around the world

PUBLISHED : Sunday, 25 August, 2013, 12:00am
UPDATED : Sunday, 25 August, 2013, 7:56am


A series of system crashes affecting Google, Amazon, Apple and Microsoft in the past fortnight has brought warnings that governments, banks and big business are over-reliant on computer networks that have become too complex.

The alarm was sounded by industry experts in the aftermath of a three-hour network shutdown that paralysed the operation of the Nasdaq stock market in New York on Thursday.

Jaron Lanier, the author widely credited with popularising the concept of virtual reality, warned that digital infrastructure was moving beyond human control.

He said: "When you try to achieve great scale with automation and the automation exceeds the boundaries of human oversight there is going to be failure. That goes for governments, for consumer companies, for Google or a big insurance company.

"It is infuriating because it is driven by unreasonable greed. In many cases the systems that tend to fail, fail because of an attempt to make them run automatically with a minimal amount of human oversight."

The Nasdaq collapse was caused by a communication failure between its platform for processing quotes and trades and that of another party - reportedly the New York Stock Exchange. It resulted in a third fewer shares being traded in the United States that day.

"These outages are absolutely going to continue," said Neil MacDonald, a fellow at technology research firm Gartner.

"The complexity of the systems created to support big data is beyond the understanding of a single person and they also fail in ways that are beyond the comprehension of a single person."

From high volume securities trading to the explosion in social media and the online consumption of entertainment, the amount of data being carried globally over the private networks, such as stock exchanges, and the public internet is placing unprecedented strain on websites and the networks that connect them.

Internet traffic today per person is measured in gigabytes, with six gigabytes of information exchanged per human per year. In 2017, that number will have risen to 16. By then, global data will be counted in zettabytes - roughly one trillion gigabytes.

High frequency trading by computers built to automate buying and selling high volumes of shares by hedge funds and banks has triggered and magnified the impact of IT failures on stock markets.

In May 2010, US$862 billion was erased from the value of US shares in 20 minutes when one company triggered a cascade of selling. This month's spate of outages came to international attention with the two-hour failure of The New York Times' website on 14 August, during which it resorted to publishing articles from its Facebook page.

A malicious attack was initially suspected, but a scheduled system maintenance was later found to be the cause.

On the same day, Microsoft customers began to report email failures. The outage was traced to problems with the Exchange ActiveSync service which serves email to many of the world's smartphones. When Exchange hit a glitch, the sheer volume of phones trying to connect triggered a ripple effect that took three days to control.

On 16 August, many of Google's websites, from e-mail to YouTube to its core search engine, suffered a rare four-minute global meltdown. The episode, whose cause Google has not explained publicly, served to illustrate the sheer volume of traffic its servers process. One monitor reported a 40 per cent fall in global internet traffic. Three days later Amazon was hit. The company gave no explanation, but Forbes estimated Amazon lost nearly US$2 million in sales. And Apple's iCloud suffered an 11-hour blackout on 22 August.