bodyType

customContents

entityId

entityUuid

articlePageQuery

filter

article

articleBaseAdvertisingInfoArticle

articleListArticle

articleSeoArticle

hooksArticleAcknowledgementGateOverlayArticle

hooksTrackPageViewArticle

hooksContentInterestArticle

articleLinkingDataContent

hooksAmpRedirectArticle

articleSchemaContent

liveArticle

advertZone

sentiment

contentLock

topics

types

sections

paywallTypes

createdDate

publishedDate

sponsorType

urlAlias

moreOnThisArticles

commentCount

authorLocations

authors

corrections

displaySlideShow

multimediaEmbed

printHeadline

seriesContainer

articleSpeeches

socialHeadline

headline

images

flag

firstTopic

glossary

flattenTypeIds

image

relatedLinks

relatedNewsletters

__typename

sponsor

pdfFiles

seriesReverseOrder

series

disableOffPlatformContent

published

updatedDate

summary

subHeadline

body

keywords

readingTime

customTimezone

liveContents

SCMP Reporter

One of the wonderful things about the World Wide Web is its limitless ability to surprise and inspire.
And Internet Archive  is one of the Web's most surprising achievements. The Internet Archive's Wayback Machine, at www.archive.org,  came into being eight years ago when new online ideas were conceived, born, mutated and bought out at a dizzying rate. 
A group of Californian researchers realised  we were watching an important piece of history rush by and decided to record it.
The Wayback Machine comprises racks of x86 Linux servers, each  storing a terabyte of data. The system currently holds 30 billion webpages - adding up to a petabyte of data - and adds another  20 terabytes every month.
The Internet Archive has evolved into one of the most remarkable and consistently useful resources on the internet.
Just as an example, I was recently in the market for a new suit. I ventured into meatspace and asked a few friends for recommendations. One friend suggested  Maxwell's Clothiers, in Tsim Sha Tsui. Unfortunately, the former Maxwell's web address is now occupied by cybersquatters. But if I decide to follow my friend's advice, all I need do is check the Wayback Machine to dig out the old Maxwell's site, together with a handy map.
Besides the archive's own web crawlers, the system also accepts data direct from content creators. So it has downloaded a number of very large data collections, including much of the Library of Congress, the UK Central Government Web Archive and Project Gutenberg.  
The site has also added a multimedia dimension, with presidential speeches, short movies and the Live Music Archive, which features 2,454 free downloads of the Grateful Dead along with dozens of more obscure artists. 
The archive is not alone in its goal. 
Last month, the British Library and several  British charities set up the UK Web Archiving Consortium, which 'aims to expand the lifespan of website materials from around 44 days (the same life expectancy as a housefly) to a century or more'. 
However, that site is starting with only 6,000 websites because  it intends to get permission from the site owners before adding their pages to its database. The National Library of Australia has a similar project, named Pandora, which archives sites from down under. 
 Despite its massive size, the Internet Archive is far from perfect. Although it is free and attempting an apparently impossible task, it comes in for frequent criticism. 
Just this month, the site was attacked for removing a downloadable copy of Michael Moore's anti-Bush documentary Fahrenheit 9-11. It has also been accused of pandering to the whims of corporate censors and the oversensitive.
When a site downloads 20TB a month, it is inevitable that sooner or later it will run into copyright or other legal problems. And as a non-profit body, the archive is poorly equipped to deal with the potential dangers. So while removing pages must be a painful job, there is probably no other way to ensure the bulk of the collection remains available.
But my biggest gripe has always been that the archive is not searchable. 
The only way to get a result is to know the address before you begin. So it was a pleasure to discover that the archive has been beta-testing a free text search service named Recall. 
The service, at recall.archive.org, has actually been online since September last year but few people have noticed it, possibly because it is not linked from the Internet Archive's top-level pages.
Like many beta products, Recall is buggy and incomplete. It only searches pages from 1996, when the archive was started, and last December, and its index covers only a third of the archive. 
But it does let you search by date, and offers a series of drop-down menus to help you refine searches. So when I searched for Tung Chee-hwa, I was offered a dozen topics, ranging from Chris Patten to Article 23.
The site also returns graphic representations of the results, showing that references to Mr Tung peaked in 1998 and quickly dropped off until 2002, when they  spiked again. 
The 3D imaging proved impressive enough for Google to hire its creator, Stanford University researcher Anna Patterson. Ms Patterson apparently still works on the site in her spare time. 
If the Internet Archive could only integrate some of Google's search technology with its gigantic archive, we could have one of the world's most remarkable libraries.



Too bad the great internet library is teasingly out of reach