It is a gloomy morning in London, but rain hasn't stopped about 100 international journalists from cramming into a press conference at publisher Penguin. All this for a book? But which one? Dan Brown's The Lost Symbol?
Few would have guessed that The Defence of the Realm, by Cambridge professor Christopher Andrew, could have drawn such a crowd. Six years in the writing, the book is more than 1,000 pages long. And yet, in the days following its publication, The Defence of the Realm outsold Brown's latest book on Amazon.
The reason for the book's success is its subject matter. The Defence of the Realm is the first fully authorised history of MI5, the intelligence agency that recently celebrated its centenary. Andrew illuminates some of the darkest corners of recent British history: the Cambridge Five; the Profumo Affair; and 'Death on the rock', when MI5 operations foiled an IRA attack on Gibraltar. The final chapters take the story up to the recent July 7 bombings in London.
Speaking at the press conference, Andrew nominates MI5's attempt to goad Neville Chamberlain into action against Adolph Hitler as his favourite of the agency's actions. Among the tactics MI5 employed was revealing that Hitler privately referred to Chamberlain as an 'arsehole'.
Flanked by Stuart Proffitt, publishing director of Allen Lane Books, and Sir Stephen Lander, MI5's director general from 1996 to 2002, Andrew admits to trepidation at the scale of the project, which began in 2002. After an intensive selection process, he was required to join MI5 and submit to their rigorous background checks.
Andrew then faced the real work of trawling an archive comprising 400,000 folders, which in turn hold several million separate files.
Asked the day after the press conference why spies are so fascinating, he says, 'That's your job, isn't it? Finding out other people's secrets? There's little chance we are going to stop.'
Andrew is both proud of his accomplishment and relieved that he didn't miss out on the opportunity. In the summer of 2002, he saw the advertisement for the position of MI5's official historian. 'I got my application in just before the deadline. I would really never have forgiven myself for not putting in for it.'
There were, Andrew says, some inevitable 'creative tensions' during the research. In one corner was Andrew, trying to 'understand as much as possible about what's been going on'. In the other were the authorities, who could not allow certain information - particularly that pertaining to current operations - to enter the public domain.
Gaining the trust of MI5 became essential, one reason why Andrew joined them. 'If you are a member of any organisation, people speak to you in a way they wouldn't to an outsider,' he says. 'I had the sort of knowledge only an insider would have.'
Andrew found working through the archive both daunting and thrilling. 'I really enjoyed that,' he says. 'It was beyond super-fiendish Sudoku.' Helped by a team of researchers, he began by reducing MI5's first century into a history that highlighted some major themes such as the vital but fluid relationship between MI5 and the prime minister and home secretary.
Another major topic was the central role of women in the service. Their glass ceiling was considerably lower in MI5 than elsewhere in the workforce, Andrew notes. Indeed, Britain's first female financial controller worked for MI5 during the first world war.
Perhaps Andrew's defining theme is the shift in the service's priorities from espionage to terrorism. 'Between the wars and during the cold war, it is obviously Russian espionage. Since the end of the cold war, it is, equally obviously, terrorism. First it was the IRA, then within months of the Belfast Good Friday Agreement, bin Laden hit his first major targets in East Africa.'
The mainland seems never to have been a priority. This seems odd given that Andrew claims the mainland and Russia currently do the most spying in Britain. 'Russian espionage is back to cold war levels. But the point is that it no longer has the significance in the new international environment. There is no threat.'
The mainland, by contrast, was never perceived as a threat, initially because it existed outside the Empire, but later because its targets were concerned with science and technology rather than national security secrets.
Andrew would like to see the agency focus on the long term in the future. The unexpected will occur, he says, but learning from history to anticipate the future will be crucial to the security service's effectiveness in years to come.
'That has been a recurrent failure in MI5,' Andrew says. 'But ... it's also a recurrent failure in early 21st-century culture. There has never been a generation which thinks it has so little to learn from the experience of previous generations.'
Andrew remains convinced that MI5 will thrive, not least because it retains a distinct glamour - thanks partly to Messrs Le Carre and Fleming. Andrew relates the story of a young intelligence officer meeting a tribal leader in a remote part of the globe. Assuming the elder would know little English, he was surprised by the greeting, 'Hello, Mr Bond.'
'If he'd said it's the German BND or the French DGSE, it wouldn't have been quite the same. The Europeans are no brand,' Andrew says. 'We are Nike.'
The Defence of the Realm by Christopher Andrew, Penguin, HK$428