Last year, my book club read William Dalrymple's Nine Lives: In Search of the Sacred in Modern India and for once (possibly the only time) opinion was unanimous - we all loved it.

The book, released in 2009, comprises intimate portraits of nine people who represent India's diverse sacred traditions, encompassing mysticism, monasticism, music and dance. Through these biographies, Dalrymple examines how ancient faiths and rituals maintain their grip as India convulses with the rush to modernity.

The first story is the spellbinding tale of a nun, an adherent of the Digambara, or Sky-Clad Jains, thought to be the most severe of all India's ascetic sects. Dalrymple first encounters Prasannamati Mataji while ascending Vindyagiri, a hill topped by the largest monolithic statue in India, a place of pilgrimage. The slender woman, barefoot and wrapped in a white cotton sari, is climbing the stairs ahead of him. She gently wipes each step with a peacock fan before placing her foot, ensuring she does not inadvertently step on an insect. Jain strictures dictate that no harm can be caused to any living thing.

"She had large, wide-apart eyes, olive skin and an air of self-contained confidence that expressed itself in a vigour and ease in the way she held her body. But there was also something sad and wistful about her expression as she went about her devotions; and this, combined with her unexpected youth and beauty, left one wanting to know more."

This is how Dalrymple hooks the reader, conjuring a picture of a whole person with a few deft brushstrokes. We have a sense of the nun's spirit, we like her and we wonder what's behind her melancholy. He's very good at showing, not telling; stepping back to let the subject reveal their own story.

Mataji tells him that she was a pampered child, born to a wealthy merchant family. When she was 13 years old she met a Jain monk who impressed her and she decided she wanted to be like him.

Sky-Clad Jains believe that emotional attachments bring suffering. To become a nun, Mataji had to abandon her former life, relinquish her money and belongings and sever ties with her family. She also had to pluck her hair, strand by strand; a painful test of commitment.

Dalrymple soon finds out what's troubling her. There is one attachment from which she cannot free herself. Recently she had nursed her best friend, Prayogamati, another nun, through a painful, difficult death. Mataji was consumed by grief. "I could not bear it," she tells Dalrymple. "I wept, even though we are not supposed to."

The two women had spent 20 years walking the dusty roads of India together. Then Prayogamati contracted tuberculosis and malaria. At the age of 36 she decided to undergo sallekhana, a ritual fast to the death that is the culmination of life for an ascetic Jain. Mataji explains the long, drawn out process. "First you fast one day a week, then you eat only on alternate days … you give up rice, then fruits, then vegetables, then juice, then buttermilk. Finally you take only water."

As we are digesting this, Mataji reveals that she, too, has embraced sallekhana. She has already given up various foods.

"It's not suicide," she says, emphatically. " Sallekhana is a triumph over death, an expression of hope."

It's heartbreaking, and utterly baffling.

"It's not an easy one to grapple with," says Dalrymple. "What intrigued me was that the nun herself was clearly struggling with it. She obviously felt a very intense love for her friend and felt confused about whether she should remain detached and see everything as part of the eternal cycle, as her guru had taught her."

He suspects she may have chosen to go down the same path out of loneliness and despair.

Was he tempted to try to dissuade her? His answer is very honest.

"I only met her for 48 hours. I was following my instincts as a journalist and writer. When you're in hunting mode there's a double thing going on. At one level you're empathising entirely with the person and feeling all that they're feeling, but there is also a sliver of ice in your heart saying, 'Yes, this is my story'."

Dalrymple seems to have spent his entire career on a good story. An acclaimed author of travel and history books, the Briton has won multiple awards, including the Duff Cooper and Wolfson prizes and the Asia House Literary Award. He's also a journalist and broadcaster, and co-founder and co-director of the Jaipur Literature Festival. He has spent most of the past 25 years living in Delhi.

For someone so scholarly, Dalrymple writes with a profoundly human touch. He brilliantly evokes the atmosphere and colour of past times, writing non-fiction with the elegance and vibrancy of a novel.

We meet at Petrus, the French restaurant at the Shangri-La hotel in Admiralty. Dalrymple has just enjoyed "the best breakfast of my life" (lobster and scrambled eggs) and is in fine spirits. He's talkative, enthusiastic and he's got a lovely big chuckle of a laugh. He laughs a lot. It's highly infectious, so I laugh a lot, too.

The previous evening I watched him present "The Last Mughal", an event held at the Ovolo Hotel, in Wong Chuk Hang, as part of the India by the Bay festival. It was an electric performance. Dalrymple recounted stories of India's Mughal empire and read excerpts from poets and philosophers, interspersed by piercingly beautiful songs from the period, sung in Urdu by classical vocalist Vidya Shah.

Dalrymple has written two books that cover this slice of history, exploring the relationship between the British and the Mughals in India.

White Mughals tells the true story of a love affair, and marriage, between Lieutenant Colonel James Kirkpatrick, a British officer, and Khair-un-Nissa, a noblewoman from Hyderabad. In the 18th century, the early days of colonisation, the British were surprisingly well integrated. They embraced the cultural life of India and one-third of British men married Indian women.

Dalrymple projected paintings on the Ovolo wall that showed British officials wearing full Indian garb, smoking hookahs and being entertained by dancing girls. There was even one gentleman going out for a ride on his pet tiger.

As time passed, the balance of power shifted and the culture changed. Paintings from later periods show British men wearing formal Victorian attire, sweltering under the Indian sun and being waited on by the subjugated locals.

The Last Mughal: The Fall of a Dynasty, Delhi 1857 recounts the final days of the great Mughal capital. At the heart of events is the last Mughal emperor, Bahadur Shah Zafar II. King in name only, he nonetheless created a court of great brilliance. Then, in 1857, the Indian army mutinied, culminating in one of the bloodiest upheavals in the country's history.

The two Mughal books explore how Christianity, Hinduism and Islam inter-reacted and were influenced by each other. Religious syncretism - what happens when one religion rubs up against another and the border blurs - is a recurring theme in Dalrymple's work.

Dalrymple attended Ampleforth College, in Yorkshire, northern England, a school run by monks who wear full robes. Did his education inspire his interest in religion?

"It started at home, actually, because my family are very seriously Catholic. I am not, but once you are taught to look at the world through religious spectacles, you realise how much they can alter your vision. I'm intrigued by how people's religious beliefs skew and alter and transform their lives. Religion is often at the heart of the human condition, and it's an essential part of understanding a country like India."

After Ampleforth, Dalrymple took a gap year and, quite by accident, stumbled onto the experience that would change his life.

"I was about to participate in a dig with the British School of Archaeology in Iraq when Saddam Hussein closed it down, believing it to be a nest of British spies. I had nine months to kill. A friend was going to India, so I went with him.

"I experienced total confusion for about three weeks, followed by a passionate love that has lasted until now. It was totally transformative. People often talk in clichés about things which change their lives but in a very real, clear and demonstrable manner, it did."

Dalrymple returned home to take a history degree at Cambridge University but he'd been bitten by the travel bug. Towards the end of the degree, he applied for a travel scholarship for medieval historians, having received a tip that the more ambitious the proposal, the better his chances of winning.

"I simply went to the library and worked out the longest medieval journey I could possibly do."

His idea was to follow in the footsteps of Marco Polo, from Jerusalem to Shangdu, in Mongolia, the site of Kublai Khan's stately pleasure dome. His trip formed the basis of his first book, In Xanadu, which he wrote at the tender age of 22.

Dalrymple went on to write three more travel books and then embarked on another big adventure, parenthood.

"With a little baby in the house, the urge to go on ambitious trips and risk your life humping along the Karakoram Highway suddenly disappears."

So he switched to historical writing, and has never looked back.

"To do first-rate travel writing is terribly difficult. Often travel can be quite banal. With history you choose an amazing story and you can't go far wrong. The challenge is to dig out new sources and find great material.

"Temperamentally, history writing is easier for me. You're marshalling vast quantities of data so it's more of an operation but, creatively, it's more straightforward because the events give you the backbone of your structure."

Dalrymple started writing history books in London but, in 2004, he moved to Delhi with his wife, artist Olivia Fraser, and their three children. Has his favourite city changed much since his gap year?

"It has transformed. When I was first there it was a government town of one million inhabitants. Now it's a huge, dynamic city of 26 million."

Given that he lives in a seething metropolis, Dalrymple's home life sounds surprisingly bucolic. His family occupies a farmhouse just outside the city with a 1.5-hectare plot of land, a swimming pool, chickens, goats and a "really serious" vegetable garden. They grow "nice things like cavolo nero and three types of chard" and are self-sufficient in vegetables for half the year. Books get written because the house - which is rented - comes with three gardeners.

"It's not quite Downton Abbey, but we're living in a style that no writer's salary could bring you anywhere else in the world."

They get lots of visitors and reactions are polarised.

"You can tell within about 10 minutes if people are going to get it or not. There are some that react with complete horror and want to drive straight back to the airport, and others who return home gloopy-eyed, as if they've just had a religious experience."

Do Westerners often misunderstand modern India?

"It's a classic Western mistake to come to India thinking it's a wonderfully spiritual land when, in fact, there are very few people more spectacularly materialistic than a good, urban, middle-class Punjabi, busy collecting designer labels and trophy cars."

I ask if he ever finds it a struggle to witness, daily, the grinding poverty and injustice that blight the poorer sections of Indian society. He leaps to the defence of his adopted home: "I don't feel that the poor in India are any more desperate than they are in the West."

I'm not sure I agree - you don't see children eking out a living on the streets of London, or Hong Kong, for that matter. He's emphatic though: "For all that you see beggars at traffic lights, there's huge amounts of ambition and aspiration in India. People work hard, send money back to their villages, educate their children any which way they can.

"So, no, it doesn't upset me."

Dalrymple's most recent work is Return of a King - The Battle for Afghanistan, a doorstop of a book that tells the story of Britain's catastrophic first invasion of Afghanistan. In 1839, the British captured Kabul, ejected the ruler Dost Mohammad Khan and replaced him with a pliable, pro-Western monarch, Shah Shuja.

The presence of the British aroused intense hostility among Afghans and, in 1842, a rebellion exploded in Kabul. The British soldiers, accompanied by an army of Indian sepoys, fled towards Jalalabad, 130km away. They marched in blizzard conditions and, on reaching the high mountain passes, were slaughtered by Afghan tribesmen. Those who weren't killed in action, froze to death. Their bleached bones can still be found scattered on the hillside near the village of Gandamak. Of the 18,000 British soldiers who set off, just one man, an assistant surgeon mounted on a dying pony, made it to safety in Jalalabad.

It was Britain's greatest imperial disaster, and humiliation, of the 19th century.

Much has been made of the way Dalrymple's book demonstrates that history is now repeating itself, but I wonder if that's a little contrived - an easy hook for book reviewers. How specific are the parallels once you examine them closely?

"More so than in any other instance I've come across in my life," says Dalrymple. "In both cases, the West goes blundering in and installs a client state. Hamid Karzai [who in 2004 became Afghanistan's president, following the American-led overthrow of the Taliban] is the chief of the Popalzai, the same tiny sub-tribe that Shah Shuja once headed. Meanwhile, the descendants of the tribe that brought the British down in 1842 - the Ghilzais - now make up the foot soldiers of the Taliban."

While researching the book, Dalrymple read the diaries of Henry Rawlinson, political agent at Kandahar during the first Afghan war. Rawlinson liked to relax after work by watching the sun go down at a shrine overlooking a lush, green valley on the edge of town. Dalrymple headed out there, diaries in hand, keen to experience it as Rawlinson had. In one passage, Rawlinson describes watching a party of British lancers, with their scarlet coats and plumed shakos (military hats), riding down to cross the bridge over the Arghandab River, where they are ambushed by Durrani tribesmen. As Dalrymple was sitting, reading this, 170 years later, a convoy of American Humvees came thundering down the hill and on the exact same spot an improvised explosive device went off.

"I had a very spooky sensation of history repeating itself precisely, almost like déjà vu."

When Return of a King was published, in 2013, Dalrymple was summoned to brief United States officials in the White House.

"There's a whole floor dedicated to the AfPak [Afghanistan-Pakistan] team. They're young and super smart. They knew all the tribal breakdowns and political affiliations, but they had very little cultural or historical knowledge."

He also spent considerable time discussing his findings with Karzai.

"The lesson he took from it was that he should distance himself from his backers in the eyes of Afghans. If he was perceived as a Western puppet, he risked being assassinated. The British ambassador later told me I had made things more difficult because Karzai was issuing public statements, distancing himself from the West, and this was partly my fault. But Karzai's still alive and democracy has survived in Afghanistan!"

The other jewel in Dalrymple's crown is the Jaipur Literature Festival.

"It started off as a little thing I did with some friends. The first time, 14 people turned up, 10 of whom were Japanese tourists who had got lost."

Within a decade it has grown to become one of the largest literary festivals in the world. This year's event involved 300 authors and 140 musicians, and the organisers recorded a footfall of 245,000 people. The festival, which is free to attend, is famous for its party atmosphere.

"It's fabulous," says Dalrymple. "At English literary festivals, the average age is about 60. Our average age is 21. Thousands and thousands of kids buoyed up on literature, longing to be part of this whole thing. And the writers have to shut up at 6.30pm and then we have music and dance until dawn. It's like Glastonbury for books."

Persuading the best writers in the world to participate has been easy: "Everyone says yes, even the writers who famously don't travel anywhere. It's January, so it's often freezing in their home countries. It's Jaipur, it's warm, it's gorgeous."

The problem Dalrymple faces every year is finance: "We rely on sponsorship and, unfortunately, millionaires in India simply aren't that interested in funding the arts."

Despite an ethos of inclusivity, the organisers can't exercise complete creative freedom. Salman Rushdie, whose 1988 novel The Satanic Verses upset Muslims so much he has received death threats, has been forced to withdraw on several occasions.

"We're not in Manhattan. We're in the most conservative state in India, so we have to be sensitive. Topics like the war on terror, the history of the Koran, Hindu nationalism, Dalit politics and homosexuality have the potential to cause problems. We can and do discuss them but we're careful how we frame and title such events. That's a trade off for hosting it in one of the most wonderful cities in the world."

I'm keen to find out what my book club can look forward to in the future. Dalrymple says his next project is a history of the East India Company.

"We still talk about the British conquering India - I'm guilty of it myself - and that implies it was the British government. But it wasn't. It was a multinational corporation. The company had its own army of 200,000 men. There's no modern equivalent - it's like Microsoft with an army, or PepsiCo with nuclear submarines."

I, for one, can't wait to read it.

 

BY THE SAME AUTHOR: Richard Leakey - Angelina Jolie's latest muse