The Ile d'If is just over a mile from the mouth of the harbour in Marseille, but the two are worlds apart.

France's largest port is thronged with people and traffic; the island has just a couple of tourists strolling around. Marseille is a cacophony of car horns, street musicians and voices speaking a dozen languages; the loudest noise on If is the sea breeze as it gently blows in from the Mediterranean. The waterfront of Marseille is a succession of bars and restaurants with a clientele that seems to be constantly in motion; the Ile d'If has a single cafe with a limited menu. You get the idea …

The hurly-burly of the nation's second city can be entertaining, but it does wear thin after a while.

The island is only barely overseas - the ferry ride takes just 20 minutes and costs €10.50 (HK$90) return - but it does transport the visitor to another time.

The city elders of Marseille realised very early on the strategic importance of the Frioul archipelago, of which If is the smallest of the four islands, and fortified the rocky outcrop to protect the mouth of the harbour and its trading ships.

The chateau that dominates the island dates back to 1529, although extensive modifications have been carried out over the centuries. Made from a dusky yellow stone, the square, three-storey castle has a trio of large towers punctured by fearsome-looking gun embrasures. The only entrance to the fortress is in the middle of the east facade, protected by a drawbridge over a dry moat. A portcullis was installed for good measure. Inevitably, the castle has a dungeon, although it is not as dank and rat-infested as some of the younger visitors would like it to be. The ground floor of the structure once housed the kitchens, a grain store and a barracks.

Remarkably, the walls of the small courtyard - in the middle of which is a well that holds rainwater funnelled down from the roof - still bear the legible graffiti (names and dates) of one group of former inhabitants: political prisoners.

One of the earliest prisoners to be held on If was the knight Anselme, who was accused in 1580 of plotting against the monarchy. More than 3,500 Protestants were imprisoned here over a period of two centuries from the 1680s. In the impossibly cramped conditions and with negligible hygiene, many died and their bodies were cast into the ocean.

Wealthy convicts were more fortunate and were permitted to pay for a better cell on one of the upper storeys of the castle. In 1774, the Count of Mirabeau, later to be a leader in the early stages of the French revolution, was dispatched to the fortress by his own father for his libertine leanings. But it is another count, a fictional one, who has made the Ile d'If famous.

In 1844, Alexandre Dumas published The Count of Monte Cristo, in which the main character, Edmond Dantes, finds himself imprisoned on the island. After 14 years, Dantes makes a daring escape, becoming the first person, albeit a literary one, to survive the powerful currents that swirl around the island. In reality, no one is known to have escaped the isolated prison.

The island was demilitarised in the final decade of the 19th century and the prison shut down, although it became strategically important again when the Germans occupied southern France during the second world war. Concrete platforms, the remnants of anti-aircraft batteries, are dotted around the island, including close to the red-tipped lighthouse that dominates the eastern end.

Ferries depart from the landing stage on the north side of the island, carrying day-trippers who, as the journey nears its end, look on with thinly disguised jealousy at the sailing boats and sleek powerboats that are moored the length of the Vieux Port of Marseille.

Passing between the imposing twin forts of St Jean and St Nicolas, which stand like sentinels on either sides of the harbour, is undoubtedly the best way to enter the city.

A port since ancient times, Marseille is fiercely proud of its identity and heritage. Its lifeblood is the Mediterranean, which turned the city into a melting pot of cultures, languages and customs, with a particularly large North African community. The architecture and flavour, however, remain unmistakably French.

One block back from the eastern side of the harbour is the Place aux Huiles, which in years gone by was a canal where barrels of olive oil - hence the name - were loaded aboard trading vessels. The canal has since been filled in and the square that sits over it is home to the umbrella-shaded tables of a dozen or more fashionable restaurants.

Whichever establishment you choose to dine at, the menu will inevitably include bouillabaisse, a fish stew that is a staple all over the city. Today, a pungent bowl of bouillabaisse may contain conger eel, monkfish, mullet or hake, while most chefs can be relied upon to throw in some mussels, crab or octopus.

Where the broad La Canebiere gives out onto Vieux Port is a construction that stands in sharp contrast to the traditional architecture that overlooks the harbour. L'Ombriere, designed by British architect Norman Foster, is a mirrored sunshade that encourages those who stroll beneath it to follow their progress in the mirror above their heads.

Appropriately, Marseille served as European Capital of Culture in 2013, but it is one of its oldest landmarks that boasts the most prominent position in the city.

The Catholic basilica of Notre-Dame de la Garde stands atop a 149-metre-high limestone outcrop on the south side of the port and dates from the 1860s. Incorporating Romanesque and Neo-Byzantine styles and topped by a square, 41-metre bell tower bearing an 11-metre gold-leaf statue of the Madonna and Child, the church has traditionally been seen as a guardian of Marseille.

"And there high on the mountain, the good Virgin, the good mother, looked out …" wrote French essayist and historian Michel Mohrt in the book Mon royaume pour un cheval ("My kingdom for a horse"; 1949). "The Good mother of the Garde who takes care of the sailors who are ashore … as for those who are at sea, let them sort themselves out!"

The basilica offers unmatched views across the city and beyond. At sea, those sorting themselves out when I take in the view are passengers disembarking from the latest ferry to moor alongside the Ile d'If.

Getting there: British Airways flies from Hong Kong to London and from the British capital to Marseille. Lufthansa flies to the French city via Munich.