Ushuaia’s waterfront stares out on to a small port, its container ships and the Beagle Channel. Inevitably, every visitor to this apparently dismal place takes a windswept selfie in front of the sign proclaiming this the “Fin del Mundo”, and probably most agree that the nickname isn’t wrong – not just because of geography but because there is an “end of the world” feel to this Argentinian town.

Ringed by mountains, dominated architecturally by its former prison and filled with houses seemingly knocked together from flotsam, this is a frontier town that was, it seems, deliberately positioned to face away from the rest of the world. Most ports face out towards other great trading nations; Ushuaia looks out over islands that even the penguins appear fed up with, and the decidedly uncommercial continent of Antarctica. Its tourism industry might say otherwise, but my first impression is of a place that peers grumpily out to sea and would be happier if you’d follow its gaze.

Most who come to Ushuaia visit at both ends of a trip to Antarctica. Ships varying in scale and luxury leave regularly on a southern heading, taking travellers from the end of the world and depositing them, for a while, right over the edge. As a result, Ushuaia’s main streets and tourist attractions appear set up to efficiently enter­tain, feed and educate visitors who won’t spend more than 36 hours there, visitors who will spend half that time fighting to suppress excitement about their big Antarctic trip and the other half dealing with the comedown after it’s all over.

Ushuaia, then, has but a few museums, chocolate shops and decent restaurants sell­ing Fuegan lamb and king crab. The weather, even in summer, is frequently brooding and overcast, and the skies over the mountains have a habit of turning black just as you reach the furthest point of a walk around town. Sights include a boat slowly rotting in a prime waterfront spot, a memorial to the Argentinian fallen of the Falklands war and a partly restored, convict-built railway that manages the impressive feat of connecting two stations that are equally hard to reach and equally unremarkable when you do.

Antarctic adventure: just take a sleeping bag for night on the ice

This is where my father and I find ourselves before and after our own big trip, sharing a warm, stuffy room in a guest house halfway up one of Ushuaia’s many steep side streets. The sense of being on something of a precipice is accentuated by the timing of the adventure. Antarctica is a trip we have talked about doing for a number of years while never being able to find the time and money needed to make it happen. The stars have, however, aligned, and a small window of opportunity has opened.

Big, expensive trips come weighted with expectations, and I envisage this one will involve us talking deeply on the big subjects, perhaps while staring wistfully out to sea, whisky in hand. So much has this idea taken hold, in fact, that I’ve been describing the trip as a father-son bonding affair, even in the knowledge that neither my dad nor I are “sharers” and sensing it is highly unlikely that we’ll break with tradition simply because we’re trapped on a boat together.

In the two days before our trip, we knock off most of Ushuaia’s cultural offerings.

From Pyongyang to Antarctica: how marathon-running holidays have taken off

We saunter studiously around the former prison, now a museum, taking in the eclectic exhibits in former cells: Antarctic parapher­nalia and many model boats. One wing of the old institution has been left well alone, a slowly rotting, chilly insight into the town’s former status as Argentina’s own Alcatraz or Robben Island – somewhere to throw criminals deemed too dangerous to be imprisoned in civilised places.

Blasted into submission by the rain and wind during a short walk along the front, we venture into the Galeria Tematica Historia Fueguina. Whereas the prison gives the impression of being lovingly but shoddily put together by committed volunteers, this museum seems to have been crafted at some expense by people who understand that tourists prefer exhibits with buttons to push and comical photos with life-size models.

Nevertheless, I leave the Galeria with a definite impression of early life in Tierra Del Fuego, although that has already begun to fade by the time we settle into a bottle of malbec and an Argentinean barbecue. The alcohol flows, my dad practises his limited Spanish and we feel we’ve “done” Ushuaia.

Ten days later, as we stare down the boat’s prow at the approaching Ushuaia dock, we see the row of buses and taxis waiting to take our fellow Antarctic travellers straight to the airport and home. Ushuaia appears as before – passive, unwelcoming and encircled by mountains; all the more gloomy for the inevitable contrast with Antarctica’s brilliant whites.

What a trip to Antarctica teaches sleep-deprived Hong Kong about need for rest

This suits the mood, really. Given all the information available, it is interesting how little you will hear about the subdued, impatient atmosphere that pervades the return leg of an Antarctic cruise. We have at least been fortunate with the sea, which was calm as we spent two internet-free days moping around the lounge with passengers who had achieved a bucket list “must see” and were now contemplating real life again, aimlessly scanning a horizon that was unlikely to offer up any more whales or penguins.

The cruise was truly a “dream trip”; it’s just that nobody mentioned we would spend two days in a purgatorial half-sleep before waking from it.

By virtue of my own fear of delayed arrivals and missed flights, and my dad’s misplaced faith in my judgment, we have no transfer service waiting. I have allowed us a far-too-comfortable two-day cushion on the return, and so we heft our bags back down the dock, find our steep side street once again and are welcomed back to our stuffy room by a landlord who is obviously unused to returning guests.

A photographic tour of Buenos Aires shows city in a whole new light

Eking out the town’s remaining sights takes us to the End of the World Museum, and its annex in the old governor’s house. A smattering of artefacts and some ship­wreck tales aside, it is the looping newsreel of early explorers and pilots in the museum proper that provides the most memorable aspect of any of Ushuaia’s cultural offerings.

One video displays the excitably undertaken – though in hindsight disastrous – transport of Canadian beavers to the National Park; another contains uncomfortable footage of Ushuaia’s original inhabitants, staring dejectedly into a camera pointed at them by eager anthropologists thrilling at their clothes and practices.

Far less grim are the treks we take – a scenic, steep and blustery hike towards Glacier Martial followed by a far less taxing walk around part of the Tierra Del Fuego National Park. Both are largely ignored by the Antarctic horde, Argentinean tourists the only ones sharing the forest trails and views of the Beagle Channel with us.

For all that Ushuaia exists as a place in between, the beauty of its wild coastline is undeniable, even by those suffering from overexposure to astounding Antarctic scenery.

Standing at the end of the world, turned away from the rest of South America, Ushuaia’s isolation is palpable. But there is a charm to the place, although you may have to devote more time to finding it than most visitors are ever likely to spare.

How to get there

Aerolineas Argentinas flies between Buenos Aires and Ushuaia. Travellers from Hong Kong can reach Buenos Aires via various North American and European hub airports.