On November 4, 1962, the South China Morning Post ran a story under the headline: "Colony to have its first art gallery". Dorothy Swan, a teacher at the Diocesan Girls' School and originally from Massachusetts, in the United States, was about to open Hong Kong's first permanent exhibition space for artists, at 103 Chatham Road. The idea was to provide a space where small, one-man shows could run for longer than at the colony's alternatives: City Hall or St John's Cathedral Hall. It would be known as the Chatham Gallery and would feature the work of Asian artists.

The Chatham Gallery, near where the Hong Kong Museum of History now stands, closed in 1966. That year's Kowloon riots, which began 50 years ago tomorrow, didn't encourage the tourism on which business relied. Years later, Miss Swan (by then Mrs Brown) was asked why she'd done it. She replied that when City Hall opened, in March 1962, she'd realised Hong Kong artists in the large inaugural exhibition would have nowhere to continue showing their work individually afterwards. By the time she left the colony for Scotland, in 1971, that was still pretty much the case.

In this era of Art Basel, satellite art fairs, proliferating galleries, art-hungry celebrities and (myriad) press releases, Swan's ur-gallery - where a customer once paid with travellers' cheques dating back to 1927 - seems quaint beyond belief. But it had a presence. In 1963, she invited the Fifth Moon Group from Taiwan to exhibit; their name paid literal homage to the artistic "Salon de Mai", in 1940s Paris. One of the group's co-founders was Liu Kuo-sung, who had started doing contemporary, i.e. abstract, ink painting. When he exhibited his work Clouds Know No Emptiness at Swan's gallery, it was bought by the new City Hall Museum and Art Gallery (which, in 1975, would become the Hong Kong Museum of Art).

In 1971, Liu moved to Hong Kong, and became head of the fine arts department at Chinese University, a position he held until 1976. He returned to Taiwan in 1992 and, now 84, is among the younger of the surviving artists in an exhibition of Hong Kong art from the '70s.

That exhibition is at Ping Pong, which stylish barflies will know as a high-ceilinged, former table-tennis hall in Sai Ying Pun and not, strictly, a gallery. But of all the gin joints in all the towns in all the world, Ping Pong's name most surely echoes the diplomatic back-and-forth between East and West, Mao and Nixon, of that decade; and, architecturally, it's a perfect place in which to indulge a visual retro vibe.

Hugh Zimmern, Ping Pong's owner, loosely divides the work on show into three categories. There are the China-born artists, such as Ding Yanyong and Irene Chou, who came down to Hong Kong and worked in ink. There are the Europeans, who'd usually had a traditional art education: Brian Tilbrook, Kitty Burns, Arthur Hacker (possibly best-known to Hongkongers for another 70s feat of creativity - Lap Sap Chung, a verdant litterbug who starred in a government clean-up campaign). And there are the Hong Kong artists who, typically, were self-taught - the two exceptions being Kwok Mang-ho, better known as Frog King, who trained as an ink painter (and this year collaborated with Shanghai Tang to create a limited-edition range of sunglasses for Art Basel), and Antonio Mak, who studied at Goldsmiths and the Slade in London, and was the forerunner of a generation that would attend art schools. (Mak died of cancer, at 43, in 1994.) Most of them had day jobs.

Luis Chan, for example, who was born in Panama and came to Hong Kong in 1910, aged five, was a legal typist who went on to design fonts for a ferry company. He tutored himself in Western art but distinctions between East and West weren't the point: it's his fantastical landscapes for which he is now most renowned. (Incidentally, despite already considerable acclaim, he wasn't one of the artists in the inaugural City Hall show in 1962; none of his work was selected by the committee.)

The exhibition also includes a pair of photographers who were skilled in the distinct forms of artifice to which Hong Kong people were subjected during that era of cinematic and cultural revolution: Yau Leung, who worked for the Shaw Brothers film studio, and Meng Minsheng, who was born in Shanghai but grew up in Hong Kong and posed people in the star-struck attitudes favoured by Communist propaganda.

Liu isn't the only Chatham Gallery alumnus present; others include Jackson Yu (whose work, Autumn Leaves, arrived from the auction house complete with an old Chatham Gallery receipt) and Rosamond Brown. Brown, who was born in England in 1937, graduated from the Academy of Fine Arts at Belgrade University (then in Yugoslavia, now Serbia) in 1958, continued on to London's Central School of Art and Design (now Central Saint Martins) and arrived in 1964 in Hong Kong, where she still lives in the magnificent house on The Peak that her late husband, Charles, wisely bought during the economic uncertainty of the - considerably more serious - riots of 1967.

She came, she says, with two suitcases and three paintbrushes; but her greatest practical asset was her knowledge (gleaned at Harvard, where she took an art course while Charles finished his architectural studies) of acrylic paint. Hearing this, an American painter friend, Sheila Isham, took her down to City Hall shortly after she arrived to meet the Hong Kong artists who gathered in its coffee shop every week, and were eager, in this humid climate, to discover acrylic's possibilities for themselves. That circle included Hon Chi-fun, Gaylord Chan and Cheung Yee, all of whom are in the Ping Pong exhibition.

"Wasn't I lucky?" Brown says. "I know young people today who never get in with the Chinese crowd but we all used to meet and have these exchanges. I ordered acrylic paints for them, they showed me how to use a Chinese brush. It was a lovely period, there was hope and excitement. Hon Chi-fun was my best friend, a marvellous man who's not had the recognition he's deserved."

Hon, who was born in Hong Kong 94 years ago and is not in good health, used to send his friends exquisite hand-painted Christmas cards every year; one of these, originally sent to Brown, is in the show, and features his characteristic circles. (He co-founded the Circle Art Group in 1964, a year after his first solo exhibition at the Chatham Gallery.) His full-time job was at the post office.

Chan, meanwhile, worked for Cable & Wireless as a telecommunications engineer and was 42 when he learned how to paint by doing an extra-mural course in art and design at the University of Hong Kong. He subsequently became a founding member, and chairman, of the Hong Kong Visual Arts Society; in 1985, he was awarded an MBE by the British queen.

He's now 91, and in a recent email exchange, explained how lung cancer meant he had to stop doing "oxygen-depriving" work with the acrylic paints of which he had been such a fan. He then moved into what he calls "hybrid art" in the form of digital prints (one of which he composed to illustrate this cyber interview).

"I have no skill to paint neither sweet beautiful things nor ugly subjects," he writes. "I just pick up images from nowhere, when my vision is stimulated by a lump of laundries or stain marks on walls or a sudden exposure to colours and build on that, until they have lives."

Of the past, he writes, "The society was calmer then; even the air was much cleaner! Due to my immobility I can't go anywhere now but wander the huge world in my computer when I'm not sleeping and I still can't stop drawing with a mouse."

Society may have been calmer back then but it wasn't necessarily easier for an artist. Gladys Perint, Hungarian-born but London-bred, arrived in 1972 to marry Simon Palmer, who worked for Swire, whom she'd met skiing in Austria. As she knew hardly anyone in the art world, apart from Brown ("incredibly generous with advice"), she says she spent her time drawing at home and putting her work under the bed.

In 1976, Kim Schmidt, of Quorum Gallery, offered her an exhibition. Quorum, which opened in November 1975 at 11 Wyndham Street, was run by artists on a cooperative basis. The idea, the SCMP remarked approvingly, was "to fill the gap for Hong Kong people who wanted something between oil paintings of junks and traditional scroll paintings". This was evidently high-class modern art: a month after its opening, five paintings were stolen. Chan, Cheung and Chou, who was one of the original founders, all showed there.

By then, GPP, as Perint Palmer signs herself, was doing a column and illustrations for the SCMP and invited its art critic, Nigel Cameron, to look at her work. He immediately told her to give up. Cameron, who is now 94 and unwell, was famously trenchant. He took no hostages, especially when he sniffed out the art of expatriate dabbling: "pretentious nonsense", he wrote of one unfortunate. A review of a Quorum show featuring 21 Hong Kong artists was described as "a clutter of non-events on the walls".

GPP, however, kept going, unlike Quorum, which lasted only a few years. Schmidt was murdered in New York in 1978, by her husband, an American actor called Gig Young, who'd won an Oscar for his role in They Shoot Horses, Don't They? (1969). They'd met in Hong Kong when he was filming a new version of Game of Death, five years after Bruce Lee died during the original production.

Two years later, GPP left the colony for San Francisco, when her husband joined a shipping company. She's now probably the most famous fashion illustrator in the world. (It's a pleasing irony that her drawing in the Ping Pong exhibition is of a nude.) Even if you think you don't know her work, you'll almost certainly have seen it in magazines and newspapers, and her professional spirit still lingers here: she's the artist responsible for the murals at Sevva, in Prince's Building, Central.

Leaving an imprint on the skin of this hectic city, however, has always been a form of artistic roulette. Gerard Henderson's murals are still on view at the Mandarin Oriental, more than half a century after he painted them; Douglas Bland's work in the Hilton and Hong Kong hotels has turned to dust. Bland, who died in 1975, was a manager with Hongkong and Kowloon Wharf. Every evening, according to his son Diarmuid, he came home at 5pm (those were the days!), changed into his "paint-splattered kit" and worked until 7.30pm. His lovely Reflection 9 - neither definably Eastern nor Western but clearly an artwork of the 70s - is on the cover of the booklet Ping Pong has produced for the exhibition. Would Bland have liked the city's newest incarnation as an artistic hub?

"Absolutely," says his son. "He was gregarious, a real character. He and my mother were very much part of the Hong Kong colonial cocktail-party lifestyle of that era. He would have loved the Art Basel scene."

Four years ago, when her art-collecting, art-loving husband died, Brown asked her two sons - one of whom, Ben Brown, has galleries in both Hong Kong (in the Pedder Building) and London - how best he should be remembered. As a result, in 2014, the family set up the Brown Family Annual Acquisition Fund: HK$5 million to be spent by M+, West Kowloon's museum for visual culture, on acquiring works from Art Basel Hong Kong over the next decade. And so the gap between decades is bridged.

"It's not a complete retrospective, lots of people are missing," says Zimmern, of his show. "We're a bar, not a museum. I just want people to have fun looking at them, to see what was here at an interesting time."

"Hanging Out", an exhibition of Hong Kong art of the 1970s, is on at Ping Pong Gintoneria, 129 Second Street, Sai Ying Pun, from 6pm daily until May 15.