Future tech

Forget Pokemons – in world first, Hongkonger applies augmented reality to surgery

Medical graduate Catherine Chan, armed with a master’s degree in medical imaging, is fine-tuning AR application that solves the biggest problem in keyhole surgery – having to look at images on two screens simultaneously

PUBLISHED : Thursday, 25 August, 2016, 12:02pm
UPDATED : Thursday, 25 August, 2016, 1:12pm

A Chinese University of Hong Kong graduate has pioneered the use of augmented reality for medical imaging during minimally invasive surgery, removing the need to look at multiple images which can be a distraction during surgery.

Catherine Chan Po-ling, an orthopaedic surgeon at Prince of Wales Hospital in Sha Tin, says the pioneering technology has solved one of the limitations of minimally invasive, or keyhole, surgery that has dogged surgeons.

Demonstrating with a pig liver, she explains: “To check whether the liver has cancerous cells, a doctor has to use an ultrasound probe to see beneath the surface of the liver. There are two cameras showing images. One, from an ordinary camera, shows the surface of the liver. The other is the camera from the ultrasound probe which shows the same black-and-white images expectant mothers see of their fetuses. The doctor has to look at two images on two screens a while manoeuvring two probes at the same time. This requires a lot of hand-eye coordination. This has long been the biggest problem regarding minimally invasive surgery.”

In the hit augmented reality game Pokemon Go, a digital cartoon monster image is laid over real images. With the new technology pioneered by Chan, augmented reality combines the images from the ordinary camera and the ultrasound probe using computer vision tracking technology.

“While monsters pop up on phone screens of Pokemom Go players, combined real-time images of the probed body parts pop up on the screen.”

Chan was the first graduate sent by the university to take a master’s degree in medical robotics and image-guided intervention at Imperial College London last year. It was part of a move by the university to boost collaboration between the medical engineering and medical fields after the establishment of the Chow Yuk Ho Technology Centre for Innovative Medicine last year. The AR software was written by engineers at Imperial College, says Chan, “but they just invented the software without any clinical input. Among my 14 classmates at the college, only three [including me] have a background in medicine. Representing Chinese University and working under the guidance of my supervisors at Chinese University, we are the first in the world to put AR technology into medical applications.

“The technology is far from perfect now. The Imperial College team is improving the software. We have to improve the image stability. In two years’ time, we will put it into clinical use. At the moment we are using pigs’ livers for research. Later we will switch to human cadavers and use the system on live patients to collect data. In future, this AR technology can be applied to all minimally invasive surgeries in the same way as the Da Vinci robot arm surgical system.”

Chan has been keen on engineering since she was young.

“Since childhood, I have wanted to become an engineer. Imperial College gave me an offer to study mechanical engineering before. But I chose to study medicine at Chinese University as my parents think that being a doctor is a better career path.”

While at the college last year, she co-founded a start-up called MedEXO Robotics in the UK with a physics graduate from the University of Hong Kong. Among their inventions is a robotic device to reduce hand shaking in patients with Parkinson’s disease. The invention recently won HK$450,000 from a start-up competition organised by DBS Bank.

Chan says there’s a need to boost technological invention in medicine. “Medicine has a lot fewer technological discoveries if you compare it to the industrial sector which has pioneered many breakthroughs like self-driving cars.”