Real and virtual worlds blur

The fantastic technologies often shown in Hollywood science-fiction productions such as The Matrix may no longer be as fanciful as they seem.

According to a visiting expert from Singapore, many of these technologies will soon become common as mixed reality enables people to play games in combined real and virtual worlds.

David Cheok, an assistant professor at the National University of Singapore, said the technology would extend the digital world to the physical world.

'[Mixed reality] will become an important component of future entertainment computing systems. It allows direct communication between the real world and the virtual world. Humans can interact with computers in a way that goes beyond the desktop computers we have now,' he said.

With mixed reality, a person's movements can be captured in real time and added to a computer-generated environment. Users can also use this augmented reality to interact with others in the virtual world.

Even a single-person environment, such as a book, can be augmented with three-dimensional characters, which you can watch moving from one page to another.

'When you read a Shakespeare play, you will see a 3D character narrate to you over the book. And children can take out a virtual Hello Kitty, put it on another page and build a garden for it,' Mr Cheok said.

He said mixed reality did not require workstations or networks to run such applications. What a user needs are a home personal computer and a virtual-reality headset with attached Webcam, as well as the story software and accompanying book.

Black-and-white markers printed on the pages of the book indicate where the objects will appear and are used to send commands to the headset via the Webcam.

'When the children read the books, instead of viewing the pictures, there are actual 3D objects that pop out of the page. If it is Alice in Wonderland, they can see Alice walking around and talking,' he said.

The Malaysian tourism board has recruited Mr Cheok's team to develop a prototype program to promote the country abroad. When users open the book, they can see a tour guide describing Malaysian history in front of the Twin Towers in Kuala Lumpur.

The technology will have its public debut at this month's Siggraph convention in San Antonio, an annual industry gathering of companies, researchers and animators from the worlds of 3D animation, virtual reality and interactive technology.

Mr Cheok said several companies had expressed an interest in manufacturing mixed-reality computer graphic cards.

He believed the next generation of storybooks would be available next year, priced at as little as HK$80 each.

Meanwhile, it is also possible to have head-mounted see-through displays where the user's real-world view is overlaid with 3D computer graphics. In a room where sensors are installed, users with the mixed-reality headset can play games with virtual creatures which seem to appear alongside them. The system can sense the players' movement, so the virtual characters will move in the same way as the players do.

'Usually the first industries which take up new technologies are the computer entertainment and computer game industries. But more serious applications will be found in governments and the military,' Mr Cheok said.

Mr Cheok's research programme in Singapore was funded by the military, which uses the technology in combat training.

'It is a multiple capturing system that lets soldiers enter the virtual environment. Or you can have a big map on your desk and see little soldiers walking around on it. It will be from a sky view that the commander can watch them like the God in the sky.

'And imagine if George W. Bush would like to meet Tony Blair. They can be captured in 3D and meet as in real life.

'When [the technology] becomes cheap enough, multinational companies will use it instead of having people coming around for a meeting,' he said.