How an opera powered by AI in Hong Kong uses audience’s voices to create unique performances each time
- Finnish opera Laila is held to small audiences of just three or four people in a custom-made dome that projects images and shifting colours from its walls
- Visitors can use gestures to influence the music and also control a group of virtual creatures in a story about life, death, redemption and transformation
Even though the Finnish original opera Laila was conceived back in 2019 before anyone had heard the word “Covid”, it could hardly have been better designed for a pandemic.
The production, one of the most extraordinary works in this year’s Hong Kong Arts Festival – performances of which organisers announced on February 10 would be postponed – is held for small audiences of just three or four people in a custom-made dome, where music is “made” through recorded vocal interactions with the other audience members.
“The whole process of creating Laila was like a premonition of what was going to follow,” says conductor and composer Esa-Pekka Salonen, from his home in Helsinki.
As an audience member you enter the space via a small antechamber, where an AI-powered computer verifies your height, so it can later distinguish you from the other visitors. It then asks you to say some words into a recorder: notions such as love, future, desperation, hope.
Is there gender bias against women in Hong Kong’s performing arts?
“Our theory was that when you start reading these words out loud, you kind of reveal something that you were not expecting to reveal,” Salonen says. It’s a little like the Rorschach ink test, he adds – a way of tapping into your inner psyche and preparing for the event that is about to unravel.
The recorded voice of each person then becomes part of Laila’s sound output. “Every visitor’s voice will stay as part of the white noise up on the ceiling.”
Then the computer voice Laila – a name that contains the modern acronym AI, but which is also one of the most ancient female names known – tells you to go into the dome. As you enter, you hear the first sounds of this strange world and see bright colours on the walls shifting and changing. You also meet a group of virtual creatures that has been assigned to you.
“It’s a bit like in the Greek story when Prometheus makes creatures out of clay,” Salonen says. “At first they’re clumsy and don’t know how to behave, and at this point you learn in a rudimentary way to control them [and the music] by your gestures.”
You also realise that the other audience members are also controlling the movements and the music, and that the interactions between all of you are creating the musical composition that will be unique to this performance.
This is an opera, not a video game, and there is a plot which, without giving too much away, involves the creatures becoming more threatening and less predictable as they grow up, leading to a crisis. And a great change.
Put simply, Salonen says, it tells the great ancient story of life, death, redemption and transformation. “And all this happens in 18 minutes. So there’s no wasting time!”
In the beginning Salonen wanted the chorus to be recorded singer by singer, “because I wanted the option of moving every individual voice around in 360-degree space”.
This turned out to be incredibly data-heavy. “An IT person told me that there wasn’t enough computing capacity in the Northern countries to hold it all, so I had to modify my plan a little,” he says with a laugh.
The production was originally scheduled to premiere in Hong Kong on February 25, but on February 10, festival organisers said the staging of Laila would be rescheduled, and most other in-venue performances cancelled, because of Covid-19 social-distancing restrictions and uncertainty over the availability of venues, which will remain closed until February 23.
The show was also planned to visit several places including San Francisco and Stockholm, but because of the pandemic Laila has so far been performed only once, in Helsinki in August 2020.
“We were the only show in town,” Salonen says. “It became hugely popular, of course.”
After the premiere, he went a few times as a normal visitor, to observe.
“The fun part was that it seemed there was no age bracket. Kids did very well because the video-game aesthetic is very close to what this is. And also some older people could just get the hang of it and do all kinds of things and enjoy the response.”
Then there were others, who seemed almost unwilling to be active participants.
“I guess the basic reason for that is fear. You don’t want to do anything wrong … And if you don’t know what is right and what’s wrong, then you do nothing. If you’re just standing there in the middle and just watching and listening, that’s all fine. But you’ve missed some of the most fun bits.”
The art lovers paying big money to be patrons of China’s private museums
When Salonen was a child in the 1960s, his family used to spend the holidays renting a summer cottage near a farm on an island northwest of Helsinki.
He would take a bucket to the farm and watch someone fill it with fresh milk straight out of the cow. While he was waiting he would watch all the traditional activities of the farm going on around him. One of them was churning milk into butter.
“The part that takes surprisingly long is the part where nothing seems to be happening at all. Then eventually these little grains appear, and all of a sudden there’s this cohesion and the bits and pieces want to stick together like magnets, and that’s the fun bit to watch.”
His own creative process is like that, he says.
He knows some composers who work in a straight line from beginning to end, but for him, he says, it’s much more like churning butter. He keeps jotting down thoughts and notes and trusts that everything will coalesce. So far, it always has.
“It means that I always have something written down, so I never have to face the blank page. Because as we all know, that’s the worst thing of all.”
Laila. Finnish National Opera and Ballet. Date to be confirmed.
