How Google Glass can improve autistic children’s social skills by reading facial expressions
Children with autism have difficulty reading social cues and discerning other people’s emotions. Studies have shown that using Google Glass improves their ability to read facial expressions, making it an ideal tool in places such as China that have few therapists available
Google Glass – the once globally hyped smart glasses – seemed to have slipped off the radar after sales were suspended in 2015, just three years after they were launched. Now it is being hailed as a life-changing device for people with autism spectrum disorder (ASD).
People with autism have trouble with social skills, and verbal and non-verbal communication.
Researchers at Stanford University have harnessed Google Glass to develop a form of self-guided therapy that families can use to coach an autistic child to read emotions in faces, ultimately improving their ability to interact with others.
Catalin Voss is the founder of the Autism Glass Project. The Stanford School of Medicine graduate student spoke at the EmTech Hong Kong 2018 conference in Hong Kong this month about his team’s augmented reality therapy that taps the Google Glass technology.
“The goal is to give a learning aid to kids and families,” explains the entrepreneur, who had previously sold his start-up Sension, a face- and eye-tracking-based innovation that discerns facial expressions, to a Toyota-owned company.
“In general, it has improved eye contact, increased emotional recognition abilities and increased engagement in the family,” he says.
First Hong Kong run for play specially adapted for autistic children
One of those trials involved 14 families who used the technology at home over a six-week period. The participants used the device for 20 minutes, three times a week. Researchers then analysed participants’ interactions.
“What we found is more complex than just an emotional intervention,” says Voss. Initially the researchers thought the technology would teach kids how to better recognise emotions such as “happy” and “sad”, but they discovered the process also sparked more conversations within the family about emotions.
“It helped kids realise there is something for them to see in faces, so they look at faces, and it sort of captures engagement,” he says. The research will be published in Nature Partner Journal’s Digital Medicine.
Several parents were stunned by changes, Voss says. “We refer to them as a ‘Light Switch Group,’ families that emailed us saying, ‘This is like a switch has been turned on [in their child] … what happened?’”
The device coaches users on eight emotions such as happiness, sadness, anger, fear and disgust, all based on psychologist Paul Ekman’s theory that these facial expressions show the basic human emotions. “Our theory was: if we can teach and give them confidence to [read these emotions], it would unlock their ability to learn more complex emotions,” says Voss.
Therapy that gives Hong Kong’s autistic children a fighting chance
The app they developed classifies the streamed images from the Google Glass, via a machine learning-based system, and beams information back to the user through the headset in the form of an emoji in a colour associated with the emotion. Audio feedback is attached to the headgear so the user hears a robotic voice saying, “Happy,” for example, if that facial expression has been recognised.
No known cure exists for autism. Symptoms are detectable in children as young as two years in the form of difficulties in learning, communication, and social interaction. A hallmark of this condition is difficulty in identify emotions.
As a result, many parents join a waiting list for government-subsidised care, which can take from 12 to 24 months, according to the Rainbow Project’s website, an autism charity group in Hong Kong.
Autism group calls on Hong Kong government to offer more support services
The predicament is similar in China. Although figures vary, the prevalence of autism in China is estimated to be one per cent, or more than 13 million people.
“My guess is autism in China is probably under diagnosed, and even if you ramp up to get really solid diagnoses done to catch the kids early, you wouldn’t have the resources to take care of them,” remarks Voss.
“There aren’t enough therapists to take care of them so there is a crucial need to give tools to families they can use on their own while they are on a 12-month waiting list to see a therapist, or drive say five hours to see one,” he says.
Existing therapies include applied behaviour analysis that uses flash cards to coach autistic learners to identify emotions. Voss says such therapy is done in an isolated setting which makes it difficult for kids to translate insights in real-world settings.
His team is working on a licensing deal with an undisclosed company to get the device to market, hopefully in two years.
Asked about the cost – the original Google Glass had a US$1,500 price tag – Voss says ideally one would rent the device rather than own it, and the cost could be recovered through medical insurance. “Once you get an autism diagnosis, you should be eligible for guided therapy at home while you wait for therapy,” he says, adding that by the time you finally have access to a therapist, the autistic child would have made considerable progress.
Keith Lee, a project director at the Rainbow Project Learning Centre, says children with autism do respond well and sometimes better with visual aids than from reading words or listening to instructions. He cautions that the technology must be tested with a larger group with a range of degrees of autism, adding, “it’s hard to say if it will work for every kid, as everyone is different”.
Autism is a condition with complex and lifelong needs, he says, adding that it is hard to meet those needs as autistic children mature to adulthood. Many in Hong Kong have nowhere to go, so they stay at home, putting a lot of pressure on parents – especially those whose children display impulsive or self-harming behaviour and who may unintentionally hurt others.
“If they can find a place for their adult child to take part in meaningful activities, parents can have a break and have a better quality of life.”
