Talking to a smartphone isn't new. Siri and Google Now listen to specific commands and attempt to give an answer, but what if they were listening constantly? Upcoming smartphones, tablets and other gadgets will come with microphones that respond to a voice command at any time. Meanwhile, music recognition companies like Shazam are turning their attention to personal marketing, using embedded audio in television programmes and advertisements to trigger advertising content on smartphones and tablets.
Further in the future are headsets like Google Glass that listen to our every utterance and, eventually, the mood-sensing television that will listen to your tone of voice, as well as your words, to choose TV programmes and movies from the cloud designed to match your mood. Impressive, but does the constant microphone create a privacy problem?
"It's only just beginning," says Christer Eriksson, regional planning director for advertising consultancy R/GA Singapore. "The new Moto X smartphone can listen to you all the time … but it's just a first step as it's command-based only." The Moto X is all about hands-free control, allowing the user to ditch touch screen navigation for simple voice commands, and it's a feature that's coming to all Android devices. A free app on the Google Play store, Open Mic+, lets all Android users running the new Jelly Bean operating system say "OK Google" - or wave at the phone - to kick-start voice control. Unlike previous attempts, this feature runs in the background so can be enabled any time, whether the screen is on or off. In short, your smartphone is always listening, waiting for your next utterance.
"Once we have devices that observe rather than listen to commands, then we're talking about a really smart device," says Eriksson. "Google Now is just starting on that trend by analysing your calendars and bookings, but add listening and you're into something special - or scary, depending on how you look at it."
More and more gadgets exist that potentially expose our privacy Already anyone can put their smartphone in a top pocket and leave the video camera running to capture every moment of a day - or even a life. There's a natural fear of being spied on, but it's debatable whether the smartphone or camera that's always on threatens privacy any more than existing technology.
"I can sit in a cafe and sniff the internet packets of the people using the free Wi-fi," says Nigel Cannings, the CTO of Intelligent Voice, which does a lot of work in the "sentiment analysis" side of voice recognition - the all-important emotional meaning of a conversation. "If I'm feeling really ingenious, with not too much work I can persuade the mobile phone of any person in a 100-metre radius to connect to my 'network', and listen to every call they make," he says.
It's just as easy to secretly record any meeting or conversation just by leaving a smartphone on the desk with the voice recorder on. Such intrusive activities will go to a new level when Google Glass goes on sale; its core feature will be a camera on the front that can take time-stamped, GPS-tagged photos and video of whatever the user is seeing, and share them instantly.
"They have the potential to be one of the most invasive technologies ever created," says Kevin Curran, senior member at the Institute of Electrical and Electronics Engineers (IEEE).
"The ability to record video and images without the knowledge or permission of those within view will lead to much debate … and already some bars, strip joints and restaurants have announced they are banning the wearing of smart glasses in their establishments."
The likes of Apple, Baidu, Microsoft, Sony and Samsung are set to reveal rival smart glass devices though whether they become socially acceptable will depend on personal preferences and experiences. "Some will not find them intruding on their privacy while others will always have their guard up when faced with a Glass user," Curran says.
"The issue is not so much what the capture method is, but what you do with it," says Cannings, who thinks Google Maps' Street View feature is invasive. "Are you like Google where you go wardriving while mapping the streets, sucking-in and retaining vast amounts of personal data? Or do you just listen to what is relevant, and act on that?"
It depends not only on who is listening, but who is talking, too. In a world of always-on microphones, each device needs to know who's the master. "You don't actually want to act on the instructions of the wrong person, so it is key that the person whose voice you capture is the one you want," says Cannings. "We work mainly in the corporate sector, and a key element for us is we make sure that when we analyse a conversation, the system has correctly identified the speaker using our voiceprint technology. You don't want to rely upon a device ID or phone number, because it could be used by a different person."
More of a problem of the always-on smartphone in the home could be the unwanted actions of marketeers trying to personalise their adverts still further. From its beginnings as a text message service that could answer the perennial question 'what's that song?', Shazam is becoming a constant listening service. "The Shazam app is no longer just for identifying music - in the US, Shazam has evolved its business to essentially be an aural QR [quick response] code for any type of programming, now including advertising and television," says Fiona Harkin from Stylus, a research and advisory firm for creative industries. "The Shazam for TV initiative allows viewers to unlock special messages and additional content from both adverts and television shows."
Shazam for TV uses the same technology as the music recognition service, but instead of a series of recorded music cross-referenced with a database of songs, the same process produces extra adverts or discount codes. The problem with this is that the viewer needs to launch the Shazam app during the advert; they usually have a Shazam logo in the corner, but by the time you have your phone out of your pocket the advert has invariably already finished. The microphone that's never off solves this, but to the benefit of whom?
"While smartphones can already listen to what we're watching on TV, most people aren't interested and the rest of us are doing something more important on our other devices, or watching content," says Matt Oxley, head of creative technology at advertising firm Tribal DDB London. "Second screen behaviour has the opportunity to flourish, but only if content creators focus on the experiences of the individual and the group, not using watermarking or other technology as glorified QR codes for advertisers."
It may be misjudged as an advertising medium, but audio watermarking could provide an instant link to useful extra content on YouTube, or a link to download a show to a tablet or smartphone.
The same technology could also help smartphone owners join up the gadgets in their home for ever more amazing entertainment experiences.
"What other products in our home will also be listening to what we watch on TV?" asks Steven Kalifowitz, group director of production at R/GA Singapore. "Might a TV programme send a signal that light bulbs like Philips' Hue Connected Bulb can receive, causing them to automatically change colour and ambience of the room you're in?"
We've entered the so-called Internet of Things, where anything is possible - but only if everything is switched-on and listening.