Now, a software to read moods via tone of voice
IN A YouTube clip from one of the late Apple co-founder Steve Jobs' last interviews, he appeared to be enjoying reminiscing about how he first came up with the idea for a keyboard-less tablet, which eventually became the iPad.
"I had this idea of being able to get rid of the keyboard, type on a multitouch glass display and I asked our folks, could we come up with a multitouch display that I could type on, I could rest my hands on and actually type on?" Mr Jobs said, smiling slightly as he recounted his enthusiasm at seeing the iPad prototype. "It was amazing."
But in a billboard superimposed over the nearly two-minute video clip, an emotion-analytics company called Beyond Verbal has added its own algorithmic evaluation of Mr Jobs' underlying feelings.
It is an emotion-detection system meant to parse not the meanings of people's words, but the intonations of their voices.
As he spoke, the ticker above Mr Jobs' head reports: "Conflict between urges and self-control. Loneliness, fatigue, emotional frustration."
Moments later, it suggests a further diagnosis: "Insistence, stubbornness. Possibly childish egoism."
And then concludes: "Sadness mixed with happiness. Possibly nostalgia."
Humans generally have inklings when their interlocutors, out of solicitousness or sarcasm, utter phrases aloud that contradict their inner feelings.
But now, new techniques in computational voice analysis are promising to help machines identify when smiley-sounding phrases belie frustration and grief within.
Although the software is still in its early phases, developers like Beyond Verbal, a start-up in Tel Aviv, are offering the nascent technology to call centres and other customer services as a deeper approach to reading and responding to consumers' emotions in real time.
Beyond Verbal said its software can detect 400 variations of different moods.
Mr Dan Emodi, its vice-president for marketing, said: "It's not what you say. It's how you say it. Listening to these patterns, we can allow machines to, for the first time, understand the emotional side of our communications."
The more invasive audio-mining technique also has the potential to unnerve some consumers, who might squirm at the idea of an unknown operator getting an instant entree into their psyche.
Industry analysts said firms that adopt emotion detection should be transparent with consumers, alerting them to the use and analysis of their data.
Another question is whether emotion detection is any more valid than novelties like handwriting analysis.
Dr George Loewenstein, a professor of economics and psychology at Carnegie Mellon University, said: "It seems to me that the biggest risk of this technology is not that it violates people's privacy, but that companies might believe in it and use it to make judgments about customers or potential employees.
"That could end up being used to make arbitrary and potentially discriminatory decisions."
In the early days of telephone recordings, call centres archived calls and reviewed a handful of them, examining the conversation patterns and giving agents feedback on their performance.
But as software and server power have improved, call centres are now using a more advanced approach called "word spotting" to examine each call.
Call centres, for instance, can program their speech engines to search for specific words or phrases - like, "This is the third time I have called in!" or "I've been a loyal customer for 10 years!" - which tend to be emotionally charged, indicating mounting consumer dissatisfaction.
Audio mining helps call-centre agents decide how to respond, said Mr Emodi. If someone is a customer-is-always-right type of client, the agent would want to give him proper appreciation and respect.
He said: "If the caller is seeking friendship, the agent should speak in a friendly, direct way."