Rana el Kaliouby: ‘We should think of ourselves as stewards of technology’

It is hard to convey empathy on Zoom. After more than a year of relying on video calls and virtual meeting platforms during the pandemic, the limitations of the technology are clear.

But some are trying to redesign this technology, to pick up on subtle social cues and the nuances of human expression — making digital communication closer to the way we interact in person.

“Digital platforms are emotion-blind,” explains Rana el Kaliouby, co-founder and chief executive of Affectiva, the Boston-based artificial intelligence company. It specialises in artificial emotional intelligence, or what it calls Emotion AI: technology that aims to improve how computers interact with humans. 

The 42 year-old Egyptian-American scientist and entrepreneur is based in the US but grew up in Egypt and the Middle East. She studied at Cambridge university then at the Massachusetts Institute of Technology, where she co-founded Affectiva. She is the author of Girl Decoded: A Scientist’s Quest to Reclaim our Humanity by Bringing Emotional Intelligence to Technology.

“Emotional intelligence — or EQ — is a key component of how humans operate,” says el Kaliouby, whose mission is to make machines more emotionally intelligent by linking EQ and IQ.

At present, Most artificial intelligence focuses on mimicking or exceeding the cognitive intelligence of humans, but it does not usually account for emotional intelligence. The idea behind el Kaliouby’s work is that machines can be taught to recognise and respond to human emotions by reading facial expressions, scanning body movements and analysing voices.

Much of human communication is non-verbal; it comes through facial expressions, hand gestures, voice and posture. Much of this is lost when we communicate virtually.

However, through a combination of machine-learning (using data to detect patterns and train computers); computer vision (cameras and microphones), deep learning (a subset of AI using artificial neural networks to carry out advanced pattern recognition), and “tonnes of data”, el Kaliouby says her team has trained algorithms to capture these non-verbal signals in real time, using sensors.

In some cases, this could save lives. For example, it can be used in car safety systems. By using Emotion AI to put “eyes and ears” in cars — cameras and microphones to measure signals such as driver drowsiness or distraction — the company has developed technology to help to keep drivers and passengers safe.

But el Kaliouby believes the tech has broader applications for the ways we live, work and communicate.

“We are all craving the sense of a shared experience, and it’s so hard to do that with digital platforms,” she says. “But the last year has spurred innovation.”

Emotion AI could help teachers better gauge how bored or interested their students are in an online classroom, for instance. Similarly, it could give speakers at an event a clearer picture of how engaged their virtual audience is, by tracking emotional responses — such as smiling, or eye-rolling — as indicators of an emotional or cognitive state.

“When I move to a digital platform my EQ drops,” says el Kaliouby, based on her theory that much of our ability to pick up on non-verbal cues is lost in digital communication. “If a computer or technology can help me be more emotionally intelligent, I would take that.”

As AI grows in sophistication, some critics fear Emotional AI could be misused to manipulate people, eroding trust in technology. For el Kaliouby, this is where ethical responsibility comes in.

“I’m a big advocate of thoughtful regulation as there is huge potential of this technology for positive impact for society, but I also see how it can be abused,” she says. “We should think of ourselves as the stewards of the technology.”

She says Affectiva ensures personal data is only captured with consent, making sure people get some value in return for sharing it — as with the car example.

This has meant turning down business opportunities and lots of funding, she says, given the interest expressed by some governments in buying the technology for surveillance purposes. In a global market, el Kaliouby believes Affectiva’s commitment to protecting users’ data privacy and consent appeals to its clients.

“They want to be on the right side of this equation . . . because of regulation or consumer backlash,” she says.

Cultural perceptions can vary, though. There are differing views in China and the west about what is ethical or fair when it comes to an individual’s right to privacy, and how accurate EQ tools can be. El Kaliouby argues it should be possible to reach common ground: “We can agree on general guidelines . . . We should be able to converge on what a biased algorithm looks like.”

Eric Schmidt, former chief executive of Google, said in 2010 that the company’s policy on many things was “to get right up to the creepy line and not cross it”. For those now involved in emotional and cognitive data sharing, then, the question is: where is that line?

For el Kaliouby, there is a trade-off: “What value am I as a consumer getting for sharing this very personal data?” In the car-safety example, a consumer may view better physical safety — sharing data to make sure they stay alert enough to focus on driving — as a worthwhile trade, she says. But she thinks they would want to know how the data is being used, and to have some form of control.

“Overall we need more transparency. This is where we as consumers have a say.”

Teaching machines to pick up on emotional data also raises questions about what data is inputted. For Affectiva and many AI companies, the priority is to ensure this data is as wide-ranging and diverse as possible.

“If our data is biased, our algorithms are 100 per cent going to be biased,” she warns. “So, if we train our algorithms on middle-aged white men, it’s not going to recognise somebody that looks like me.”

El Kaliouby reflects that, 20 years ago, when she was a PhD student at Cambridge university, the way to evaluate algorithms was to “give the machine some data, train it, give it new data and then assess the results”. 

“That’s not enough any more,” she says. “We have to really break down the results by gender, by age, ethnicity — [to add in, for example] people who wear the hijab or have a beard,” she says. “That’s how you uncover the biases that sneak in.”

Ultimately, she says, the most effective way to remove bias is to recruit developers from a range of different backgrounds and ethnicities, and ensure they are included in decision-making. At Affectiva, the management team’s bonus is tied to implementing processes to remove bias.

Through this work, the team has discovered cultural and gender differences in how people express their emotions.

“It’s a complex issue that makes this field exciting,” says el Kaliouby. “It’s not just, ‘Oh if you smile you’re happy, or if you do that you’re angry’ — humans are complex, which makes it an interesting machine learning problem.”