Emotion-detecting tech should be restricted by law - AI Now

By Leo Kelion, BBC Technology desk editor

12 December 2019

A leading research centre has called for new laws to restrict the use of emotion-detecting tech. The AI Now Institute says the field is "built on markedly shaky foundations". Despite this, systems are on sale to help vet job seekers, test criminal suspects for signs of deception, and set insurance prices. It wants such software to be banned from use in important decisions that affect people's lives and/or determine their access to opportunities. The US-based body has found support in the UK from the founder of a company developing its own emotional-response technologies - but it cautioned that any restrictions would need to be nuanced enough not to hamper all work being done in the area.

AI Now refers to the technology by its formal name, affect recognition, in its annual report. It says the sector is undergoing a period of significant growth and could already be worth as much as $20bn (£15.3bn). "It claims to read, if you will, our inner-emotional states by interpreting the micro-expressions on our face, the tone of our voice or even the way that we walk," explained co-founder Prof Kate Crawford. "It's being used everywhere, from how do you hire the perfect employee through to assessing patient pain, through to tracking which students seem to be paying attention in class. At the same time as these technologies are being rolled out, large numbers of studies are showing that there is... no substantial evidence that people have this consistent relationship between the emotion that you are feeling and the way that your face looks." Prof Crawford suggested that part of the problem was that some firms were basing their software on the work of Paul Ekman, a psychologist who proposed in the 1960s that there were only six basic emotions expressed via facial emotions. But, she added, subsequent studies had demonstrated there was far greater variability, both in terms of the number of emotional states and the way that people expressed them. "It changes across cultures, across situations, and even across a single day," she said.

AI Now gives several examples of companies that are selling emotion-detecting products, some of which have already responded. Oxygen Forensics was cited for offering emotion-detecting software to the police, but defended its efforts. "The ability to detect emotions, such as anger, stress, or anxiety, provide law-enforcement agencies additional insight when pursuing a large-scale investigation," said its chief operating officer, Lee Reiber. "Ultimately, we believe that responsible application of this technology will be a factor in making the world a safer place." Another example was HireVue, which sells AI-driven video-based tools to recommend which candidates a company should interview. It uses third-party algorithms to detect "emotional engagement" in applicants' micro-expressions to help make its choices. "Many job candidates have benefited from HireVue's technology to help remove the very significant human bias in the existing hiring process," spokeswoman Kim Paone told Reuters news agency. Cogito, which has developed voice-analysis algorithms for call-centre staff to help them detect when customers are becoming distressed, was also mentioned. "Before emotion detection can own making automated decisions, the industry needs more proof that machines can in fact effectively and consistently detect human emotion," its chief executive Joshua Feast told the BBC. "What can be done today, is to evaluate the behaviours that proxy for certain emotions and provide that intelligence to a human to help them make a more informed decision. For tomorrow and the future, it's up to all practitioners and leaders in the field to collaborate, research, and develop solutions that help foster deeper, common understanding that will eventually lead to more connected relationships with one another - not in spite of technology, but because of it." The BBC also asked some of the other named companies for comment, but got no reply.

Emteq - a Brighton-based firm trying to integrate emotion-detecting tech into virtual-reality headsets - was not among those flagged for concern. Its founder said that while today's AI systems could recognise different facial expressions, it was not a simple matter to deduce what the subject's underlying emotional state was. "One needs to understand the context in which the emotional expression is being made," explained Charles Nduka. "For example, a person could be frowning their brow not because they are angry but because they are concentrating or the sun is shining brightly and they are trying to shield their eyes. Context is key, and this is what you can't get just from looking at computer vision mapping of the face." He, too, thought there was need to regulate use of the tech. But he expressed concern that in doing so, lawmakers did not restrict the work he and others were doing to try to use emotion-detecting software in the medical field. "If things are going to be banned, it's very important that people don't throw out the baby with the bathwater," he said.

[It does seems that we (or at least some companies) are determined to automate discrimination and disadvantage based on algorithms – and flawed algorithms at that. If your face doesn’t fit – literally -  based on a computers reading you could fail an interview or even get arrested in some pre-crime style dystopia and just because a computer programme makes a decision and because people believe what’s on a screen in preference to what’s in front of their eyes. We will rue the day that this sort of thing because accepted and in widespread use.]