On August 22, 2022, the Federal Trade Commission published an advance notice of proposed rulemaking (ANPR) requesting public comment on an intended Trade Regulation Rule on Commercial Surveillance and Data Security. 3 The Knowing Machines Research Project ("Knowing Machines") appreciates the opportunity to comment. 4 In response to ANPR Questions 37., 5 38., 6 55., 60., and 76., 9 we urge the Commission to consider new rules that forbid or limit the development, design, and use of a biometric commercial surveillance practice known as "emotion recognition." 10 Emotion recognition technologies claim to be able to detect a person's internal emotional state based on algorithmic systems trained to analyze facial expressions that appear in photo or video content. These systems are based on the dual premises that (1) facial expressions map onto a limited, universal set of emotions, and (2) algorithmic systems, including machine-learning (ML) tools, can accurately gauge consumers' feelings to serve commercial ends.
While both these premises lack scientific support, the harms these systems pose to consumers are real. Emotion recognition companies deceive consumers with misleading and unsubstantiated claims about what their tools can do, a prototypical "faulty conclusion" flaw fueled by failures in experimental theory, methodology, and design. 11 These tools are unfair to consumers, especially people of color and people with disabilities (PWD) who are likely to bear substantial, unavoidable injury without any countervailing benefits to FTC ANPR, R111004 Emotion Recognition November 21, 2022 3 consumers or competition. These tools also threaten consumer privacy by encouraging the surveillance and assessment of our intimate, private emotions. With this Comment, the Knowing Machines Project asks the Commission to investigate and, where appropriate, prohibit the deceptive and unfair practices that emotion recognition companies conduct. To aid the Commission's consideration, we have collected several marketing claims and captured images from promotional materials depicting emotion recognition tools, some of which are described in Section II.B. This is not a comprehensive survey and is offered merely to demonstrate claims made by companies selling these tools. It can be found at the end of this submission in "Appendix: Companies' Marketing Claims and Depictions of Emotion Recognition Tools."
The Commission should consider new rules that forbid or limit the development, design, and use of commercial emotion recognition tools that lack scientific support and harm consumers.
For decades, experts have grappled with many open questions concerning human emotions: What are they? Why do we have them? Can we categorize them? Are they static or in a constant state of flux? And how can we know what ourselves and others are feeling?
While these questions continue to be debated, one controversial theory has emerged to shape major aspects of the field of emotion recognition. In the 1960s and 70s, psychologist Paul Ekman theorized that all humans exhibit a small number of universal emotions that are categorical, innate, cross-cultural, and uniform. 13 Ekman argued that using a series of observational techniques, we can know what people were feeling based on their facial expressions alone, an idea he developed based on Charles Darwin's observation that primates make similar facial expressions and that expressions serve an adaptive function. 14 Ekman categorized six, fixed emotions-happiness, sadness, anger, fear, surprise, and disgust. Countless other emotions like shame or guilt fell out of this taxonomy because they are not facially expressed, while those six "do have expressions, which meant that they were amenable to study." 15 This became the perfect model for the modern approach to machine learning for emotion recognition, a universe comprised of unstructured data with just six viable classifications for them to be sorted into. As one of us notes, "[t]he theory fit what the tools could do." 16 Ekman was also inspired by the earlier work of psychologist Silvan Tomkins, who paired basic emotions with distinct facial expressions and believed that emotion was an innate set of evolutionary responses. 17 While he assumed that facial expressions were FTC ANPR, R111004 Emotion Recognition November 21, 2022 5 claims for things there is no evidence for." 23 Yet his research still undergirds such tools being developed and deployed on consumers with increasing regularity. 24 B. The market for emotion recognition technologies is growing, with business customers deploying these tools on increasingly diverse sets of consumers.
By one estimate, the global market for emotion recognition will grow by almost 13% from $23.6 billion this year to $43.3 billion by 2027. 25 Tech companies big and small participate in this market, producing tools found in education, hiring and recruitment, mental health units, for marketing, on video calls, at national borders, in predictive policing programs, and even in courtrooms. Their tools rely on the idea that emotions can be detected automatically by training algorithms to measure facial expressions. 26 And their tools affect an increasingly wide range of consumers.
Over the past few years, several tech giants like Google, Amazon, and Microsoft have developed technologies that claim to recognize a limited set of emotions from consumers' facial expressions. 27 Only one-Microsoft-has since removed its service. 28 Amazon sells a facial recognition service as part of a suite of image-analysis features called Rekognition. In addition to claims to recognizing faces, Rekognition analyzes images for eight emotional states: happy, sad, angry, surprised, disgusted, calm, confused, and "fear." If a business customer processes more than 1 million images, the cost of running Rekognition's algorithms is less than $0.001 per image. While online resources for Rekognition warn the service "is not a determination of the person's internal emotional state and should not be used in such a way," Amazon's webpage suggests that stores could use Rekognition to analyze and track consumers' emotional states in real time at different locations over time. 29 FTC ANPR, R111004 Emotion Recognition November 21, 2022
Since 2016, Google has offered an emotion-detection feature in its AI cloud service that claims to detect and analyze faces in photos uploaded to its platforms, including personal photos, estimating age, gender, and four emotions: joy, sorrow, anger, and surprise. 30 Users can also search for emotions like "happiness," surprise," or "anger" in Google's Photos app. This feature survived the same internal ethics review in 2021 that prompted Google to remove facial recognition capabilities. In that review, Google decided against expanding the feature to be able to detect additional emotions after "determin[ing] that inferring emotions could be insensitive because facial cues are associated differently with feelings across cultures, among other reasons." 31 Until June 21, 2022, Microsoft offered an emotion recognition service that claimed to label expressions as anger, contempt, disgust, fear, happiness, neutral, sadness, or surprise. 32 The tool was able to provide real-time analysis of a crowd, assigning to each individual a unique serial number, estimated gender and age in addition to detected emotions. 33 As part of Microsoft's multiyear-long review of their AI products, however, the company ended its emotion recognition service to general users. 34 Microsoft's Chief Responsible AI Officer Natasha Crampton explained that numerous experts have "highlighted the lack of scientific consensus on the definition of 'emotions,' the challenges in how inferences generalize across use cases, regions, and demographics, and the heightened privacy concerns around this type of capability." 35 Crampton raised reliability concerns stemming from the "huge amount of cultural and geographic and individual variation in the way in which we express ourselves." 36 Microsoft was also concerned about privacy harms and the ways in which emotion recognition could be misused to discriminate unfairly against consumers. 37 Ultimately, Microsoft chose to regulate itself in part due to a lack of government oversight of AI systems. 38 Several other companies are developing emotion recognition tools that could affect a wide range of consumers. In schools, companies like Intel and Hong Kong-based 4 Little Trees provide tools that can run on top of video conferencing software, analyzing students' faces to allegedly detect whether students are bored, distracted, or confused via their facial expressions; 39 for example, 4 Little Trees claims to detect Ekman's six emotions, as well as a student's motivation and forecasting their grades. 40 Intel developed its service by using data gathered from students in real-life classroom settings. It then hired psychologists to view videos of students and categorize the emotions they detected in their expressions based on predetermined categories-a method referred to as a "forced-choice" response format, where subjects are alerted to the connections that study designers have already made between certain expressions and emotions. 41 But emotion recognition tools in schools assume that all students learn in similar ways, reflecting their amount of focus, motivation, stress, and attention in their facial movements. One study found that although such tools might accurately detect "obvious emotions such as happiness," they tend to misidentify a frown of deep focus with a frown of anger or sadness. 42 Workers are also impacted by emotion recognition tools. In response to increased scrutiny following an FTC Complaint filed by the Electronic Information Privacy Center in 2019, the leading interview software provider HireVue removed its facial emotion recognition feature. 43 The company "concluded that for the significant majority of jobs and industries, visual analysis has far less correlation to job performance than other elements of our algorithmic assessment." 44 HireVue's emotion recognition tool was used by hundreds of companies, turning its service into a gatekeeper for countless jobs and encouraging consumers to maximize the facial gestures most likely to land them a job. 45 HireVue's are of particular concern for researchers, who consider analysis of candidate videos for emotional "fit" as "pseudoscience." 46 In one study, computer scientists built their own simplified AI recruitment tool and trained the algorithm to analyze candidates' photos for five personality traits; they found that the personality score changed when the photo's contrast, brightness, or saturation were altered. 47 In another study, researchers tested another tool from Retorio, a German AI hiring startup, and found that the algorithm responded differently to the same candidate in different outfits, including glasses and headscarves. 48 Beyond schools and workplaces, emotion recognition tools are increasingly deployed in mental health products to detect depression, in personal cars to detect "road rage", and at national borders, in courtrooms, and by law enforcement to detect deception.
After buying an emotion recognition startup back in 2016, Apple has recently revealed its intent to help diagnose depression and cognitive decline, as well as autism in children, through an array of iPhone and Apple Watch-based behavioral data, including users' facial expressions. 49 While Apple's venture is currently confined to an academic study subject to strict rules concerning transparency and participant consent, the company's interests could evolve into consumer-facing emotion recognition deployed from anyone's iPhone or Apple Watch. In a similar vein, one new emotion recognition startup has partnered with researchers at several large hospitals to see whether its ML models can identify symptoms of depression and schizophrenia. 50 Facial expressions are normally part of a complex, multifactorial formulation of a patient's condition(s) considered by psychiatrists, while emotion recognition tools overlayed on facial recognition services are limited to reading facial expressions alone. As companies increasingly look to emotion FTC ANPR, R111004 Emotion Recognition November 21, 2022 9 recognition solutions to diagnose and treat complex mental health conditions, practitioners are sounding the alarm on the limited reliability of these tools in this sensitive context. 51 In 2018, emotion recognition startup Affectiva launched an emotion tracking system for drivers. Its service uses AI models to identify drowsiness, yawning, and other signs of driver fatigue, as well as signs of road rage. Its CEO has stated she has hopes of using its service to link together home-based smart assistants like Alexa and Siri with consumers' cars to make an emotional profile "for better user experiences." 52 Tech vendors like Affectiva and Visteon are developing these "interior sensing" features in part responding to the European Commission's requirement for mandatory distraction-detection systems in all vehicles and to the U.S. Infrastructure Investment and Jobs Act, which mandates the use of technology to monitor for signs of drunk driving-related impairment. 53 Emotion recognition tools focused on detecting deception are increasingly used by border officials internationally, enabling these automated processes to influence important decisions over the mobility of travelers and migrants. From 2016 to 2019, Hungary, Latvia, and Greece piloted a lie-detection tool called iBorderCtrl that analyzed the facial microgestures of travelers crossing the borders at three undisclosed airports. The point of the program was to see if iBorderCtrl could detect individuals lying about the purpose of their trip. 54 Specifically, the tool searched for "biomarkers of deceit," nonverbal facial expressions that presumably all humans make when lying; the emotion detection component looked for 38 such biomarkers, such as blinking left eye, increasing facial redness, head movement in certain directions, and even gender. 55 Additionally, the data set that trained iBorderCtrl's algorithm was comprised of 32 actors who answered 13 questions as "truthful" or "deceptive" people, with 22 actors of White European ethnicity and 10 actors of Asian/Arabic ethnicity (and 22 men, 10 women). 56 While the goal of iBorderCtrl and similar programs may be to discern lying travelers to prevent illegal border crossings, the assumptions that fuel this tool and the limited makeup of its training data cut against its supposed accuracy and validity when applied to large swaths of travelers. The assumption that there are such biomarkers of deceit is based on Ekman's theory that lying is an emotionally-demanding task that leaves behavioral traces on our faces. But research testing that assumption has found that using these biomarkers to tell whether someone is lying is as accurate as randomly guessing. 57 Additionally, the small number of samples for the data set FTC ANPR, R111004 Emotion Recognition November 21, 2022 10 and its uneven ethnic and gender representations undermines the model's validity when applied to new individuals whose data was not a part of the training process. 58 Another realm for emotion recognition is the courtroom. Companies are developing ML tools to identify and classify facial expressions, again looking for signs of deception. 59 Like iBorderCtrl, these tools are trained on data sets containing actors pretending to be truthful or not, and their algorithms focus on predetermined indicators of deceit like "lips protruded" and "eyebrows frown." 60 One company has sold its emotion recognition software, EmotionTrac, to lawyers nationwide who use it in civil and criminal courts. EmotionTrac's facial action coding identifies 100 points on a person's face and analyzes how those points change to calculate discrete emotions. The tool is primarily for use on mock jurors to gauge the effectiveness of lawyers' arguments. 61 EmotionTrac partnered with a large legal technology solutions company to accelerate adoption among lawyers and legal departments in the company's consortium of 20 global law firms. 62 As a final example, law enforcement agencies are also looking to emotion recognition to predict someone's criminality. Oxygen Forensics, which sells software that the FBI and other agencies use to extract data from smartphones, has added facial recognition and emotion detection to its product. Its emotion detection feature enables officers to filter faces by race or age group, as well as emotions such as "joy" and "anger." 63 The software is provided through a startup called Rank One, whose CEO claimed he was unaware that Oxygen Forensics had implemented emotion recognition in addition to facial recognition. 64 And in the U.K., a police force tested a system recently that would enable officers to search live CCTV footage for specific emotion-based parameters, including mood and facial expression. 65 These instances suggest that emotion recognition is creeping into public spaces, affecting entire populations in inescapable and concerning ways. 66 C. Commercial emotion recognition tools are based on deceptive claims to business customers, result in unfair harms to consumers, and pose a serious threat to consumer privacy.
Section 5 of the FTC Act is a powerful consumer protection tool in part because it applies to business customers as well as individual consumers. 67 Companies selling emotion 58 Id. at 421-22; see also infra II.C.1. (discussing related data set issues).
See Michael Byrne, AI System Detects 'Deception' in Courtroom Videos, VICE (Dec. 19, 2017), https://www.vice.com/en/article/zmqv7x/ai-system-detects-deception-in-courtroom-videos. 60 Id. 61 Jim Nash, Emotion Recognition AI Finding Fans Among Lawyers Swaying Juries and Potential Clients, BIOMETRICUPDATE.COM (May 20, 2022), https://www.biometricupdate.com/202205/emotion-recognitionai-finding-fans-among-lawyers-swaying-juries-and-potential-clients. 62 Reynen Court Partners with EmotionTrac to Expand eDiscovery Software Offerings, PR WEB (Aug. 4, 2021), https://www.prweb.com/releases/2021/8/prweb18104674.htm. 63 Simonite, supra note 29. 64 Id. 65 Fiona Hamilton, Police Facial Recognition Robot Identifies Anger and Distress, LONDON TIMES (Aug. 15, 2020), https://www.thetimes.co.uk/article/police-facial-recognition-robot-identifies-anger-and-distress-65h0xfrkg. 66 recognition tools harm both businesses and individuals when they overpromise what the technology can do, make unsubstantiated claims, and profit off tools that lead to unfair and discriminatory results. Many companies sell their tools directly to business customers in the retail, human resources, education, and medical industries. These businesses are materially harmed when they rely on deceptive claims to purchase and deploy these tools. Individual consumers like shoppers, job candidates, students, and patients are harmed in turn by the unfair and discriminatory results that flow from business customers' use. And as the emotion recognition market grows, these harmful tools could facilitate increased biometric surveillance that affects all consumers by diminishing their privacy rights. Fortunately, Section 5 empowers the Commission to address the deceptive, unfair, and invasive practices that harm consumers of all stripes.
1. Emotion recognition companies mislead reasonable consumersbusiness customers-in a material way by promising that their tools accurately and automatically identify discrete emotions based on facial features.
The Commission can use its deception authority "in connection with algorithmic harms where the marketers of products . . . represent that they can use machine-learning technology in unsubstantiated ways." 68 Commissioner Slaughter has drawn attention to a particular algorithmic harm referred to as "faulty conclusions," which involves "the feeding of data into algorithms that generate conclusions that are inaccurate or misleading." 69 To avoid deceiving customers, AI companies need to ensure their marketing claims are "truthful, non-deceptive, and backed up by evidence." 70 They must avoid "exaggerat [ing] what [their] algorithm can do or whether it can deliver fair or unbiased results" and "overpromise [ing] what [their] algorithm can deliver" or risk an FTC enforcement action. 71 Companies selling emotion recognition tools have missed this message. They claim their technologies can "predict human emotions," 72 "interpret[] emotions the same way humans do," 73 and "Mak[e] Emotion Recognition an Exact Science." 74 They boast that their large data sets are essential "to train and evaluate unbiased empathetic technologies," 75 race, color, religion, nationality, age, and gender," 77 and their tools "measure . . . customers' unfiltered and unbiased emotional and cognitive responses, unobtrusively and at scale" 78 and "provide deep insight into unbiased emotional responses." 79 They frequently substantiate their claims using Paul Ekman's theory that there are only a handful of universal emotions, and rarely do they cite anything else in the literature. 80 While a few companies provide disclaimers that their tools only detect facial movements and not peoples' actual emotional states, most do not; even those who do still claim to accurately detect emotional states with their tools and sell that presumed capability to consumers. 81 When companies claim their services can accurately identify a person's emotions, they push a deceptive practice under Section 5 of the FTC Act. Their claims are likely to mislead business customers and others who reasonably believe them, and materially harm business customers who would not have paid for these tools if they knew they lacked scientific validity and reliability.
At its core, emotion recognition tools sell an idea that discrete emotions can be measured and identified by various technologies based on facial features alone-but this is unfounded at best and disproven at worst. The scientific community, including Paul Ekman, has called such technologies into question. Through various studies, researchers have demonstrated the dearth of evidence supporting these tools. A recent survey of hundreds of emotion recognition studies found that the science did not strongly support the idea that certain emotions can be accurately and reliably expressed through facial expressions. 82 There is no scientific consensus on what an emotion is, if and how certain emotions are expressed, and whether facial gestures are at all connected to emotional states. Despite companies' claims of diverse, cross-cultural data sets feeding unbiased algorithms, these tools assess facial movements without accounting for cultural and social context, a necessary element of detecting emotional responses. And they are built off faulty 77 EMOTIONTRAC, Welcome to EmotionTrac, VIMEO (Mar. 2, 2022), https://vimeo.com/683839685. 78 AFFECTIVA, Affectiva Media Analytics, https://go.affectiva.com/affdex-for-market-research (last visited Nov. 21, 2022). 79 REALEYES, supra note 73. 80 See Appendix, infra at 25-29 (gathering references to "universal," "basic," or "primary" emotions and Ekman's work in marketing materials). 81 Compare AMAZON WEB SERVS., FaceDetail, (last visited Nov. 21, 2022) ("The [emotion detection] API is only making a determination of the physical appearance of a person's face. It is not a determination of the person's internal emotional state and should not be used in such a way. For example, a person pretending to have a sad face might not be sad emotionally"), and NOLDUS, Ethics in Facial Expression Analysis, https://www.noldus.com/about-noldus/ethics-facial-expression-analysis (last visited Nov. 21, 2022) ("It is also impossible to use FaceReader as a 'lie detector.' Nevertheless, we fully understand that there can be some discomfort about technology that is capable of analyzing facial expressions. Therefore, Noldus disapproves of the use of FaceReader without informed consent from the individuals whose facial expressions are being captured"), with João Aragão Pereira, Henrique Fugita, & Rafael Werneck, Liveness Detection to Improve Fraud Prevention in Financial Institutions with Amazon Rekognition, AMAZON WEB SERVS. (Oct. 22, 2022), https://aws.amazon.com/blogs/industries/liveness-detection-to-improve-fraud-prevention-in-financialinstitutions-with-amazon-rekognition/ (providing image of Amazon Rekognition that detects the percentages of "Happy" and "Surprised" emotions expressed by the subject in the context of selling Rekognition to financial institutions), and NOLDUS, Baby FaceReader, https://www.noldus.com/facereader/baby-facereader (last visited Nov. 21, 2022) (advertising facial emotion recognition tool for infants from age 6 to 24 months, a consumer population that cannot provide informed consent). 82 methodologies that use posed photographs, forced-choice responses, and a narrow focus on a handful of emotions that assumes one facial expression for each coded emotion. Some of the most basic assumptions underlying these marketing claims lack scientific evidence. The first assumption is that emotions fall into discrete categories with clear boundaries that can be assessed based on corresponding facial movements. There are multiple theories about whether emotions are fixed, discrete states or are more continuous processes, however. 83 A more fluid view of emotionality challenges the notion that emotions are easily classified and detectable, undermining the ability of companies to train algorithms to categorize and label distinct emotions. 84 Even the term "recognition" assumes "emotional categories are given, rather than emergent and relational." 85 Psychologists bristle at this reduction of emotion, as "the idea of collapsing the richness of human feelings into a handful of categories for all people and contexts doesn't make much sense" and leads companies to "be overconfident about what computers can do." 86 Emotion recognition researchers also challenge the claim that these tools can actually assess an individual person's emotional state. They suggest such tools merely estimate how one's emotions might be perceived by others or suggest broad, group-based trends of emotional responses (for example, a tool might assess that a specific movie scene generates a positive audience reaction generally but not that a specific movie-goer is happy while watching it). 87 Other foundational assumptions underlying the claims of emotion recognition companies are that an emotion can be uniformly expressed and perceived from a corresponding set of facial movements and that specific facial movements can be mapped uniquely onto an emotional category. The obvious problem is that people can and do fake emotions, and they can experience feelings without moving their faces. 88 In 2019, social scientist Lisa Feldman Barrett conducted a wide-ranging review of around 1,000 studies inferring emotions from facial expressions. 89 She found the studies that dealt with how people move their faces during emotional events do not strongly support the idea that people reliably and uniquely express certain emotions with spontaneous facial movements. 90 At one extreme, she cited studies finding no clear link between facial movements and emotional states, including the work of psychologist Carlos Crivelli who worked with residents of the remote islands Ekman studied decades earlier and found no evidence for 83 See, e.g., James A. Russell, Emotion, Core Affect, and Psychological Construction, 23 COGNITION & EMOTION 1259-83 (2009) (offering an alternative theory to fixed emotional categories that relies instead on recognizing the combination of simple neurophysiological states with psychological construction processes that produce the experienced components of a particular emotional state, associations among various components, and the categorization of patterns of component-associations as a specific emotion). 84 See ATLAS OF AI, supra note 12, at 175; see also Güneş & Pantic, supra note 20 (finding that categorizing emotional states with a single label is ill-suited when people express themselves through tens or hundreds of subtle and complex facial expressions, bodily gestures or physiological signals). 85
November 21, 2022 14 emotional universality. 91 While Barrett found that people do express instances of anger, disgust, fear, happiness, sadness, and surprise with the presumed facial expressions at above chance levels, the reliability of that finding was weak. 92 She concluded that it is "not possible to confidently infer happiness from a smile, anger from a scowl, or sadness from a frown, as much of current technology tried to do." 93 The facial expressions coded into these tools are at best "Western gestures, symbols or stereotypes that fail to capture the rich variety with which people spontaneously move their faces to express emotions." 94 Even Paul Ekman agrees that companies today are misusing his research to build commercial tools. He "[doesn't] think much of the latest research, [as] it's not been replicated and it seems ideologically driven," referring to the motivation for profit instead of scientific knowledge. 95 He believes companies must invest in more research to prove that facial expressions are linked with emotional behavior: "Simply measuring the face doesn't tell you whether your interpretation of it in that instance is correct or incorrect." 96 Like several other experts, he sees these companies' tools as "pseudoscience-they weren't doing the research to show the interpretation of the [facial] measurements was correct . . . [y]ou need to separate measurements from significance." 97 And what is considered significant is up for debate. If a study has participants choose one of six emotional labels for each facial expression, some researchers might consider an option picked more than 20% of the time as significant consensus while others might set a higher benchmark. 98 Psychologists James Russell and José Miguel Fernández-Dols gather this and other open questions in a survey of facial expression research, showing that the most basic aspects of the science remain unknown. 99 Ekman has tried to challenge companies' claims directly, asking to see evidence that their algorithms work, but he has not heard back. In his opinion, "they're making claims for things there is no evidence for." 100 Emotion recognition tools fail in yet another way by ignoring social and cultural context, a necessary condition for perceiving human emotions accurately. A recent study challenges the widely adopted idea that identifying facial expressions is key to emotion recognition and that ML models are accurate when they focus mainly on facial features in "static, unnatural conditions." 101 The researchers obscured the faces and bodies of people in silent videos to see whether viewers could infer the emotions depicted based solely on visual context. Viewers were able to do so accurately and with high consensus, even when no facial information was present. 102 They found that context is not only a sufficient but necessary condition to accurately perceive emotion over time, revealing that emotion 91 Id.; see also Heaven, supra note 14 ("Trying to assess internal mental state from external markers is like trying to measure mass in metres, Crivelli concludes."). 92 Barrett et al., supra note 19. 93 Id. 94 Id. 95 Murgia, supra note 23 (quoting Ekman). 96 Id. 97 Id. 98 See Heaven, supra note 14. 99 recognition "is, at its heart, an issue of context as much as it is about faces." 103 When an emotion recognition tool purports to identify emotion based on someone's facial movements alone, it is not "interpreting emotions the same way humans do" because it does not account for the necessary context cues humans rely on to gauge emotions; it views a fundamentally human experience through the mathematical gaze of the machine, mechanically labeling a raised eyebrow as "surprise," unable to grasp the bigger picture. 104 Companies that base their emotion recognition tools on Ekman's cross-cultural theory of emotion also ignore the fact that culture is a major influence in emotional expression. 105 Ekman himself later showed that there are significant cultural differences between how individuals of one culture express emotions compared to another. In studying how students in Japan and the U.S. react to violent films, he observed that the Japanese students adopted "a completely different set of expressions" if someone else was in the room, especially an authority figure. 106 Other research suggests that there is an in-group advantage to both the expression and recognition of various emotions. One meta-analysis examining emotion recognition within and across cultures found that, although emotions were universally recognized at better-then-chance levels, accuracy was higher with emotions displayed by members of the same national, ethnic, or regional group. 107 When cultural groups are exposed to other groups, for example by living in the same nation, sharing physical proximity, and communicating verbally, the study found the in-group advantage was smaller. Crucially, majority group members were worse at recognizing minority group members' emotions than the reverse. And importantly for researchers, how a study was designed did not mitigate the size of the in-group advantage. Cross-cultural accuracy was lower in studies that used a mix of posed and spontaneous emotional expressions but higher in studies that used expression imitation instead. 108 The lack of context also complicates the notion that a particular expression maps on to a specific emotional state. For example, of the 19 different types of smiles, only six happen when people are experiencing enjoyment. 109 This is intuitive to anyone who has smiled in pain, embarrassment, or discomfort, but distinguishing one type of smile from another requires social context. 110 And culture influences the way people respond to 103 Id. at 7559. 104 Russell & José Miguel Fernández-Dols eds., 1997), https://www.cambridge.org/core/books/abs/psychologyof-facial-expression/spontaneous-facial-behavior-during-intense-emotional-episodes-artistic-truth-and-opticaltruth/8FD3064C9927B62E40C4D393CC4CB873; Mark Pyrdt, John Zeally, & Omaro Maseli, The Risks of Using AI To Interpret Human Emotions, HARV. BUS. REV. (Nov. 18, 2019), https://hbr.org/2019/11/the-risksof-using-ai-to-interpret-human-emotions. 110 See, e.g., Chen & Whitney, supra note 101, at 7563 (noting a smile "could be faked to hide nervousness in an interview setting," it could "signal friendliness when celebrating other peoples' successes," or it could "show hostility when teasing or mocking others").
November 21, 2022 16 emotional experiences. For example, scowling is a facial movement often associated with anger, but Barrett found that people in urban, large-scale cultures only scowl in anger about 30% of the time. 111 People might be scowling for other reasons, such as deep concentration or confusion-an expression like the "focus frown" that students often experience when paying attention or working through a problem. 112 There are also significant methodological issues that lurk behind the foundational research underlying many emotion recognition tools. As discussed earlier, Ekman's theory of basic emotions and the Facial Action Coding System (FACS) are the most used schemes in training these technologies. 113 They rely on Ekman's categorical approach to emotions, where a small number of emotions are shared by all humans and expressed similarly. Ekman relied on a forced-choice response format that alerted subjects to the connections that designers had already made between certain expressions and emotions. 114 He also relied on posed photographs depicting actors making deliberate expressions with their facial muscles, and several other systems follow this model of using posed images or video. That means that these systems are "trained to recognize faked expressions of feeling." 115 But deliberate movements use separate motor pathways in the brain, and the differences between natural or spontaneous expressions and deliberate ones may significantly impact the conclusions researchers can draw from them. 116 Developers attempting to capture spontaneous data are still limited by their subjects' awareness that they are being filmed (and where those cameras are filming them from). 117 Ultimately, most emotion recognition tools are not trained on genuine emotional reactions, only contrived ones.
Even systems that attempt to target spontaneous emotions and incorporate realworld data confront what researchers call "the Baseline Problem." This problem refers to the challenge of finding a particular frame in the recording of an individual's face where the subject is expressionless-a baseline against which their later emotional expressions can be assessed. 118 This is usually chosen manually by study designers who constrain a recording to an emotional prototype of a "neutral" expression, or by having the first frame of the video contain a baseline or neutral expression. In both cases, the designer's assumptions of what a neutral expression looks like becomes the basis for measuring non-neutral expressions. Limiting recordings to a particular frame also creates an unrealistic constraint, since emotions typically flow from some prior state of activation. 119 Another methodological issue lies in the categorical view of emotions, one that does not account for complex emotional states, blended emotions, or more dynamic emotions. Reliance on Ekman's basic emotions and the FACS demonstrates the difficulty of developing an intuitive and unambiguous data-labeling scheme that can capture dynamic 116 Güneş & Pantic, supra note 20, at 76. 117 Id. at 76-77; see also ATLAS OF AI, supra note 12, at 173 ("Even for images that are captured of people responding to commercial or films, those people are aware that they are being watched, which can change their responses"). 118 Güneş & Pantic, supra note 20, at 81. 119 Id.
November 21, 2022 emotional states. 120 Researchers are still trying to create a coding scheme that can account for dynamic emotions, as well as other communicative cues (i.e., vocal intonation, vocalization, physiological signals, gait, body movements, and context). 121 Despite decades of scientific controversy and a lack of consensus on any method for inferring emotional states from a person's face, companies continue to develop and market deceptive, unsubstantiated one-size-fits-all emotion recognition tools that impact consumers' lives in harmful, discriminatory ways.
2. Emotion recognition tools are unfair as they are likely to cause substantial unavoidable injury to consumers that is not outweighed by any benefits, particularly when deployed in high-stakes contexts and on people of color and people with disabilities (PWD).
In addition to their deceptive marketing, emotion recognition tools are discriminatory on many levels, from the biased data sets that train the algorithms to the racially disparate results they produce. These tools consistently injure consumers in myriad important and sensitive contexts-from job applicants who are judged unfairly because their facial expressions do not match those of other employees to students flagged because their faces appear angry or distracted to customers questioned at the checkout line because their facial cues indicate they might be stealing-all based on technology that lacks scientific support. For PWD, the risks of harm are even greater, such as when a system flags them for having abnormal expressions, or when companies market tools that claim to diagnose mental illness, catch developmental conditions, or improve accessibility for PWD. In multiple ways, these tools discriminate against PWD who may move their faces differently than others or not at all. 122 Section 5 of the FTC Act also prohibits these technologies as unfair because (1) they are likely to cause substantial injury to those falsely classified by these systems, (2) consumers cannot reasonably avoid such injury, especially in high-stakes contexts, and (3) the injury is not outweighed by any benefits to consumers or competition. 123 Until they are regulated, unsuspecting business customers and consumers will bear the costs of these dangerous systems. We agree with Chair Khan and Commissioners Slaughter and Bedoya that the Commission's unfairness authority can reach unfair, discriminatory conduct, even if such conduct may be subject to other legal or regulatory regimes. 124 These practices are "not 120 Id. at 78-79. 121 Id. 122 See, e.g., NOLDUS, Baby FaceReader, https://www.noldus.com/facereader/baby-facereader (last visited Nov. 21, 2022) ("Baby FaceReader has been developed as a state of the art system to automatically detect infant facial expressions in order to help address questions in developmental psychology related to affect and developmental disorders such as Autism Spectrum Disorder (ASD) and other attention-deficit hyperactivity disorder (ADHD)"). Several emotion recognition companies have attempted to claim a countervailing benefit in that their tools are free of the human biases that affect decisionmakers. Fundamentally, however, emotion recognition tools are attempting to assess people's inner states-and even their character-based on their faces. This has been described as a "phrenological impulse: drawing faulty assumptions about internal states and capabilities from external appearances, with the aim of extracting more about a person than they choose to reveal." 126 These tools "take us back to the phrenological past, where spurious claims were made, allowed to stand, and deployed to support existing systems of power." 127 Others agree that these tools are a form of "modern phrenology," referring to the false theory that a person's skull shape could reveal their character and mental abilities. 128 Instead of directly measuring people's interior mental states, these tools optimize correlations of certain physical features from images of faces with a predefined set of emotions. They are based on a dangerous method of divining and categorizing people based on a statistical analysis of physical markers, interpreting minute muscular movements as significant indicia of emotion. The judgments based on these tools have the capacity to harm any and all consumers who happen to move their faces in specific ways.
While emotion recognition has the capacity to harm a diverse set of consumers in various high-stakes contexts, research suggests these tools will disproportionately harm people of color and PWD, as well as older individuals and women and nonbinary people. One source of discrimination is in the training data sets that teach algorithms how to identify certain emotions. One study found that at least one of the public data sets often used to train emotion recognition systems contains far more faces of white people between ages 20 and 39 than faces of Asian or Black people of any age. 129 While augmenting the data marginally improved the accuracy of the baseline model in this study, adding more diverse faces could not mitigate the strong bias of recognizing white facial expressions more accurately than those of Asian or Black people. 130 Unsurprisingly, tools based on these data sets produce biased results. One study ran a set of male NBA players' photos through two popular emotion recognition tools. The researcher found both tools deemed Black players to have more negative emotional scores than all other players, with one labeling Black players as more "aggressive" and the other classifying them as having more "contempt"-even when they were smiling. 131 She notes the severe consequences of her findings: "If an AI system mistakenly views a candidate as statements/joint-statement-chair-lina-m-khan-commissioner-rebecca-kelly-slaughter-commissioner-alvaro-mbedoya. 125 Joint Statement, id., at 2. 126 Crawford, supra note 40. 127 ATLAS OF AI, supra note 12, at 177. 128 Vallance, supra note 46 (quoting Dr. Eleanor Drage, author of a study assessing emotion recognition hiring tech); see also ATLAS OF AI, supra note 12, at 222 ("These approaches resemble phrenology and physiognomy in their desire to essentialize and impose identities based on external appearances"). 129 angry, then the person may never receive a call-back interview or find a position in their field. If an AI system identifies an individual as a threat, then that person could be detained, followed, placed on a no-fly list, or some other significant consequence." 132 She suggests that "[p]rofessionals of color should exaggerate their facial expressions-smile more-to reduce the potential negative interpretations [of these tools]." 133 While some companies have attempted to build vast databases full of more diverse faces, this still neglects the fundamental scientific uncertainty about whether facial imagery out of context offers any consistent insight into people's inner states. Further, it does nothing to address the impact these tools can have on PWD whose facial movements may not fit the categorical norms embedded within the ML systems. For example, many emotion recognition tools today are built on ableist views of emotion that ignore how many PWD communicate emotions beyond facial expressions alone. 134 Several common and rare conditions affect facial movements and control. 135 A person with Moebius syndrome may not be able to smile, frown, or raise their eyebrows in a way that ML algorithms have been trained to understand, 136 and a person with Autism Spectrum Disorder (ASD) may be judged as having less intense and clearly distinct emotional states by tools trained only on facial expressions made by people without ASD. 137 When an emotion recognition tool is used to assess job candidates, for example, it might systematically disadvantage candidates with disabilities because they may present differently than most of the company's other applicants or employees. 138 To these tools, these individuals may never present as "happy," 132 Id. at 1. 133 Id. at 6. 134
n part because algorithmic technologies that are trained on data that already embeds ableist (or relatedly racist or sexist) outcomes will entrench and replicate the same ableist (and racial or gendered) bias in the computer system"). 135 Many common facial nerve disorders are typified by unusual facial movement, weakness of facial muscles, or paralysis of all or part of the face. For example, Bell's palsy, Lyme disease, strokes, skull base tumors, viral infections, and certain congenital anomalies may impact a person's facial movements and control. See Disorders of the Facial Nerve and Skull Base, NIKLAUS CHILDREN'S HOSP. (last visited Nov. 21, 2022), https://www.nicklauschildrens.org/conditions/disorders-of-the-facial-nerve-and-skull-base. 136 Rarer conditions like Moebius syndrome also primarily affect the muscles that control facial expression and eye movement, meaning people with this syndrome may not be able to form facial expressions. See Moebius Syndrome, MEDLINEPLUS.GOV (last visited Nov. 21, 2022), https://medlineplus.gov/genetics/condition/moebius-syndrome/. 137 One meta-analysis of studies of group differences on the production of facial expressions in participants with ASD revealed that those participants display facial expressions less frequently and for a shorter amount of time, are less likely to share facial expressions with others or mimic the expressions of other faces or face stimuli, and their facial expressions are judged by others to be more ambiguous and assessed less accurately than expressions of people without ASD. Dominic A. "sad," or "surprised," and their use on many PWD is likely to cause them significant, unavoidable harm. As emotion recognition tools are integrated across society, such as into cars, airports, and predictive policing programs, the dangers of being "misread" are serious, especially for numerous groups including people of color, PWD, and the elderly. For example, a Black candidate may feel pressure to smile wider and longer to avoid being categorized with a negative emotion and passed over for the job. An older driver may feel forced to move her face in particular ways to avoid her car incorrectly identifying her as having driver fatigue, which could lead her insurance provider to raise her premiums as a dangerous driver. And a person with a disability affecting their facial movements may be forced to disclose their disability to a prospective employer, at the checkout line, or in the airport, to avoid the danger of being misread when compared to faces trained with discriminatory norms. Each of these consumers and countless others are likely to be injured by the deployment of emotion recognition in their everyday life, in contexts that they cannot reasonably avoid, and with no other benefit served through these technologies. Without regulatory intervention, these tools will place an unfair, additional burden on consumers, especially from marginalized communities, for the mere convenience of profiling people to fit commercial ends. 139 3. Emotion recognition poses a serious threat to consumer privacy because surveilling facial expressions is an invasion of privacy that is essential to the development and use of these tools. 140 Emotion recognition targets our facial expressions, a biological feature necessary for social connection. These tools attempt to classify our innermost states and extract our emotions, quantifying the most minute facial gestures to categorize and judge whether we are feeling what we should be feeling. In order to train these systems and conduct these assessments, immense amounts of private consumer data is required. These tools surveil and measure individuals' natural emotional responses, especially in high-stakes contexts like seeking employment, crossing a border, or performing in the classroom. And as emotion recognition becomes more prevalent in public spaces-for example, in smart advertising in city centers or in policing large crowds-consumers may struggle to protect their privacy by https://arxiv.org/abs/1907.02227; Alex Engler, For Some Employment Algorithms, Disability Discrimination by Default, BROOKINGS INST. (Oct. 31, 2019), https://www.brookings.edu/blog/techtank/2019/10/31/for-someemployment-algorithms-disability-discrimination-by-default/. 139 See Rhue, supra note 131, at 6 ("Is this additional burden fair? No.").
November 21, 2022 21 obscuring their faces routinely from pervasive emotion recognition tools. In this way, they may face the same privacy harms that inhere in other forms of biometric surveillance. Historically, most emotion recognition tools were built from data of actors who were filmed or photographed performing dramatic versions of certain emotional states. 141 As discussed above, the constructed nature of these training sets limits ML models' reliability to recognize natural emotional states in the wild. To increase their reliability, developers have begun to fill training data sets with photos and footage of people experiencing genuine emotional reactions to stimuli. At least two companies have filmed consumers directly to train their emotion recognition tools. Disney built its data set by filming audiences during 150 showings of nine family movies, and Intel developed its classroom tool by filming students in real-life classroom settings with 3D cameras. This increases the intrusiveness of emotion recognition tools and increases surveillance in varied spaces to ensure the most "accurate" results. Other tools are built off less obvious means of surveillance. SmileML offers a sales analytics tool, SaleSpot, that uses an emotion recognition model trained from a data set collected through a mobile game application, MatchMoji, where players match emojis to the expressions of a person's facial image; players can play the game in "Train" mode, sharing data to improve the model. 142 Companies are faced with a challenging tradeoff: continue using algorithms trained on posed data with low fidelity to real-life emotional states, or train algorithms on data sets of genuine emotional reactions through surveillance.
But even emotion recognition trained on posed data presents a serious threat to consumer privacy when it is deployed to watch and measure peoples' faces. Even if emotion recognition could be made more reliable, our thoughts and emotions are some of the most intimate parts of our personality, shielded from extraction by the right to privacy. The European Union's proposed AI regulations define emotion recognition technologies as "high-risk" and call for explicit consent from individuals. 143 The proposal acknowledges a person's right to privacy over their emotions. A recent, comprehensive report on the thriving emotion recognition market in China demonstrates the risks this technology poses under international human rights standards, including the right to privacy. 144 The authors consider use cases today as early warnings of eventual mass surveillance; for example, requiring emotion recognition tools in vehicles to assess driver distraction "includes systemic invasions of privacy and significant mission creep, in the case of biometric information potentially being used for insurance purposes." 145 And Paul Ekman, the founding father of this field, considers "[w]atching someone's facial expressions [as] an invasion of privacy, especially if it's done without their knowledge." 146 According to Ekman, "there should be FTC ANPR, R111004 Emotion Recognition November 21, 2022 22 laws passed that prohibit the recording of facial expression, let alone its interpretation or measurement, without informed consent." 147 Some developers try to side-step legitimate privacy concerns by claiming their tools do not identify specific people. 148 Taking this claim at face value, one researcher explored the privacy implications of presumably non-identifying emotion recognition tools. 149 The study adopted a dignity-based view of privacy that "diagnoses the problems with [emotionbased] passive profiling" as "recognizing that phenomenological experience is important, innately worthy, and should not be appropriated." 150 Based on this view, the practice of inferring a person's emotional state based on aggregate data about what a particular emotion might look like compromises the innate value of their interior experience. This on its own is a threat to consumer privacy, without needing to go further to specifically identify a consumer with personal information.
Our emotions are deeply personal and private. It should be up to individuals to share what they are experiencing internally, not the purview of companies profiting from profiling our innermost states to decide who to hire, who to promote, who to charge a higher insurance rate to, and who deserves a higher grade. Even if the technology worked as companies claim it does, it poses a significant enough threat to human dignity to deserve the most aggressive regulatory treatment. Emotion recognition tools should be prohibited before they become commonplace, permanently compromising our emotional privacy. D. The Commission should consider rules under its Section 5 authority to prohibit the development and use of commercial emotion recognition tools, as procedural protections sidestep the issue of whether this type of sensitive, biometric information should be collected and processed in the first place.
In her Statement on the ANPR, Chair Kahn asks whether procedural protections like requiring consumer consent merely sidestep more fundamental questions about whether certain data practices should be permitted in the first place. 151 She asks when bans and prohibitions may be most appropriate. 152 Emotion recognition tools are an example of when prohibition is appropriate. The Knowing Machines Project supports new rules forbidding emotion recognition tools that claim to know our innermost states in deceptive, unfair, and invasive ways. These tools are often offered on top of facial recognition systems, a type of technology that has received significant attention and scrutiny in recent years. Commercial emotion recognition practices remain largely unquestioned, however, despite a lack of scientific support for their most basic underlying assumptions and their capacity to alter the lives of consumers in harmful ways. By prohibiting emotion recognition tools today, the Commission can prevent the next 147 Id. 148 See, e.g., Nash, supra note 61 (noting EmotionTrac CEO's claim to this effect). 149 In 2019, several of us recommended that regulators ban the use of emotion recognition in important decisions that impact peoples' lives and access to opportunities. 153 Since then, the FTC has been asked to investigate HireVue's potential deceptive and unfair practices, 154 Illinois has enacted a law requiring disclosure and consent for the use of AI analysis of job interview videos, 155 and New York City passed the first law in the country prohibiting employers' use of similar tools without first subjecting them to auditing. 156 Just last month, the United Kingdom's Information Commissioner's Office (ICO) warned companies implementing emotion recognition tools of the "risk of systemic bias, inaccuracy and even discrimination" they pose; the ICO will be publishing guidance in Spring 2023 to further aid businesses in considering the risks of biometric technologies, including emotion recognition. 157 These interventions are a good start to holding companies accountable and ensuring that until scientific evidence exists to substantiate their tools' claimed accuracy and reliability, they are not allowed to harm consumers.
There is much more that lawmakers and regulators can do in this space, beginning with the Commission's authority under Section 5 to clamp down on deceptive and unfair practices. There is a need for "strategic and well-informed advocacy against the design, development, sale, and use" of emotion recognition tools. 158 Such advocacy must come "before these technologies become widespread" to ensure people's rights are protected. 159 The Commission should consider limiting or prohibiting the development and use of emotion recognition tools that lack scientific foundation in all domains that impact consumers-especially in education, employment, and health care. As the Commission pays careful attention to the effects of surveillance on children and other vulnerable populations, we encourage it to protect these communities from bearing the burdens of these technologies. A potential limitation may protect business customers and individual consumers differently by, for example, requiring emotion recognition vendors to provide scientific substantiation for their claims in sale and license agreements to business customers while requiring business customers to have individual consumers opt-in to emotion recognition analyses.
A prohibition would also save the Commission from the challenges of enforcing a consent-based regulation instead. Informed consumer consent is infeasible when the practices for which consent are sought are deceptive and unsupported by scientific evidence. The companies developing these tools cannot scientifically vouch for their validity or FTC ANPR, R111004 Emotion Recognition November 21, 2022 24 reliability, let alone their utility, in a way that would fully inform consumers of the types of surveillance and consequences they might experience. And consent implies a choice that is simply not present in many of the situations where emotion recognition tools are being used. A job candidate may feel pressured to consent to a hiring AI system or risk losing the opportunity to interview; a student may not be able to request a different class, school, or school system to avoid having her emotions monitored throughout her day; a consumer may not be able to afford a car that does not subject him to "distraction detection" systems purporting to measure his anger and stress; a traveler may not be able to reroute her journey to avoid an airport equipped with emotion-detecting border kiosks. Scientists cannot agree on what an emotion is, how it should be measured, how to account for context, and how to develop detection systems that are racially and culturally sensitive. Therefore, companies should not be allowed to profit off these faulty systems. In protecting all consumers, the Commission should pay particular attention to the ableist assumptions underlying emotion recognition tools and consider whether answering these questions is possible in ways that address discrimination against PWD. Finally, the Commission should consider the inherent privacy harm that stems from surveilling consumers' faces in the attempt to access their intimate, personal emotions without consent.
Additionally, we encourage the Commission to develop its expertise by studying the use and impact of emotion recognition tools further. 160 Under its Section 6(b) authority, the Commission may also order companies to share information on their data sets, models, and other emotion recognition practices, especially those that may impact people of color, PWD, and other marginalized populations. We are grateful for the Commission's attention to this complex, timely topic and look forward to any opportunities to aid the Commission's regulatory processes.