"What Can I Say?: Evaluating a Spoken Language Interface to Email," in Proceedings of CHI '98 582 (1998) (with Marilyn A. Walker, Giuseppe Di Fabbrizio, Craig Mestel & Don Hindle)

"What Can I Say?: Evaluating a Spoken Language Interface to Email," in *Proceedings of CHI '98* 582 (1998) (with Marilyn A. Walker, Giuseppe Di Fabbrizio, Craig Mestel & Don Hindle)

Marilyn A. Walker et al., What Can I Say?: Evaluating a Spoken Language Interface to Email, in Proceedings of CHI '98 582 (1998)

Marilyn A. Walker
Jeanne C. Fromer
Giuseppe Di Fabbrizio
Craig Mestel
Don Hindle

Debate about the utility of spoken language interfaces (SLIs) vs. graphical user interfaces (GUIs) has been ongoing for many years. It is widely agreed that GUIs provide immediate feedback, reversible operations, and incrementality, that they support rapid scanning and browsing of information, and that they are easy for novices to use because visual information indicates the currently available options. In contrast, SLIs have been criticized for the inherent limitations of a sequential channel (speech) for delivering information, for requiring the user to learn the language the system can understand, for hiding available command options, and for leading to unrealistic expectations as to their capabilities. The SI strategy prompts users what to say at each stage of the dialogue with directive prompts [2]. Since prompts must be short, the SI strategy typically executes actions by small increments at each turn. The MI strategy assumes that users know what to say, and never volunteers information. The MI SLI is designed so that users can specify a system function and all of its arguments in a single utterance if they desire. While it might appear obvious that users would prefer the more flexible interaction style of the MI interface, there are several complicating factors. First, MI requires more knowledge on the part of the users about what they can say, while the SI strategy directs the user. Second, because SI users are directed to produce very short utterances, the ASR performance of SI may be much better. If more error recovery and correction dialogues are necessary for MI, users may find this onerous. Too many spoken language understanding errors may also make it difficult for users to acquire a model of the system's behavior. Without knowing the effect of these factors, it is impossible to predict whether the SI strategy or the MI strategy is better. This paper discusses the design and evaluation of ELVIS, a research prototype SLI that supports voice access to email by phone. We report experimental results from testing users with both an SI and an MI version of ELVIS. Our experimental data consists of 144 dialogues with 48 users, consisting of a total of 6481 turns. Our results show that even though the MI system is more efficient, as measured by either number of turns, or elapsed time to complete a set of email tasks, users prefer the SI interface. In addition to the dialogue strategy design, a second key aspect of SLI design is deciding what options should be available to the user at each point in the dialogue. Previous work has demonstrated the utility of Wizard of Oz studies [1,4,8,9], so we began our design process with a Wizard of Oz (WOZ) study where a person played the part of an SLI for accessing email. We collected, recorded and transcribed 15 extended conversations (1200 utterances) with 6 different prototypical users, mobile professionals accessing their email while away from their office. We then categorized each utterance in these dialogues in terms of its use of key email access functions. Categories were based on the underlying application, as well as on language-based functionality, such as reference to messages by their properties, such as the sender or the subject of the message (e.g. the message from Kim), or in context (e.g. as them, it, that), Table 1 summarizes the functions used most frequently in the WOZ study. This study suggests that the SLI should minimally support: (1) reading the body of a message and the header information; (2) summarization of the contents of an email folder by content-related attributes such as the sender or subject; (3) selection of individual messages by content fields such as the sender or subject; and (4) request for clarifying help, repetition of something that was said, and undoing of previous actions. Reading the message and header information requires the use of text-to-speech (TTS) since it is impossible to prerecord messages with a human voice. Reading the body of the message also requires filtering the message body for things that are unpronounceable by TTS, and recognizing attachments in the message body. In the WOZ study, users typically requested summaries on entering a folder and referred to messages by attributes such as sender or subject in order to randomly select messages of interest in that folder. We hypothesized that summarization and selection capabilities could provide a way to scan and browse information in SLIs. In other words, one way to obviate the limitations of a sequential speech channel is to give users the ability to overview the data (with summarization) and then select the subset of items that they are interested in (with reference). Summarization and selection of messages by content attributes required reproducing searching and sorting functionality available in many email GUI interfaces. For folder summaries, the list of messages had to be converted into a coherent summary that was appropriate to the context, i.e. whether the folder had been created by selecting by sender or by subject. Even though our WOZ subjects did not typically attempt to access to messages in the sequence in which they were received (or reverse chronological order), we felt that it was necessary to provide this as an additional option because other voice and touch-tone interfaces to voice and email messages provide this option [2,3,9]. Thus both the SI and the MI system support access to messages by relative position within a folder: users can select messages by saying First, Next, Previous and Last. The WOZ study also confirms that the system should provide help so that users can learn its capabilities. We were particularly concerned about the design of help messages for the MI system. The SLI platform that we used to build both versions of ELVIS included a facility for specifying help messages associated with each state of the dialogue. These context-sensitive help messages indicate to the user what command options are available at each point in the dialogue, and provide a verbal analog to the visible icons in GUIs that indicate the available command options. Context-sensitive help was available to the user in two ways: at the user's initiative if the user says Help, and at the system's initiative with timeout messages. The system plays timeout messages when the user doesn't say anything, i.e. after some expected response delay has timed out. The system keeps track of how many times a timeout occurs in each state, so that timeout messages can be modified to be more informative after each timeout. Because help messages (and email messages) can be long, users must be able to interrupt the system to take control of the interaction at any point, while the system is talking or carrying out a command. This is called Barge-In. Supporting barge-in requires that a speech recognizer is always listening, even when the system is currently busy recognizing something the user said previously. Barge-In also involves the ability to abort common procedures in midstream, e.g the system needs to be able to send TTS instructions to stop talking in midstream. Finally, the SLI must provide some way of undoing a previous command. This is useful in two cases: (1) if the user simply decides they would rather do something different; and (2) if the SLI misunderstand the user. ELVIS supports reversibility by providing an always available cancel command that returns the user to the dialogue state before the previous interaction. The experiment required users, randomly assigned to either the MI or the SI version of ELVIS, to complete three tasks involving telephone access to email. All of the users regularly used computers in the course of their everyday work and were familiar with email. In one study, the 12 users were administrative assistants or researchers whose area of research was not related to SLIs. We reported results from this study elsewhere [5]. Subsequently, we noticed that response delay was longer than we wanted and that there was a different way of communicating with the email application layer that would significantly reduce it. After implementing the improved version, we then tested another 36 users in both versions of ELVIS. These subjects were summer interns, with little exposure to SLIs, many of whom were not native speakers of English. Below we discuss results from these 36 users. Experimental instructions were given on three web pages, one for each experimental task. Each web page consisted of a brief general description of Elvis, a list of hints for using Elvis, a task description, and information on calling ELVIS. Subjects read the instructions in their offices before calling ELVIS from their office phone. Each user performed three tasks in sequence, and each task consisted of two subtasks. Thus the results consisted of 108 dialogues representing 216 attempted subtasks. The task scenarios that the subjects were given were as follows, where subtasks 1.1 and 1.2 were done in the same conversation, similarly for 2.1 and 2.2, and 3.1 and 3.2. • These tasks were based on representative tasks from the WOZ study, involving the use of summarization and reference as in Table 1. Each subtask specified the information about criteria for selecting messages, and information within the message body, that the user and the system had to exchange. For example, in scenario 1.1, the user is expecting email from Kim about a meeting and needs to find out the time and place of that meeting (as in Dialogue D1 and D2). Following [6], this scenario is represented in terms of the attribute value matrix (AVM) in Table 2. The AVM representation for all six subtasks is similar to Table 2. Note that the task's information exchange requirement represented in the AVM is independent of the dialogue strategy used to accomplish the task. The use of the AVM to calculate task success is discussed below. We designed the experimental email folders so that for each task, the desired messages were not among the first two messages (as ordered chronologically). Thus users who accessed messages by chronological order would often have to listen to all five messages in order to complete the task, while users who accessed messages using selection by content could complete the task by listening to two messages. Thus accessing messages by relative position should have led to inefficient dialogues, while the instructions specified that users should be as efficient as possible and avoid listening to messages unnecessarily. 2D 516 The general description and the hints on the web page for each task were identical. The subjects were asked to impersonate a different user for each task and were told that they needed to talk to ELVIS to find out some information that had been sent to them in an email message. We decided not to include any specific examples of what users could say in the hints for using ELVIS for three reasons: (1) we wanted the instructions to be identical for both SI and MI; (2) users could get information as to what they could say from the context-sensitive help messages; (3) we wanted to be able to quantify the frequency with which users accessed information on what they could say, and would not have been able to do so if this information had been presented visually. The hints were: We collected four types of data and extracted a number of variables. First, all dialogues were recorded. The recording supports utterance transcription and measuring aspects of the timing of the interaction, such as whether there were long system response delays, and whether users barged-in on system utterances (the variable named BargeIn). BargeIn may reflect learning; as users learn what they can say, they can barge in over the system's utterances. In addition, the recording was used to calculate the total time of the interaction (the variable named Elapsed Time). Second, the system logged its dialogue behavior on the basis of entering and exiting each state in the state transition table for the dialogue. For each state, the system logged the number of timeout prompts (Timeout Prompts), the number of times the confidence level for ASR was too low and the system played a special rejection messages, e.g. Sorry, I didn't understand you (ASR Rejections), and the times the user said Help (Help Requests). The number of System Turns and the number of User Turns were calculated on the basis of this data. In addition, the results of ASR for the user's utterance was logged. A measure of the system's understanding (concept accuracy) was calculated from the recordings in combination with the logged ASR result for each utterance. Mean concept accuracy was then calculated over the whole dialogue to provide a Mean Recognition Score (MRS) for the dialogue. Third, users were required to fill out the web page forms after each task specifying whether they had completed the task and the information they had acquired from the agent (Task Success), e.g. the values for Email.att1 and Email.att2 in Table 2. This supported the use of the Kappa statistic to measure Task Success [6], where Kappa is defined as: K = P(A) -P(E) / 1-P(E) P(A) is the proportion of times that the AVM for the dialogue agrees with the AVM for the scenario key, and P(E) is the proportion of times we would expect the AVMs for the dialogues and keys to agree by chance. When agreement is perfect (all task information items are successfully exchanged), then Kappa=1. When agreement is only at chance, then Kappa=0. Finally, users responded to a survey on their subjective evaluation of their performance and their satisfaction with the system's performance with the following questions: • Did you complete the task? (Comp) • Was Elvis easy to understand in this conversation? The user satisfaction survey was multiple choice, and the possible responses to most questions ranged over values such as (almost never, rarely, sometimes, often, almost always), or an equivalent range. Each of these responses was mapped to an integer between 1 and 5. Some questions had (yes, no, maybe) responses. Each question emphasized the user's experience with the system in the current conversation, with the hope that satisfaction measures would indicate perceptions specific to each conversation, rather than reflecting an overall evaluation of the system over the three tasks. A Cumulative Satisfaction (CSAT) score for each dialogue was calculated by summing the scores for each question. The survey also included a free text field where users were encouraged to enter any comments they might have. The goal of the experiment was to evaluate the usability of an SLI for accessing email by phone and to compare the MI dialogue design to the SI dialogue design when the task is held constant. We wished to investigate how users would adapt to the version of the system they were using as they performed a sequence of three similar tasks. Our primary experimental variable was dialogue strategy: whether the user interacted with the SI or the MI version of ELVIS. However, we were also interested in whether the availability of summarization and selection by content increased the functionality of the system. Our hypotheses were: • H1: The MI strategy is potentially much more efficient than the SI strategy, but its efficiency depends on ASR performance, and the lower the ASR performance the less efficient it will be. • H2: Users will have trouble knowing what they can say to the MI SLI and this will reduce ASR performance. • H3: Users' knowledge of what they can say to the MI SLI will improve over the three tasks. • H4: Because of H1, H2, and H3, Cumulative Satisfaction for the system initiative SLI will be greater for the first task, but Cumulative Satisfaction for the MI SLI will be greater by the third task. • H5: Use of summarization will increase Cumulative Satisfaction and improve efficiency. • H6: Use of selection by content will increase Cumulative Satisfaction and improve efficiency. These hypotheses concern the relation between dialogue strategy, Mean Recognition Score, the utilization of the summarization and selection by content options, and the users' ability to learn what options are available at each point of the dialogue and to acquire a model of the system. Our experimental design consisted of two factors; strategy and task. Each of our result measures were analyzed using a two-way ANOVA for these factors. For each result, we report F and p values indicating its statistical significance. Effects that are significant as a function of strategy (SI vs. MI) indicate differences between the two strategies. Effects that are significant as a function of task are potential indicators of learning. We discuss results for each of these factors as they relate to our hypotheses. We first calculated Task Success in terms of Kappa to see whether task completion rates and scores were affected by dialogue strategy [7]. The average Kappa value over all subjects and tasks was .82, indicating that the task was almost always completed successfully. An ANOVA with Kappa as the dependent variable revealed no significant differences for Kappa as a function of task or strategy. Hypothesis H1 focuses on the relation between Mean Recognition Score (MRS) and efficiency. We examined efficiency with three efficiency measures: User Turns, System Turns and Elapsed Time. An ANOVA for each of the measures as a function of strategy and task showed that strategy was a significant predictor of efficiency in each case, and that MI was more efficient than SI: User Turns (F(1,34)=31.9, p<.0001), System Turns (F(1,34) =14.3, p=.0006) and Elapsed Time (F(1,34)=3.92, p=.05). Means for these measures are given in Table 3. Hypothesis H3 concerned the effect of learning on MRS for the MI interface, and on efficiency as a result. As we hypothesized, MRS did improve as users learned the system (F (1,70)=6.37, p<.01). The MRS of the MI strategy was .68 for task 1, .74 for task 2 and .76 for task 3, while MRS for the SI strategy was .88 for task 1, .87 for task 2 and .92 for task 3. Furthermore, efficiency was also directly affected by users' learning of the system. There was an interaction between strategy and task for both Elapsed Time (F(1,70)=4.85, p=.03) and System Turns (F(1,70)=5.23, p=.03). For the SI system, the mean Elapsed time for task 1 was 321.7s, for task 2 was 345.9s, and for task 3 was 318.2s. In contrast, the MI system started out taking more time on average (task 1=332.6s), but Elapsed Time was reduced significantly over the subsequent tasks, with task 2 taking 302.8s on average and task 3 taking 232.8s. System Turns showed a similar pattern: for the SI system, System Turns for task 1 averaged 26.7, 29.7 for task 2 and 28.1 for task 3. System Turns were reduced for each task for the MI system: task 1 took an average of 25.4 turns, task 2 averaged 22.3 turns and task 3 averaged 17.5 turns. Hypothesis H4 posited that users' Cumulative Satisfaction (CSAT) for the SI system would be greater for task 1, but that as users learned the MI system over the three tasks, that the flexibility of the interface and the gains in efficiency would cause MI to be preferred. CSAT was greater for SI for task 1: mean CSAT for task 1 for SI was 27.2 while mean CSAT for MI for task 1 was 23.8. However, despite the fact that the MI strategy was clearly more efficient than the SI strategy by the third task, there was no interaction between strategy and task. There was a significant difference in CSAT as a function of strategy (F(1,34)= 23.59, p=.02), with mean CSAT being higher for SI (26.6) as compared with MI (23.7). The increases in CSAT for the MI strategy were not significant: mean CSAT for task 1 was 23.7, for task 2 was 22.7 and for task 3 as 24.4. Thus even with the effects of learning, CSAT for MI by the third task was still lower than CSAT for SI on the first task. (See Table 4). Thus Hypothesis H4 is disconfirmed. It appears that, contrary to H4, users' preferences are not determined by efficiency per se, as has been commonly assumed. One interpretation of our results is that users are more attuned to qualitative aspects of the interaction. To explore this idea further, we first analyzed the relationship between CSAT and our other measures, drawing on the PARADISE framework [6], and its use of multivariate linear regression. We first normalized all measures to their Z scores to ensure that the magnitude of the coefficients in the regression equation would reflect the magnitude of the contribution of that factor to CSAT. An initial regression over a range of measures suggested that Users' perception of task completion (Comp), Mean Recognition Score (MRS) and Elapsed Time (ET) were the only significant contributors to CSAT. A second regression including only these factors resulted in the following equation: CSAT = .21*Comp + .47 * MRS -.15 * ET with Comp (t=2.58, p =.01), MRS (t =5.75, p =.0001) and ET(t=-1.8, p=.07) significant predictors, accounting for 38% of the variance in R-Squared (F (3,104)=21.2, p<.0001). This equation demonstrates that while efficiency and task completion are both factors in predicting CSAT, that they are not as significant as MRS. It is plausible that the qualitative behaviors that are correlated with poor MRS have a greater effect on CSAT. This interpretation is supported by measures that more directly reflect the quality of the interaction. See Table 4. First, as discussed above, there were significant differences in Mean Recognition Score (MRS) as a function of strategy. Furthermore, even though users of the MI system were not more likely to ask for help using the always available Help command (F(1,34)=1.47, NS), they were much more likely to trigger Timeout Prompts (F(1,34)=10.87, p=.002). Remember that Timeout Prompts are system turns that suggest to the user what they can say, which are triggered by occasions in which the user says nothing after a system utterance. This may happen because the user does not know what they can say, or because the user is confused by what the system just did. The mean number of timeouts was 4.15 per dialogue for users of the MI system as opposed to 2.24 for the SI users. Another qualitative aspect of the interaction is the system's production of diagnostic error messages. In our study, it was much more common for the system to reject the utterances of users of the MI system (ASR Rejects), because of low ASR confidence scores (F(1,34)=4.38, p= .04), leading the system to produce a diagnostic error message asking the user to repeat himself or telling him what he could say. Finally, there was an interaction in the use of BargeIn between strategy and task (F (1,70)=14.18, p=.0003). Remember that BargeIn may reflect learning; as users learn what they can say, they can barge in over the system's utterances. SI users increased their use of BargeIn over the three tasks, with the number of BargeIns for task 1 at 3.55, task 2 at 4.61 and task3 at 7.33, suggesting that they were learning the interface and becoming more confident. In contrast, users of the MI system started out using BargeIn more (task 1=4.17) but the use of BargeIn decreases with task 2 at 3.89 and task 3 at 2.6. One explanation for the decrease in BargeIns is that users lost confidence in knowing what to say to the MI system. Other evidence suggests that it was difficult for users to acquire a model of how the MI system worked. Even though the MI system made more errors, users of the SI system were much more likely to use the Cancel command (F(1,70)=18.41, p=.0001), which undoes the effects of the previous command. One plausible explanation of this difference is that SI users acquired a model of the dialogue flow, making it possible for them to use the cancel command effectively, while MI users did not. Further insight into the factors that affect CSAT can be found by examining the individual satisfaction measures that CSAT is composed of. Users perceive it to be easier to find a message (Task Ease) in the SI condition (F( Hypotheses H5 and H6 were that users who made use of the summarization and selection by content options provided in both the SI and the MI interfaces would have greater Cumulative Satisfaction (CSAT) and be more efficient than those who chose to listen to their messages in chronological order. In order to test hypotheses H5 and H6, we analyzed the experimental logs and transcriptions for summary use (Suse), next use (Nuse), and content-selection use (Csuse). Both SI and MI users utilized one of the summarization options that were provided in the system, averaging 1.4 summaries per dialogue. While some users did not use summarization at all, other users summarized up to 5 times in a single dialogue. However, contrary to H5, Suse did not lead to higher CSAT, nor did it lead to greater dialogue efficiency. In fact, our results demonstrate an opposite pattern. An ANOVA with CSAT as the independent variable and Suse as a dependent variable showed Suse a significant predictor of CSAT (F(5,67)=3.45, p=.008). However CSAT goes down as Suse goes up as shown in Table 5. Furthermore, Suse is highly correlated with Elapsed Time: the correlation coefficient is .49. Thus the more subjects summarized, the less efficient their dialogues were. Analysis of the dialogue transcripts for users who requested the most summaries shows that summaries were used as an error recovery strategy. Users would summarize when the system misunderstood one of the sender or subject values that they had specified when attempting to select by content. Since the sender and subject values were provided in the summary, users would listen to the summary again to make sure that they had specified the values correctly. Thus an increase in the use of summarization indicates a user who was having recognition problems. To test Hypothesis H6, we examined the relationship between Nuse (use of the Next command), Cuse (use of the content selection options), and both Cumulative Satisfaction (CSAT) and Elapsed Time (ET). Nuse is not a significant predictor of CSAT (F(1,106)=3.45,NS). There was also no effect for Cuse, the use of selection by content options. Further investigation reveals that the main reason for this is likely to arise from the poor performance of ASR for selection by content in the MI condition. In the MI condition, the probability of being correctly understood when using the Read option was only .63. Half of the time if the user specified a selection criteria for reading, e.g. Read my messages from Owen, the system misunderstood: sender values were misunderstood 43% of the time. In contrast, MI users who selected messages in chronological or reverse chronological order were correctly understood 81% of the time. Thus there was a great incentive for MI users to not use the selection by content options. On the other hand, if a user chose to access messages by content rather than by order in the SI condition, it took at least three interchanges to say so (see D1). However the overall probability of correct understanding when specifying selection by sender was .76, and for selection by subject was .78. The probability of success for selection by position was .82. Thus there was little difference in the SI condition between system performance for selection by content versus by order. In order to see whether selection by content is useful when ASR performs appropriately, we analyzed Cuse for the SI strategy alone. An ANOVA of CSAT as a function of Cuse shows Cuse to be highly predictive of CSAT (F(6,47)=3.89, p=.003). Table 6 shows that CSAT is greatest when Cuse matches optimal performance on the task, i.e. since each task required access to only 2 messages, when Cuse is 2. Thus H6 is disconfirmed for the MI condition, but confirmed for the SI condition. This paper evaluates a mixed-initiative (MI) dialogue design in comparison with a system-initiative (SI) dialogue design in ELVIS, a spoken language interface for accessing email by phone. It has been commonly assumed that spoken language interfaces that constrain the user will be less preferred than unconstrained interfaces. Our hypotheses were that users would initially prefer the SI system, which controls the interaction so that the options available to the user are obvious at each point of the dialogue, and that ASR (automatic speech recognition) would perform better with the SI grammars. However, we hypothesized that as users performed successive tasks, they would learn how to use the MI system, which does not constrain the user, and which is more efficient. We hypothesized that as users learned how to use the MI system, their confidence with the system would increase, and that ASR performance would also increase. Thus, by the end of three tasks, we hypothesized that the satisfaction of MI users would be greater than that of the SI users. Our results show that the additional flexibility of the MI interface leads to user confusion about their available options and poor performance by ASR. While user expertise and ASR performance did increase for MI over three tasks, these increases did not result in a preference for the MI interface. Despite the fact that the MI interface is more efficient in terms of both turns and elapsed time, the SI users report higher user satisfaction. A multivariate linear regression with user satisfaction as the dependent variable shows that ASR performance, user perception of task completion, and elapsed time are significant contributors to user satisfaction, but that ASR performance is the greatest contributor. We interpret these results to mean that the qualitative behaviors associated with poor ASR performance, the predictability of the system, and the ability of users to acquire a model of system performance, are more important than the commonly assumed performance factors of efficiency and task success. A user preference for SI style interfaces are also suggested by the results of other work. Previous work has found that directive prompts, and dialogue strategies that structure the interaction with the user, in a similar way to our system initiative interface, are more successful and preferred by users [2,4]. However, in all of these studies, ASR performed worse in the less constrained interface. Future work should examine the role of learnability and predictability with improved ASR performance. In addition, future work should include a longer term study of daily users in the field to determine whether MI interfaces might be preferred by some (expert) users.

DESIGNING A SPOKEN LANGUAGE INTERFACE FOR EMAIL


" What Judge Kavanaugh’s 'Originalism' Means for the Country ," JustSecurity , July 13, 2018

"What Judge Kavanaugh’s 'Originalism' Means for the Country," *JustSecurity*, July 13, 2018

Christopher Jon Sprigman, What Judge Kavanaugh’s 'Originalism' Means for the Country, JustSecurity (July 13, 2018), https://www.justsecurity.org/59392/judge-kavanaughs-originalism-means-country/

What does President Donald Trump’s nomination of Judge Brett Kavanaugh spell for the future of American constitutional and democratic life? Like Trump’s first pick for the Supreme Court, Justice Neil Gorsuch, Judge Brett Kavanaugh is virtually certain to push the Court further to the right. It is entirely reasonable to expect that in the next few years Roe v. Wade will be overturned, or at least neutered, as the Court’s right wing majority approves increasingly draconian restrictions on a woman’s right to choose an abortion. But the consequences of Kavanaugh’s appointment will reach much further. Kavanaugh’s writings and judicial record make clear his hostility to a range of economic and environmental regulations, to unions, and to workplace safety rules. While a judge on the D.C. Circuit, he voted to uphold a South Carolina voter ID law that civil rights groups complained would suppress minority voting. And his writings suggest that he is skeptical either that the Constitution permits a sitting President to be indicted, or to even be criminally investigated. But Kavanaugh’s nomination presents a question that is much deeper than whether the Court is destined to tack to the right on particular policy questions. At stake is what role the Court plays in our democracy — and, in particular, how frequently, and on what grounds, the Court’s rulings override and frustrate majority rule. For Brett Kavanaugh isn’t just a conservative. Like Justices Thomas and Gorsuch, Kavanaugh is a constitutional originalist. Defining the Terms of Discussion: Originalism and Constitutional Democracy Originalism. That term is used, and misused, often in the public debate. What does it mean to say that Brett Kavanaugh is an “originalist?” It means that Kavanaugh believes — or that he would at least like us to believe that he believes — that the meaning of the Constitution was fixed at the time of its adoption, and that judges are bound to adhere to that original meaning. Originalism is often contrasted with “living constitutionalism,” or the view that the meaning of the Constitution can evolve over time, and that judges are bound to interpret the text in a way that preserves the document’s deeper purposes and applies them to modern conditions. A large measure of originalism’s appeal lies in its seeming simplicity. The meaning of the text, originalists maintain, was fixed at the time of the Constitution’s adoption. Unlike living constitutionalism, in which the meaning of the text may shift over time, originalism promises to align our democracy with a fixed star. Another purported selling point for originalism is that it’s democratic. Judges who are committed to upholding the Constitution’s original meaning, proponents of this theory contend, are restrained from imposing their own political preferences on our democracy. In contrast, judges who believe in a “living Constitution” are practically invited to shape the Constitution’s meaning according to their own political whim. Simple, stable, democratic, politically neutral. That is how originalism is sold. But that’s not at all how it actually works. In the real world, originalism doesn’t operate as a simple, stable, or even neutral, framework for understanding the meaning of the Constitution. And originalism does little to restrain the discretion of judges. Quite the opposite. Often originalism functions as a sort of rhetorical smoke-screen. Originalist judges pledge allegiance to the Constitution’s original meaning, while actually imposing their partisan — and almost uniformly conservative — political preferences. I’ll say more about originalism’s close ties with political conservatism in a moment. But first, a few words about its infirmities both as a theory of how to understand the meaning of the Constitution, and as a tool for restraining judicial discretion. The United States is a constitutional democracy. And for most Americans, those two words — “constitution” and “democracy” — seem to go together like (to use an old reference that an originalist might nonetheless find unbecomingly contemporary) Bogey and Bacall. But that’s a naive view. Our Constitution and our democracy are indeed mutually supporting. Each gives structure and legitimacy to the other. But they are also in considerable tension. This isn’t some academic point — it’s one with very real consequences. Many times every year, federal judges declare that some law passed by a legislative majority, or some action of a democratically-elected official like the President or the governor of a state, is inconsistent with the Constitution and is therefore invalid. Courts striking down legislation, or thwarting the actions of democratically-elected officials, is something that happens so regularly in America that we take it for granted. But we should remember that what the courts are doing in these cases is extraordinary in a democracy. A judge is displacing the decision of a current majority in favor of some inconsistent rule put in place by people long dead. A judge is, in short, allowing the dead to govern the living. That isn’t democratic. It’s necrocratic. And this “dead hand” problem is endemic to constitutionalism generally. All constitutions that are backed by judicial review sometimes privilege decisions made in the past, decisions that living voters never approved, over decisions made by democratic majorities today. The question for how we govern ourselves isn’t whether this happens in a constitutional democracy. It does, and to some extent, it must. The question is how often, and on what basis. Originalist judges have shown little reluctance to override the decisions of current democratic majorities; they do so at least as often as judges who believe in a “living Constitution.” Are they doing so based on a reliable understanding of what the fixed, original meaning of the Constitution? Or are they acting out their political beliefs? Those are the vital questions. Part of the answer may be found by asking whether originalism succeeds, on its own terms, as a theory of constitutional interpretation. The short answer: it does not. Originalism is easy to describe in the abstract. But it is actually very difficult to do either well or honestly in practice. How Originalism “Works” in Judicial Decision-Making A good example of how originalism falls short is Justice Antonin Scalia’s opinion in District of Columbia v. Heller, the landmark 2007 decision. The Supreme Court, divided 5-4 along partisan lines, found (for the first time more than 200 years after the Constitution and Bill of Rights were adopted) that the Second Amendment guaranteed an individual right to bear arms, rather than simply guaranteeing the right of states to maintain “well regulated militia” (as the preface to the Second Amendment suggests), or a narrower individual right to bear arms only in connection with militia service. On the basis of this broad interpretation of the right granted by the Second Amendment, the five-justice majority struck down a DC ordinance banning handgun ownership. In his opinion for the majority, Justice Scalia inquires into the original understanding of the Second Amendment, which reads: “A well regulated Militia, being necessary to the security of a free State, the right of the people to keep and bear Arms, shall not be infringed.” Consider first what Justice Scalia (tellingly) labels the Second Amendment’s “prefatory clause” — the piece about a “well regulated Militia, being necessary to the security of a free State …” Does this language limit the scope of the right found in what Justice Scalia (also tellingly) labels the “operative clause” — i.e., the part about “the right of the people to keep and bear Arms” — or is it best understood as some sort of constitutional throat-clearing? Justice Scalia, rejecting the arguments of professional linguists, held that the prefatory clause simply provided a broad purpose with which the right must be consistent, but did not itself limit the right. Justice Stevens, in his dissent for the Court’s four liberal justices, disagreed. He argued that the Court should have read the two clauses together, and that under ordinary interpretive principles the first clause must be understood to limit the scope of the right granted by the Second Amendment read as a whole. Both Justice Scalia and Justice Stevens mustered substantial textual and historical arguments and citations in favor of their positions. Justice Scalia next proceeded to consider the original meaning of what he called the operative clause; in particular, the words “keep and bear arms.” Is this a phrase that, at the time of the adoption of the Bill of Rights, referred specifically to the military use of firearms? Or did it refer to a personal right unconnected to military service? On this question, Justice Scalia consults founding-era dictionaries and treatises, as well as English and colonial laws. Reading these, he concludes that meaning of “keep and bear arms” was not restricted to military uses, but sets forth a broad individual right to bear arms. On the other hand, Justice Stevens, consulting similar sources, including an 18th-century thesaurus and a different edition of one of the same old dictionaries on which Justice Scalia relied, reached the exact opposite conclusion. The original understanding of “keep and bear arms,” Justice Stevens concluded, focused on military service. One may tilt toward Justice Scalia, or Justice Stevens. Most fair-minded readers, I believe, find both the majority and the dissent entirely plausible — both make some nice points and both contain a few serious vulnerabilities. Crucially, neither the majority opinion nor the dissent are commanding. And this is a problem. A five-member majority of the Supreme Court announced in Heller a constitutional rule that invalidates a wide array of gun control laws that have been or could otherwise be democratically enacted. And those five people did so based not on some powerful, clear constitutional command, but rather on a so-called original meaning that is, at best, hotly contested. So originalism promises certainty, but in most cases that reach the courts it delivers, at best, “plausibility.” That should not be enough for judges to set aside the decisions of contemporary democratic majorities. For even if one accepts that we should defer to the preferences of the Founders — and to be clear, I don’t accept that uncritically — our democracy belongs to the living. We should at least not substitute the Founders’ preferences for our own unless the Constitution is speaking clearly. It’s an old, terse document, written by people we’ve never met, according to a process that was poorly documented, and in an antique language of vague generalities that rarely gives clear answers when applied to modern problems. We should heed the Constitution’s commands. Not its faint suggestions. Which brings me back to the connection between judicial originalism and political conservatism. Originalists claim that their interpretive method is politically neutral. However, on the federal bench, originalism is a right-wing sport. That is no accident. Originalism pins us to the distant past. A past that predates both the tremendous growth in U.S. population and in the scope and power of the federal government starting with the New Deal and continuing through the Second World War and the Great Society programs. Originalism valorizes a distant past that can be weaponized to halt and even reverse the progressive transformation of society accomplished during that long era of liberal dominance. Some right wing judges are hiding behind originalism, using it as a stalking horse for the very thing that originalism promises to prevent — partisan judging. These judges exploit originalism’s usefulness in imposing conservative policy preferences, but have no deep dedication to the method itself. You can see this in the selectivity with which some judges will either use or ignore originalism, depending on which way it points in a particular case. As Eric Segall and others have noted, Justice Scalia, in particular, was known for this sort of “opportunistic originalism.” For these judges, they have the choice of either picking up originalism when it suits their favored outcome and discarding it in other important cases, or they may very well stick with originalism across cases as they know the framework will in general and on average provide the policy results they have come to believe are just. But other originalist judges, the ones operating in good faith, are perhaps no less prone to imposing their partisan preferences in the name of the Constitution. These judges presumably tend to suffer, as all of us do, from a version of the confirmation bias that afflicts decision-making. Without systematic mechanisms in place to overcome such bias, they will often see certainty in uncertain arguments for original meaning when those arguments reinforce policy outcomes they like. An originalist judge who unconsciously aligns what the Constitution means with what he wants it to mean is behaving in a way that is altogether human. But it’s no way to run a constitutional democracy. So what to do about Brett Kavanaugh, the conservative originalist? In the end, and barring some change in the current political situation, Kavanaugh probably gets confirmed. But this is where I hope that Americans are ready for some new thinking. The problem isn’t just the man, or his politics, or his interpretive ideology. The problem is, more deeply and more fundamentally, that our system has slid too far away from democracy, and too far down a road that leads to the rule of judges. The answer, I’d submit, is not to demand the appointment of fewer originalists, and more living constitutionalists, to the federal bench. Viewed from a certain altitude, originalism and living constitutionalism aren’t all that different. They are both ways of interpreting the Constitution that tend to expand judicial power at the expense of democracy. The answer is to shrink the power of the federal courts over our lives, including the power of the Supreme Court. But how to do that given the systems we have inherited and the norms we have come to accept? One way to do that would be to end life tenure for judges – in a system where judges don’t serve for 40 years, politicians have less incentive to try to stock the bench with partisan ideologues. With Congress in the hands of bold lawmakers, that could perhaps be done without amending the Constitution. This may be too innovative by half but Article III of the Constitution only says federal judges “shall hold their Offices during good Behavior.” As Stuart Taylor noted, the first 10 justices served an average of under eight years. Only two lived to age 70. The 90 justices who had completed their terms by 1970 retired (on average) after 15 years on the bench, at age 68. Given life spans in the late 1700s, the Founders probably didn’t foresee the prospect of federal judges ruling over multiple generations. Couldn’t Congress simply pass a law defining “good Behavior” as service for a maximum of 12 years, or until the age of 68, whichever comes first? And then Congress could strip the federal courts of power to review that law using the Constitution’s Exceptions Clause (the Exceptions Clause (article III, § 2, cl 2) allows Congress to make exceptions to the Supreme Court’s appellate jurisdiction. Article III also gives Congress power to “ordain and establish” inferior federal courts; the power to establish the lower federal courts includes the power to limit their jurisdiction. Or perhaps the Senate could impose a rule that it will only confirm judges who commit to serving for a limited time, with impeachment of judges as a device for enforcing the commitment. If those sound extreme, well, maybe they are. But maybe they aren’t, compared to another strategy. It’s worth remembering that there’s nothing in the Constitution that says that courts have the power to strike down legislation. Or that, if they do, that Congress cannot simply override their decisions by an ordinary majority. So, if we are worried that judicial review is out of control, and that democracy is being subverted by the rule of judges, we may benefit by looking north. By which I mean, to Canada. In the Canadian constitution is something called the “Notwithstanding Clause.” That clause allows the legislature to declare that a law shall operate “notwithstanding” a judgment of the Supreme Court of Canada. That is, the clause allows national or provincial legislatures to override a court’s decision to strike down a law. The Notwithstanding Clause has seldom been used (but Canada has other mechanisms in place like mandatory retirement ages, which makes it less necessary). Regardless, the presence of the clause in the Canadian system resets the balance of constitutionalism in a way that favors democracy. We could have the equivalent of a Notwithstanding Clause in the U.S. And again, we wouldn’t have to amend the U.S. Constitution to get it. How could that be done? Congress could pass a law stating that Congress may override any Supreme Court decision (other than the negligibly small area of the Court’s original jurisdiction) by the ordinary legislative process. This undoubtedly sounds like a recipe for conflict. So be it. Americans — and liberals in particular — need to get over their false belief that the Constitution is sacrosanct. The Constitution is a human creation; worshiping it is a form of idolatry. Americans need to understand that too much constitutionalism, like too much democracy, can be dangerous to self-governance and even destroy a society. And most of all, they need to rediscover a fact that liberals in the New Deal era understood implicitly — that the surest and most durable route to progressive change is through the ballot box, and not the courthouse doors. Which brings me, finally, back to Brett Kavanaugh. The confirmation battle to come is important, but not as important as what should come next. The best future for America is one in which we have curbed the power of federal courts, and reestablished the primacy of democracy. In such a future, Supreme Court confirmation hearings may even generally pass mostly unnoticed by the public. As they should, in a working constitutional democracy. Photo: Judge Brett Kavanaugh listens to Sen. Rob Portman (R-OH) talk about Kavanaugh’s qualifications before a meeting in the Russell Senate Office Building on Capitol Hill July 11, 2018 (Chip Somodevilla/Getty Images)


" What Role for Antitrust in Regulating Platforms? ," Communications of the ACM, May 2020, at 36

"What Role for Antitrust in Regulating Platforms?," Communications of the ACM, May 2020, at 36

C. Scott Hemphill, What Role for Antitrust in Regulating Platforms?, Comm. ACM, May 2020, at 36

Using regulation to protect competition and innovation. vides important competition for Apple. Platforms also level the playing field for others. Amazon's cloud computing services enable startups to scale up without investing in infrastructure, while Google's search ads help them connect with customers at low cost. Moreover, competing hard is not an antitrust violation. For example, the leading platforms have made enormous investments in machine learning and related hardware. The result is a better algorithm, with more relevant ads and product recommendations, and an improved user interface such as digital assistants. These product improvements are competition in action, even if they make life difficult for competitors. An important antitrust problem arises when a platform, rather than competing on the merits, excludes a rival through its customer contracts. For example, during the browser wars, Microsoft's contracts with PC manufacturers restricted their promotion of the Netscape browser. Such contracts risked depriving Netscape of a critical mass of Amazon stand accused of various tactics to thwart competition and protect their market position. The Justice Department, Federal Trade Commission (FTC), and others are investigating the platforms for possible antitrust violations. These developments call to mind the browser wars and Microsoft's legal battles in the 1990s and 2000s. They also raise an important question: What role does antitrust have to play this time around? As I explain, antitrust law is highly relevant to some-but not all-of the critics' complaints. If Google uses customer contracts to weaken Bing, antitrust law can and should step in. Likewise if Facebook bought Instagram, back in 2012, to neutralize it as a competitive threat. More challenging are complaints about product design, such as a platform's arrangement of search results. Even further afield are concerns the platforms are copying their rivals' best ideas. If Amazon copies another seller's decision to market a particular product, no antitrust issue is raised, but copying combined with deception would raise serious concern. Platforms enhance competition in several respects. One way is by competing in adjacent businesses beyond their "home" market. 1 For example, Google's Android mobile operating system pro- weaken rivals? A change having both positive and negative effects is unlikely to fail this test. This lax attitude toward product design choices reflects a deep-seated reluctance to condemn product improvements. Moreover, even where an adverse effect is likely, remedies can pose a further barrier to action. Antitrust enforcers routinely block enforcement of a problematic contract. They are much more reluctant to wade into the complex details of product design, particularly where doing so requires ongoing supervision of the design. A second antitrust problem arises when a platform acquires its rival. For example, imagine Microsoft had bought Netscape instead of acting to exclude it by contract. (In fact, Microsoft approached Netscape about buying or licensing Netscape's browser code.) The harm would be similar, but without the conflict or complaining victim. Platform acquisitions that thwart competition are illegal. If Facebook customers, thereby inhibiting its development into a Windows rival. Contracts that weaken a rival's offering by limiting its access to customers are a major focus of platform antitrust enforcement. Critics see a replay of the browser wars in Google's contracts with Android handset manufacturers, which the European Commission has condemned as unduly restricting user access to competing search engines and browsers. One challenged provision had required the manufacturers to preinstall the Google search and Chrome browser apps as a condition of access to the Google Play Store, a must-have utility for Android users seeking thirdparty apps. The Commission was not persuaded by the fact that downloading the Bing app today is a lot easier than browser distribution in the 1990s. A second set of contracts pertains to Google's strong position as an intermediary between advertisers and ad inventory providers. Here, the concern is that Google might have used various contracts to weaken the competitive prospects of rival intermediaries. Platforms can also exclude rivals without using contracts, by making design choices that favor their own related businesses over rivals. For example, starting in 2007, Google introduced specialized product search results alongside the traditional "10 blue links" on the search engine results page. These so-called "product universals" pointed to third-party merchants offering the product for sale. Critics allege that by promoting these product universals, Google undermined competing comparisonshopping search engines by pushing them lower in its results. At the same time, Google arguably improved its user experience by sending users more directly to product listings rather than indirectly to another search engine. The overall competitive effect of a design change can be difficult to assess, particularly where a change constitutes a genuine product improvement yet also has a tendency to weaken rivals. One influential approach has been to ask: Does the profitability of the change depend on its tendency to viewpoints appropriate the benefits. If Google repackages content from the New York Times, while keeping a sizable fraction of the associated advertising revenues, that might harm the incentive to gather news. When Facebook copies Snap's Stories feature, a sort of temporary slideshow, this might give pause to the next would-be Snap. The problem is compounded if the copying is one-way, with the platform always the pirate, never the pirated. Platform copying becomes an antitrust issue when combined with platform deception, such as manipulation of search results. Suppose an Amazon customer is led to believe the top search result represents the best deal, when actually it delivers the greatest profit for Amazon. The FTC has not faced a case exactly like this. However, it has challenged deception as a competitive harm on multiple occasions. And outside the antitrust sphere, the FTC has repeatedly reminded search engines of their obligation to clearly distinguish ads from "natural" search results. Manipulation of product discovery, if proven, would be a problematic distortion of the competitive process. This brief discussion of platform competition issues supports several conclusions. Antitrust enforcement is a powerful tool for blocking problematic contracts and mergers, less so for questions of platform design. That does not mean that design choices, if established to harm competition, should escape scrutiny. Instead, new rules or regulators may be called for. Beyond antitrust, copying may raise important questions of innovation policy. However, the crucial underlying empirical question-to what extent is innovation really being suppressed-remains unanswered. tried to buy Snap, the FTC would likely move to block the deal on concerns ranging from reduced privacy to higher ad prices and slower innovation. But some platform acquisitions are less clear cut. If Microsoft had bought Netscape, would the Justice Department have perceived a sufficiently clear competitive threat to challenge the deal? Facebook's acquisition of Instagram presents just this issue. Instagram-today a major growth engine for Facebook-was in 2012 a fastgrowing mobile photo-sharing app. Had Instagram stayed independent, it might have become a full-fledged competitor to Facebook. Acquisition targets sometimes become giant-killers-consider, for example, Yahoo's missed opportunity to buy Google. Instagram in particular had ample venture capital support, and commentators at the time recognized Instagram had found and exploited Facebook's struggles in mobile photo sharing. Facebook recognized Instagram as a rising threat, and indeed, a top company official reportedly wrote colleagues that the point of the deal was to eliminate a competitive threat. Where a powerful platform such as Facebook is concerned, protecting competition is particularly important. 2 The public benefits of competition are unusually great, given that Facebook faces limited competition within the market and few plausible challengers in the near term. Meanwhile, Facebook has the capacity and incentive to suppress competition, even from relatively long-shot rivals. One broad lesson of the Microsoft litigation is that antitrust law protects "nascent, albeit unproven competitors … particularly in industries marked by rapid technological advance and frequent paradigm shifts." a Netscape was a nascent competitor; so was Instagram. As applied to acquisitions, protecting platform competition requires a willingness to block a deal even when we are unsure what will happen otherwise. It also sometimes means revisiting a consummated deal after the fact. For example, suppose Facebook indeed bought In- stagram in order to eliminate a competitive threat, but failed to share its true intention with the FTC at the time. An after-the-fact antitrust suit to challenge the acquisition might be appropriate, once its purpose and effects are better understood. A third complaint is that platforms copy the products and marketing decisions of smaller rivals. For example, Amazon has a knack for spotting profitable products sold on amazon.com-Duracell batteries, say-and introducing its own version. Antitrust's first response is likely to be, so what? Private-label goods are a familiar sight in supermarkets. The copyist's strategy does not depend on weakening the copied firm. Often, copying supports competition and lower prices. Thus, copying is ordinarily of little interest to antitrust. Platform copying can be more complex than the supermarket example, due to the platform's participation in multiple related businesses and its informational advantage at the center of a web of sellers and customers. For example, Amazon runs a popular Marketplace service for third-party sellers. If Amazon observes a Marketplace seller is having success selling a particular toy, it might decide to sell the the same toy itself. But here, too, copying tends to intensify competition. Though not ordinarily an antitrust issue, copying can raise important policy questions. In principle, platform appropriation might be so rapid, systematic, and effective as to discourage certain innovations. A Marketplace seller might not invest in marketing new products, lest Amazon Antitrust enforcement is a powerful tool for blocking problematic contracts and mergers, less so for questions of platform design.

Law and Technology What Role for Antitrust in Regulating Platforms?