Endogenous Choice of Stakes Under Common Ownership

" Endogenous Choice of Stakes Under Common Ownership ," 1 University of Chicago Business Law Review 217 (2022) (with Marcel Kahan) Abstract We present a simple model of common ownership in which an investor chooses its stake in competing firms in light of the effects on firm behavior and firm profits. Two firms compete in Cournot duopoly, and ownership affects a firm’s objective function in the manner posited by Bresnahan & Salop (1986) and Salop & O’Brien (2000). We show that an investor with equal stakes in both firms—?a so-called common concentrated owner (CCO)—?places a greater value on an additional share of a firm, compared to atomistic owners. The same is true of a noncommon concentrated owner (NCO) with a stake in just one firm. Both the CCO and the NCOs thus have incentives to acquire any shares held by atomistic owners. Our model yields two testable empirical predictions. First, equilibrium ownership structure in noncompetitive industries should be systematically more concentrated than in competitive industries. Second, within the investment portfolio of institutional investors, holdings in noncompetitive industries should be systematically more concentrated than holdings in competitive industries.

"Endogenous Choice of Stakes Under Common Ownership," 1 *University of Chicago Business Law Review* 217 (2022) (with Marcel Kahan)

									Abstract

We present a simple model of common ownership in which an investor chooses its stake in competing firms in light of the effects on firm behavior and firm profits. Two firms compete in Cournot duopoly, and ownership affects a firm’s objective function in the manner posited by Bresnahan & Salop (1986) and Salop & O’Brien (2000). We show that an investor with equal stakes in both firms—?a so-called common concentrated owner (CCO)—?places a greater value on an additional share of a firm, compared to atomistic owners. The same is true of a noncommon concentrated owner (NCO) with a stake in just one firm. Both the CCO and the NCOs thus have incentives to acquire any shares held by atomistic owners. Our model yields two testable empirical predictions. First, equilibrium ownership structure in noncompetitive industries should be systematically more concentrated than in competitive industries. Second, within the investment portfolio of institutional investors, holdings in noncompetitive industries should be systematically more concentrated than holdings in competitive industries.

C. Scott Hemphill & Marcel Kahan, Endogenous Choice of Stakes Under Common Ownership, 1 U. Chi. Bus. L. Rev. 217 (2022)

A common concentrated owner (CCO) holds stakes in competing firms. 1 Antitrust theorists have long posited that the interests of a CCO differ from those of an owner of a single firm. Economists have developed models in which, depending on its ownership structure, a firm maximizes a weighted average of its own and its competitors' profits. 2 Specifically, greater CCO ownership induces a firm to place a greater weight on competitor profits. At the same time, greater ownership by concentrated owners who do not hold stakes in competing firms-noncommon concentrated owners, or NCOs-reduces that weight. Recent empirical work has found that an increased level of CCO ownership is associated with anticompetitive effects. Other papers find no effect. This literature has generated a heated debate about whether common ownership in concentrated industries is compatible with the antitrust laws and whether it should be restricted. 3 In examining the relationship between CCO ownership and anticompetitive effects, it is important to be clear about the potential mechanisms that produce such effects. An influential model developed by Timothy Bresnahan, Steven Salop, and Daniel O'Brien is based on an internalization theory or so-called "unilateral effects." 4 In these models, a common owner who owns shares in firms A and B exerts some influence over firm A, and uses that influence to induce firm A to maximize, rather than its own profits, some weighted average of firm A and firm B profits-that is to have firm A internalize, to some extent, the effect of its actions on firm B. That common owner may do likewise with respect to firm B. But, importantly, its influence over firm A and its influence over firm B are independent: that is, whether the 1 This definition thus excludes common ownership of firms producing complements. Control, 67 ANTITRUST L.J. 559, 559-614 (2000). Investors, 82 ANTITRUST L.J. 221 (2018); Fiona Scott Morton & Herbert Hovenkamp, Horizontal Shareholding and Antitrust Policy, 127 YALE L.J. 2026 (2018). common owner changes firm A's objective function has no bearing on whether firm B's objective function also changes. In such models, the interests of noncommon owners differ from the interests of common owners. Noncommon owners want firm A to maximize its own profits, rather than some weighted average. Accordingly, the Bresnahan/Salop/O'Brien approach employs a measure of common ownership that increases with CCO ownership and decreases with NCO ownership. 5 Similarly, the effect of ownership on firms generally changes as ownership becomes more concentrated. 6 Thus, for example, a single common owner of firms A and B that owns a 10% stake in each firm has a greater impact than five different common owners of A and B that each own a 2% stake in each firm. A 10% noncommon owner of firm A has a greater impact than five 2% noncommon owners. An alternative, less specified, theory of how common ownership generates anticompetitive effects relies on coordinated effects. The underlying premise is that common ownership somehow facilitates collaboration and collusion among competitors-say, the formation of an effective cartel. Since collusion (putting legal issues and possible sanctions to the side) benefits both firms, the interests of common and noncommon owners coincide. Importantly, under such a theory, a common owner's effect on firms A and B is not independent: the common owner cannot succeed in inducing A to collude with B without also inducing B to collude with A. The bulk of the empirical literature on anticompetitive effects of common ownership has, explicitly and implicitly, embraced the Bresnahan/Salop/O'Brien approach by employing a metric of common ownership that increases with CCO ownership and decreases with NCO ownership and where the effects of ownership are larger as ownership becomes more concentrated. 7 employed include the difference between the Herfindahl-Hirschman Index and the Modified Herfindahl-Hirschman Index (MHHI delta) and a related profit weight calculation that, like MHHI delta, reflects the differing effects of CCO and NCO ownership. Thus far, this literature has focused on how a particular ownership structure affects firm behavior and outcomes. The ownership structure is taken as given. However, if ownership structure affects firm value, then we would expect owners to alter their stakes in light of this anticipated effect. The literature to date has provided only a limited analysis of this issue. Most importantly, the literature largely has not examined the effects on ownership structure in the unilateral setting developed by Bresnahan/Salop/O'Brien that forms the basis of the current wave of empirical work. For example, earlier works by David Flath and David Reitman present models in the distinct context of crossownership, in which one firm owns a stake in a competing firm. 8 (Under common ownership, by contrast, a third party owns stakes in both competing firms.) In these models, the cross-owning firm exerts no influence on the owned firm, an assumption that is inconsistent with the Bresnahan/Salop/O'Brien approach, in which ownership does affect the actions of the owned firm. Rune Stenbacka and Geert Van Moer follow the same approach. 9 More recent work by Alessio Piccolo and Jan Schneemeier develops a model where the firm objective function is determined purely by the percentage of shares held by common owners. 10 Their model thus deviates from the Bresnahan/Salop/O'Brien Schenone, Common Ownership Does Not Have Anti-Competitive Effects in the Airline Industry (Fed. Rsrv. Bank of Atl., Working Paper 2019-15, 2019) ECON. 313, 313 (1994). approach and from the bulk of the empirical work, in which ownership concentration rather than common ownership share affects the firm objective function. Oz Shy and Rune Stenbacka model a market with two firms, two active investors, and many passive investors, to determine how the active investors should allocate their portfolios among the two firms.foot_4 Finally, Anna Bayona, Ángel López, and Anton-Giulio Manganelli examine the stability of common ownership arrangements in a Bresnahan/Salop/O'Brien framework. 12 Neither of these papers analyzes a setting where concentrated owners can acquire or dispose of shares by trading with atomistic, dispersed investors. In this paper, we endogenize the ownership choices of concentrated owners. Concentrated owners choose their stakes in light of the effects of ownership on firm behavior and firm (and competitor) profits. We analyze these choices within a simple model of competition in a duopoly where the two competing firms are owned by a mix of NCOs, CCOs, and atomized owners. Our model is highly stylized, featuring Cournot (quantity) competition between the firms and a single CCO with equal stakes in both firms. This simple model, an extension of the Bresnahan/Salop/O'Brien approach, captures the general features of theories of common ownership. We derive and characterize the equilibrium ownership structure. We show that an NCO, compared to an atomistic owner, places a greater value on an additional share of the firm. Hence, the NCO has an incentive to acquire shares held by atomistic owners. Likewise, a CCO with similar stakes in the firms, compared to an atomistic owner, places a greater value on an additional share. Thus, both types of concentrated owner have incentives to acquire shares held by atomistic owners. Our model yields two testable empirical predictions. First, equilibrium ownership structure in noncompetitive industries (that is, industries where ownership influences competition) should be systematically more concentrated than in competitive industries (where ownership does not influence competition). Second, within the investment portfolio of institutional investors, holdings in noncompetitive industries should be systematically more concentrated than holdings in competitive industries. The current empirical literature on common ownership has sought to test whether common ownership affects competition directly-for example, by examining the relationship between blockholding and firm outcomes (such as price levels). This approach has various difficulties, including the challenge of identifying an exogenous shift in common ownership. 13 Our article points to a different empirical approach that has the potential for producing additional evidence about the effects of common ownership. Two firms, A and B, produce quantities qA  0 and qB  0 of a good at a unit cost of 0. The firms compete in Cournot quantities and face a demand function of p = 1 -qA -qB. Shares in the firms are held by three concentrated owners-NCOA, NCOB, and CCO-and a large set of atomistic owners. NCOA holds shares in A, NCOB holds shares in B, and CCO holds equal stakes in both firms. Let nA equal the fraction of A's shares held by NCOA, nB equal the fraction of B's shares held by NCOB, and c equal the fraction of shares in both firms held by CCO, with 1 > nA, nB, c > 0. At t = 0, an initial ownership structure is exogenously determined. At t = 1, NCOA, NCOB, and CCO can sequentially buy shares from (or sell shares to) the atomistic owners of A and B at a price equal to the value of a share to the atomistic owner. 14 This process continues until NCOA, NCOB and CCO no longer want to buy shares from (or sell shares to) the atomistic owners. At the end of t = 1, a final ownership structure is determined. At t = 2, firms produce qA and qB generating profits of πA and πB with πA = pqA = (1 -qA -qB)qA 13 See Katharina Lewellen & Michelle Lowry, Does Common Ownership Really Increase Firm Coordination?, 141 J. FIN. ECON. 322 (2021). 14 Our results are robust to assuming instead that NCOA, NCOB, and CCO can buy shares from (or sell shares to) the atomistic owners of A and B at a price equal to the value of A or B, respectively, that will result after the sale or at any price between such value and the value of a share to the atomistic owner. Moreover, our results also hold for tender offers by NCOA, NCOB, and CCO directed at atomistic owners. Since firm A and B are identical in relevant respects, we present this and other results only for firm A. Following Salop and O'Brien, a firm with common owners maximizes a weighted average of its own and its competitor's profits. 15 Let wi represent the weight that firm i attaches to its own profits; 1 -wi is the weight that firm i places on its competitor's profits. We adopt the standard assumption in the common ownership literature (including Salop and O'Brien) that control weights are proportional to ownership stakes. 16 Following Salop and O'Brien, Firm A maximizes πA + πB * c 2 / (c 2 + nA 2 ) which means that wA = (c 2 + nA 2 ) / (2c 2 + nA 2 ) As 1 > nA, c > 0, it follows that 0.5 < wA < 1. Differentiating wA yields: wA′(nA) = 2c 2 nA/(2c 2 + nA 2 ) 2 > 0 Intuition: An increase in nA raises wA both because it gives NCOA greater control, relative to CCO, over A's policies (the control effect), and because it increases NCOA's interest in A (the ownership effect). wA′(nB) = 0 Intuition: An increase in nB has no effect on wA because NCOB neither has control over A's policies nor any direct interest in A's profits. wA′(c) = -2cn 2 /(2c 2 + nA 2 ) 2 < 0 Intuition: An increase in c affects wA in several ways. First, due to its increase in A ownership, CCO cares more about A's profits (the A ownership effect). Second, due to its increase in control over A relative to NCOA, CCO has a greater ability to induce A to reduce its output for the benefit of B (the A control effect). Third, due to an increase in B ownership, it increases CCO's incentives to use its existing control over A's policies for the benefit of B (the B ownership effect). There is no B control effect on wA. We can examine these effects more closely if we separate the c in the formula for wA into the A ownership component cAo, the A control component cAc, and the B ownership component cB, such that wA = (cAocAc + nA 2 ) / (cAocAc + nA 2 + cAccB). Taking separate derivatives of cAc, cB and cAo yields: wA′ (cB) = -cA(cA 2 + nA 2 )/ (cAocAc + nA 2 + cAccB) 2 < 0 Thus, the A ownership component is positive, while the A control component and the B ownership component are negative. Moreover, for cA = cB the size of the (negative) effect of an increase in cB exceeds the size of the (positive) ownership effect of an increase in cA. Hence, an increase in c reduces wA. Proposition 1: The equilibrium quantity produced by firm A increases as the weight firm A places on its own profits increases and as the weight firm B places on its own profits decreases. Proof: Differentiating 𝑞 * yields: 𝑞 * ′(wA) = 2wB(2wB -1) / (4wAwB -1) 2 > 0 (since wA, wB > 0.5) 𝑞 * ′(wB) = (1 -2wA) / (4wAwB -1) 2 < 0 (since wA, wB > 0.5) Intuition: An increase in wA increases 𝑞 * because it reduces the degree to which firm A restrains its own production for the benefit of firm B. An increase in wB reduces 𝑞 * because an increase in wB increases 𝑞 * which in equilibrium leads firm A to reduce its own production. Proposition 2: The equilibrium price decreases as the weight firm A places on its own profits increases and as the weight firm B places on its profits increases. Proof: Differentiating 𝑝 * yields: p*′(wA) = -(2wB -1) 2 /(4wAwB -1) 2 < 0 (since wB > 0.5), and likewise for p*′(wB). Intuition: An increase in wA increases overall production because its direct effect on qA exceeds its indirect effect on qB. As the overall quantity produced increases, price declines. Proposition 3: The equilibrium profits of firm A increase as the weight firm A places on its own profits increases and as the weight firm B places on its own profits decreases. The University of Chicago Business Law Review [Vol. 1:217 πA′(wA) = wB(2wB -1) * (4wA + 4wB -4wAwB -3) / (4wAwB -1) 3 . The denominator and wB(2wB -1) are positive for wB > 0.5. The expression (wA + 4wB -4wAwB -3) is positive for wA, wB ∈ (.5, 1) as it is 0 at wA, wB = (0.5, 0.5) and it is increasing in both wA and wB. Hence πA′(wA) > 0. Since qA′(wB) < 0 and p′(wB) < 0, it follows that πA′(wB) < 0. Intuition: Firm A giving more weight to its own profits implies an increase in A's profits. Firm B giving more weight to its own profits reduces both the equilibrium quantity for firm A and the price and hence must reduce A's profits. Intuition: An increase in NCOA ownership in firm A raises the profit weight firm A places on its own profits. This induces firm A to increase its quantity produced. A decrease in NCOB ownership in firm B decreases the profit weight firm B places on its own profits. This induces firm B to decrease its quantity produced, which induces A to increase its quantity produced. Proposition 5: The equilibrium total industry quantity increases as NCOA ownership increases, as NCOB ownership increases, and as CCO ownership decreases. Proof: An increase in ownership in firm A by NCOA increases wA and has no effect on wB. Differentiating equilibrium industry quantity 𝑞 * on wA yields: which is positive. Likewise, for an increase in ownership in firm B by NCOB. A change in CCO ownership affects both wA and wB. To obtain the effect of a change in CCO ownership on 𝑞 * and 𝑞 * , we differentiate: Adding 𝑞 * ′ and 𝑞 * ′ yields 𝑞 * ′ 𝑞 * ′ + 𝑞 * ′ = [wA′(4wB wB -4wB + 1) + wB′(4wA wA -4wA + 1)] / (4wA wB -1) 2 which is negative since the numerator is negative (as wA′ and wB′ are both negative and (4wB wB -4wB + 1) and (4wA wA -4wA + 1) are positive when w > 0.5) and the denominator is positive. Intuition: An increase in NCOA ownership in firm A raises the profit weight firm A places on its own profits. This induces firm A to increase its quantity produced. While the increase in the quantity produced by firm A, in turn, induces firm B to lower its quantity produced, the increase in firm A's quantity exceeds the reduction in firm B's quantity. A decrease in c leads each firm to give relatively greater weight to its own profits and relatively less weight to the other firm's profits. The combined effect of this dual increase in profit weights is to increase industry quantity produced. Proposition 6: The equilibrium price increases as the ownership in firm A by NCOA decreases, as the ownership in firm B by NCOB decreases, and as CCO ownership increases. Proof: If equilibrium industry quantity increases, price decreases, and vice versa. Proposition 7: NCOA will always want to buy shares of A from atomistic owners as long as atomistic owners own such shares. NCOB will always want to buy shares of B from atomistic owners as long as atomistic owners own such shares. CCO will always want to buy shares of both firms from atomistic owners as long as atomistic owners own such shares. Any marginal share of A will have a greater value to NCOA than it does to atomistic owners of A. The reason is as follows. An acquisition of a marginal share of A by NCOA has two effects. First, NCOA acquires the value of the marginal share itself (which, conditional on sale, is the same for NCOA as it is for an atomistic owner). Second, increasing nA changes the value of the stake NCOA owned in A before NCOA acquires a marginal share. Specifically, increasing nA increases wA and thus πA (since πA′(wA) > 0) -and hence the value of the pre-acquisition stake held by NCOA. This effect is present for any nA > 0 and any wB. Hence, NCOA will have an incentive to acquire shares of A held by atomistic owners. If cA = cB, then any marginal share of A will have a greater value to CCO than it does to atomistic owners of A. If cA = cB, then an increase in cA will reduce overall quantity and hence increase industry profits.foot_8 Since cA = cB, the value of the combined stake in A and B owned by CCO before it acquires a marginal share increases if industry profits increase. In addition to obtaining the value of the marginal share itself (which, conditional on sale, is the same for CCO as it is for an atomistic owner), CCO benefits from this increase. The result that an NCO attributes a greater value to a marginal share than an atomistic owner follows directly from two of the premises underlying the theoretical literature on common ownership and will hold generally. The first premise is that common owners are interested in having a firm take actions that maximizes the value of their common ownership stakes rather than the value of that firm. The second premise is that increased ownership confers upon the owner some degree of increased control over firm decisions. Both of these premises are necessary to generate the predictions of the common ownership literature. But, by the same token, if an NCO increases its ownership, it will obtain some marginal degree of increased control over firm decisions which it will use to induce the firm to take marginally more actions that maximize firm value, thus increasing firm value. Since an NCO-unlike an atomistic owner-has a pre-existing stake in the firm that rises as firm value increases, an NCO will inevitably attribute a greater value to a marginal share than an atomistic owner. The result that CCO attributes a greater value to marginal shares of A and B than atomistic owners, as well, follows from the basic premises underlying the common ownership literature. A common owner who holds equal stakes in all competing firms wants to maximize industry profits. Holding a greater stake in all firms will give that common owner more control and induce firms to take marginally more actions that increase industry profits. This will raise the value of the common owner's pre-existing stake, with the result that the common owner will attribute a greater value to marginal shares than the atomistic owners. Since the NCOs and the CCO all want to acquire shares from atomistic owners, the equilibrium ownership structure in the model entails no shareholdings by atomistic owners. This result of the model will hold in a more general setting as long as concentrated owners face no constraints in increasing ownership stakes. In reality, of course, concentrated holders face several constraints: among others, they may have limited access to capital, they may want to reduce risk by holding diversified portfolios, or they may want to preserve liquidity. While one would thus not expect that NCOs and CCOs will completely crowd out atomistic owners, the model predicts a higher level of ownership concentration in industries dominated by publicly traded companies where firm decisions affect the value of competing firms ("noncompetitive industries") than in industries dominated by publicly traded companies where such effects are largely absent ("competitive industries"). The results of our model also yield predictions for the portfolio structure of institutional investors. Institutional investors can invest in noncompetitive industries (where ownership affects profit weights) and in competitive industries (where ownership does not affect profit weights) and, in each set of industries, can take concentrated positions in one or several competitors in a single industry (or a few industries) or diversified positions in a larger number of companies in an industry. The model has no implications for portfolio allocations within competitive industries and, to the extent that institutional investors are interested in diversified portfolios, one would expect any investor to take diversified positions in the set of competitive industries. However, our model yields a prediction for portfolio allocations within competitive industries. 18 Given the amount of investments in the set of noncompetitive industries, it is more profitable to concentrate investments in a single or a small number of such industries (either as a concentrated position in one company in the industry or as concentrated positions in multiple competing firms in the same industry) than to spread the investments over a larger number of noncompetitive industries (with less invested in each industry). In addition, our model yields a prediction for portfolio allocations across types of industries. Starting with an equal allocation of investments in firms in competitive and noncompetitive industries, an institutional investor would benefit by shifting investments from firms in competitive industries to firms in noncompetitive industries. As a result, looking at the portfolio of an institutional investor, one would expect the portfolio to be overweight in the set of noncompetitive industries relative to the set of competitive industries and, within the set of noncompetitive industries, less diversified across industries than within the set of competitive industries. In this paper, we have presented a simple model of ownership structure in a setting where concentrated ownership-both common ownership and noncommon ownership-affects the objective function of a firm. Specifically, concentrated common ownership induces a firm to place less weight on its own profits, while concentrated noncommon ownership induces a firm to place more weight on its own profits. These effects on profit weights form the basis for both the main theoretical models of and the bulk of the empirical literature on the anticompetitive effects of common ownership. In our model, both NCOs and CCOs have incentives to raise their respective ownership stakes. If an NCO increases its ownership stake, the firm's objective function changes such that the prior stake held by the NCO increases in value. The same is true for a CCO who holds equal stakes in competing firms. Our formal model addresses only a duopoly with identical firms and a simple production industry demand function. But the intuition for why both concentrated noncommon owners and concentrated common 18 These predictions obviously do not apply to indexed investors as indexed investors are constrained in their investment allocations by the indexing strategies they follow. owners have incentives to raise their respective ownership stakes carries forward to a more general setting. The incentives of concentrated owners to increase their stakes in firms in oligopolistic industries yield empirical predictions both for the ownership structure of such firms and for the portfolio structure of institutional investors. As to ownership structure, our model predicts that ownership in firms in noncompetitive industries (where ownership structure affects profit weights) will be more concentrated than ownership in firms in competitive industries (where ownership structure does not affect profit weights). As to portfolio structure, our model predicts that institutional investors will invest more in noncompetitive industries than in competitive industries and that stakes in firms in noncompetitive industries will be more concentrated than stakes in firms in competitive industries. These predictions, in turn, can be used to test whether concentrated common ownership has anticompetitive effects. In particular, it is disputed to what extent concentrated owners can effectively induce a company to take actions that lower the value of the company but raise the value of their portfolio holdings. In previous work, we identified several reasons why doing so may be difficult, especially if it entails legal or reputational risks. 19 But we have also identified several strategies for achieving this result that seem feasible. The empirical literature to date, however, has not been able to establish whether investors in fact pursue such strategies. Part of the reason for this failure is the difficulty of structuring econometric tests that causally link certain outcomes (such as a change in prices) to ownership structure. The predictions we have developed in this paper point to a different, and perhaps more fruitful, strategy to determine the effect of concentrated ownership. TheUniversity of Chicago Business Law Review [Vol. 1:217 See Salop & O'Brien, supra note 2. See generally Rune Stenbacka & Geert Van Moer, Cross Ownership and Divestment Incentives, 201 ECON. LETTERS (2021).10 See generally Alessio Piccolo & Jan Schneemeier, Ownership and Competition (Dec. 9, 2021) (unpublished manuscript), https://perma.cc/KC28-XYGC. See Oz Shy & Rune Stenbacka, Active Investors, Passive Investors, and Common Ownership, 110 AEA PAPERS & PROC. 565 (2020). See Anna Bayona, Ángel L. López, & Anton-Giulio Manganelli, Common Ownership, Corporate Control and Price Competition (May 2021) (unpublished manuscript), https://perma.cc/G7HA-ZH8K. See Salop & O'Brien, supra note 2. Quantity will change by (4wA 2 -4wA + 1)(wB′ + wA′)/(4wA 2 -1) 2 which is negative since wA′ + wB′ = -2cn 2 /(2c 2 + n 2 ) 2 < 0. See Hemphill & Kahan, supra note 3. The metrics 5 Id. 6 7

Choice of Stakes Under Common Ownership 219

D. The Effect of Profit Weight on Firm Decisions

E. The Effect of Ownership on Firm Decisions

which is positive. Likewise, for an increase in ownership in firm B by NCOB.

Choice of Stakes Under Common Ownership 231

TheUniversity of Chicago Business Law Review [Vol. 1:217

See Salop & O'Brien, supra note 2.

See Salop & O'Brien, supra note 2.

See Hemphill & Kahan, supra note 3.

The metrics 5 Id. 6 7


"Evaluating Competing Agent Strategies for a Voice Email Agent," in Proceedings of Eurospeech '97 (1997) (with Marilyn A. Walker, Donald Hindle, Giuseppe Di Fabbrizio & Craig Mestel)

"Evaluating Competing Agent Strategies for a Voice Email Agent," in *Proceedings of Eurospeech '97* (1997) (with Marilyn A. Walker, Donald Hindle, Giuseppe Di Fabbrizio & Craig Mestel)

Marilyn A. Walker et al., Evaluating Competing Agent Strategies for a Voice Email Agent, in Proceedings of Eurospeech '97 (1997)

Marilyn A. Walker
Donald Hindle
Jeanne C. Fromer
Giuseppe Di Fabbrizio
Craig Mestel

Both of these agents support the same task functionality for voice access to email, and are implemented in a system called ELVIS (EmaiL Voice Interactive System). The Elvis agent in D1 is based on a system-initiative dialog strategy, whereas the Elvis agent in D2 is based on a mixed-initiative dialog strategy. The system-initiative strategy acquires information in small increments and constrains the user to single word utterances whose content is explicitly prompted for. The mixed-initiative strategy lets the user control the dialog, doesn't provide information unless the user asks for it, and allows the user random access to all the application functionality with utterances that combine a set of information requirements. It may seem obvious that the mixed-initiative strategy in D2 is preferable to the system-initiative strategy in D1. Previous work has emphasized the utility of mixed-initiative dialog strategies in advice-giving and diagnostic dialog domains [10,8]. However, other work suggests that the performance of the system-initiative agent may be superior [5,1,6]. One reason for this is the less than perfect performance of current speech recognizers. The mixed-initiative strategy requires more complex grammars, possibly resulting in higher automatic speech recognition (ASR) error rates. This in turn may lead to a higher overall task error rate, or extremely long repair subdialogs. A second potential problem is that the mixed-initiative strategy may require users to learn what the system can understand, since the system does not explicitly prompt them with valid vocabulary. This suggests that the mixed-initiative strategy may be more suitable for experienced users. However, spoken dialog agents have rarely been evaluated in the context of repeated use by a single user [7], as would be expected in the case of an email agent. It is likely that (1) users become more expert over time; and (2) in the future systems will adapt and learn. Thus it is important to evaluate changes in performance over repeated user sessions. We hypothesize that the more experience a user has with the system, the better the mixed-initiative strategy will perform. This paper describes the implementation of these two dialog strategies in an agent for accessing email by phone. We present the results of an experiment in which users perform a series of tasks by interacting with an email agent using one of the dialog strategies. We also describe how our experimental results can be framed in the PARADISE framework for evaluating dialog agents. In order to determine the basic application requirements for email access by telephone, we conducted a Wizard of Oz study. The Wizard simulated an email agent interacting with six users who were instructed to access their email over the phone at least twice over a four-hour period. In order to acquire a basic task model for email access over the phone, the Wizard was not restricted in any way, and users were free to use any strategy to access their mail. The study resulted in 15 dialogs, consisting of approximately 1200 utterances, which were transcribed and analyzed for key email access functions. We categorized email access functions into general categories based on the underlying application, as well as language-based requirements, such as the ability to use referring expressions to refer to messages in context (as them, it, that), or by their properties such as the sender or the subject of the message. Table 1 summarizes the functions used most frequently in our Wizard of Oz study; these frequencies were used to prioritize the design of the email application module. From this exploratory study we concluded that the email agent should minimally support: (1) reading the body of a message and the header information; (2) summarization of the contents of an email folder by content-related attributes, like sender or subject; (3) access to individual messages by content fields such as sender and subject; (4) requests for cancellation and repetition by the user and for clarifying help from the system. We implemented both the system-initiative and the mixed initiative versions of the email agent within a generalpurpose platform for voice dialog agents, which combines ASR, text-to-speech (TTS), a phone interface, an email access application module, and modules for specifying the dialog manager and the application grammars [4]. The email application demands several advanced capabilities from these component technologies. First, ASR must support barge-in, so that the user can interrupt the agent when it is reading a long email message. Second, the agent must use TTS due to the dynamic and unpredictable nature of email messages; prerecorded prompts are not sufficient for email access. Third, the grammar module must support dynamic grammar loading because the ASR vocabulary must change to support selection of email messages by content fields such as sender and subject. The application module is implemented as a separate module called the Email Application Interface (EMAI). EMAI is compatible with several e-mail server protocols, e.g. POP3, IMAP4 and SMTP, and is based on the standard known as RFC822. It also decodes the MIME standard for attachments (Text, Graphics, Application specific, Video, Sound), and provides the information back to the application. EMAI provides a set of basic email access functions to the dialog manager. One feature is access to message attributes such as subject, author, body, unique message ID, and any attachments. A second set of features supports selection of messages by message attributes such as subject or sender, or by positional attributes such as previous, next, or last. Third, sets of messages can be sorted by message attributes such as the Author's Reply Address, Date, Subject, Status, Length, or Priority. Fourth, messages and message folders can have their status information updated. For example, after the agent reads a message to a user the status may be changed from new to read, or after the user says 'delete it', the status is modified from new to deleted. Fifth, the application module preprocesses the message body for TTS to modify material not easily realized in speech [3]. Finally, the recursive structure of dialog requires EMAI to manage a stack of folders reflecting the dialog structure. The main advantage of EMAI to the dialog manager is that it abstracts from the implementation of the application system, so that the same application module can be used with different underlying mail systems. Both versions of the email agent have identical task functionality. The dialog manager uses a state machine to implement both the system-initiative and the mixed-initiative dialog strategy. Each state includes parameter specifications for: (1) an initial prompt, which the agent says upon entering the state (this may include a response to the user's current request); (2) whether barge-in is enabled; (3) a help prompt which the agent says if the user says help; (4) multiple rejection prompts which the agent says if ASR rejects the user's utterance; (5) multiple timeout prompts which the agent produces if the user doesn't say anything; and (6) a grammar specifying what the user can say. Transitions between states are driven by the semantic interpretation of user's utterances. There are two main differences between the mixedinitiative and system-initiative dialog strategies. First, the mixed-initiative agent does not volunteer information in its initial prompt, or explicitly tell the user what to say. This information is obtainable at the user's initiative by waiting for the timeout prompt to play or by saying Help. Secondly, as shown in D1 and D2, the system-initiative agent severely constrains the recognition grammar available at any point, and prompts the user for each increment of information needed to carry out an application function. In contrast, the mixed-initiative agent allows most responses at all times, and allows the user to speak in full sentences which specify multiple arguments to application functions simultaneously. In terms of the state machine, the mixed-initiative version's one main state corresponds to 8 system-initiative states, reflecting the fact that the system-initiative version is more restrictive and has more constrained ASR grammars. The experiment required users, randomly assigned to one agent or the other, to complete three tasks involving telephone access to email, in three separate conversations with the agent. All of the users regularly used computers in the course of their everyday work and were familiar with email. Instructions were given on three web pages, one for each experimental task. Each web page consisted of a brief general description of Elvis, a list of hints for using Elvis, a task description, and information on calling Elvis. Each page also contained a form for specifying information acquired from the agent during the dialog, and a survey, to be filled out after task completion, designed to probe the users' satisfaction with the system. Subjects read the instructions in their offices before calling Elvis from their phone. Each user performed three tasks in sequence, and each task consisted of two subtasks. Six users were assigned to the mixed-initiative agent and 6 users to the systeminitiative agent. Thus the experiment resulted in 36 dialogs representing 72 attempted subtasks. To be consistent with the PARADISE evaluation framework [9], each subtask was represented by a scenario where the agent and the user had to exchange information about criteria for selecting messages and information within the message body. For example, in one scenario the user is expecting email from Kim about a meeting and needs to find out the time and place of that meeting (as in Dialog D1 and D2). This scenario is represented in terms of the attribute value matrix (AVM) in Table 2 The task scenarios that the subjects were given were as follows, where scenarios 1.1 and 1.2 were done in the same conversation with Elvis, similarly for 2.1 and 2.2, and 3.1. and 3.2. Successful completion of a scenario requires that all attribute-values must be exchanged. The sender and subject attributes that are usable as selection criteria are known by the user at the beginning of the dialog, while the attributes to be extracted from the body of the email message are acquired from the agent in the course of the interaction. The AVM representation for all six subtasks is similar to Table 2. Note that the task's information-exchange requirement represented in the AVM is independent of the agent strategy used to accomplish the task. Experimental results were collected by three means, and a series of variables were extracted. First, all of the dialogs were recorded. This allows utterance transcription and checking aspects of the timing of the interaction, such as whether there were long delays for agent responses, and whether users barged in on agent utterances Barge In. In addition, it was used to calculate the total time of the interaction (the variable named Elapsed Time). Second, the system logged the agent's dialog behavior on the basis of entering and exiting each state in the state transition table for the dialog. For each state, the system logged the number of timeout prompts (Timeout Prompts) , ASR Rejections, and the times the user said Help (Help Requests). The number of System Turns and the number of User Turns were calculated on the basis of this data. In addition, the ASR result for the user's utterance was logged. The recordings were used in combination with the logged ASR result to calculate a concept accuracy measure for each utterance. Mean concept accuracy was then calculated over the whole dialog and used as a Mean Recognition Score for the dialog. Third, users were required to fill out the web page forms after each task specifying whether they had completed the task and the information they had acquired from the agent (Task Success), e.g. the values for Email.att1 and Email.att2 in Table 2. In addition, users responded to a survey on their subjective evaluation of their satisfaction with the agent's performance with the following questions: • Was Elvis easy to understand in this conversation? • In this conversation, did Elvis understand what you said? (ASR Performance) • In this conversation, was it easy to find the message you wanted? (Task Ease) • Was the pace of interaction with Elvis appropriate in this conversation? (Interaction Pace) • In this conversation, did you know what you could say at each point of the dialog? (User Expertise) • How often was Elvis sluggish and slow to reply to you in this conversation? (System Response) • Did Elvis work the way you expected him to in this conversation? (Expected Behavior) • In this conversation, how did Elvis's voice interface compare to the touch-tone interface to voice mail? (Comparable Interface) • From your current experience with using Elvis to get your email, do you think you'd use Elvis regularly to access your mail when you are away from your desk? (Future Use) Most question responses over values such as (almost never, rarely, sometimes, often, almost always), or an equivalent range. Each of these responses was mapped to an integer in 1 . . . 5. Some questions had (yes, no, maybe) responses. Each question emphasized the user's experience with the system in the current conversation, with the hope that satisfaction measures would indicate perceptions specific to each conversation, rather than reflecting an overall evaluation of the system over the three tasks. We calculated a Cumulative Satisfaction score for each dialog by summing the scores for each question. Our goal was to compare performance differences between the mixed-initiative strategy and the system-initiative strategy, when the task is held constant, over a sequence of three equivalent tasks in which the users might be expected to learn and adapt to the system. We hypothesized that the mixed-initiative strategy might result in lower ASR performance, which could potentially reduce the benefits of user initiative. In addition, we hypothesized that users might have more trouble knowing what they could say to the mixed-initiative agent, but that they would improve their knowledge over the sequence of tasks. Thus we hypothesized that the system-initiative agent might be superior for the first task, but that the mixed initiative agent would have better performance by the third task. Our experimental design consisted of three factors: strategy, task and subject. Each of our result measures were analyzed using a three-way ANOVA for these three factors. For each result we report F and p values indicating the statistical significance of the results. Effects that are significant as a function of strategy indicate differences between the two strategies. Effects that are significant as a function of task are potential indicators of learning. Effects that are significant by subject may indicate problems individual subjects may have with the system, or may reflect differences in subjects' attitude to the use of spoken dialog interfaces. We discuss each of these factors in turn. We first calculated Task Success using the κ (Kappa) statistic [9]. is the proportion of times that the AVMs for the actual set of dialogs agree with the AVMs for the scenario keys, and P(E) is the proportion of times that we would expect the AVMs for the dialogs and the keys to agree by chance. Over all subjects, tasks and strategies, P(E) = .50, P(A) is .95, and κ is .9. Thus users completed the assigned task in almost all cases. Results of ANOVA by strategy, task and subject showed that there were no significant differences for any factors for Task Success (Kappa), Cumulative Satisfaction, or Elapsed Time to complete the task. However there are differences in the individual satisfaction measures, which we discuss below. We believe the lack of an effect for Elapsed Time reflects the fact that the dominant time factor for the system is the email access application module, which was constant across strategy. Strategy Effects: As we hypothesized, the Mean Recognition Score for the system-initiative strategy (SI) was better than the Mean Recognition Score for the mixedinitiative strategy (MI) (df = 1, F = 28.1, p < .001). Mean Recognition Score for SI was .90, while the Mean Recognition Score for MI was .72. Furthermore, the performance ranges were different, with a minimum score for MI of .43, as compared to a minimum score for SI of .61. The number of ASR Rejections was also significantly greater for the MI strategy, with the system rejecting an average of 1.33 utterances per dialog, as compared with only .44 utterances per dialog for SI (df =1, F = 6.35, p< .02). However, despite the poorer ASR performance that we predicted, the average number of User Turns was significantly less for MI (df =1, F= 6.25, p < .02). The average number of User Turns for SI was 21.9 utterances, as compared with a mean of 15.33 for MI. We had hoped that users of the MI strategy would avail themselves of the context-specific help to learn the agent's valid vocabulary. While use of Help Requests was not significant by strategy, more Timeout Prompts were played for MI (df = 1, F = 62.4, p < .001). The mean number of timeouts per dialog for SI was .94, while the mean for MI was 4.2. Timeout Prompts suggest to the user what to say, and are triggered by occasions in which the user says nothing after a system utterance, perhaps because they don't know what they can say. For example, the most commonly played prompt for MI was You can access messages using values from the sender or the subject field. If you need to know a list of senders or subjects, say 'List senders', or 'List subjects'. If you want to exit the current folder, say 'I'm done here'. In terms of user satisfaction measures, there were no differences in the Task Ease measure as a function of strategy; users did not think it was easier to find relevant messages using the SI agent than the MI agent, even on the first day. Users' perceptions of whether Elvis is sluggish to respond (System Response) also did not vary as a function of strategy, probably because the response delays were due to the application module, which is identical for both strategies. However users did perceive differences in the Interaction Pace of the system. Users were more likely to perceive the pace of MI as being too slow (df =1, F = 14.01, p <.001). One possible source of this perception is that the SI strategy kept users busy more of the time, specifying small incremental pieces of information. This would be consistent with claims about graphical user interfaces [2]. Thus the average pace of interaction in the SI strategy would be faster, except for those interactions that finally accessed the specified email folder or message. In contrast, every MI interaction could result in accessing the email application module, so on average each interaction was slower paced, despite the fact that average task completion times were lower. There was a difference between the MI and the SI agent in users' perceptions of whether the agent understood them (ASR Performance) (df =1, F= 14.54, p< .001). This is consistent with the fact that the Mean Recognition Score was much lower for MI. Users also perceived the MI agent as more difficult for them to understand (TTS Performance) (df =1, F = 4.30, p < .05), possibly because the help and timeout messages had to be longer for the MI agent, in order to describe what type of input it could understand. There is a trend towards users having more difficulty knowing what to say to the MI agent (User Expertise) (df =1, F= 3.41, p< .07). Task Effects: Several factors were also significant as a function of task. As mentioned above, factors that are significant as a function of task are potential indicators of learning effects. Mean Recognition Score was significant as a function of task (df = 2, F = 4.2, p < .03). The Mean Recognition Score for MI for task 1 was .69, for task 2 was .68, and for task 3 was .80, showing a potential learning effect of adapting to the system's language limitations over successive task. Mean recognition score for SI did not improve over task, in fact showing evidence that task 2 was more difficult, with task 1 mean recognition at .95, task 2 at .82 and task 3 at .94. The average number of ASR Rejections per dialog was also significant as a function of task (df =2, F = 3.3, p< .05). ASR Rejections averaged 1.33 for MI for task 1, 2.0 for task 2, and .67 for task 3. For SI, ASR rejections averaged .16 for task 1, 1.0 for task 2, and .16 for task 3. The number of Help Requests per dialog was significant as a function of task (df = 2, F = 4.8, p< .03). Users usually asked for help on the first task but not afterwards. Users' perceptions of knowing what they could say (User Expertise) also improved over successive tasks for both versions of the system (df =2, F = 4.67, p < .02), showing the largest improvement for MI, as we hypothesized. For this question, 1 was mapped to almost never while 5 was mapped to almost always. The mean for SI was 3.67 for task 1, 2.83 for task 2 and 4.0 for task 3. The mean for MI was 2.33 for task 1, 3.00 for task 2, and 3.67 for task 3. Thus at the beginning of the experiment, most MI subjects thought that they rarely knew what to say, and by the end of the experiment, felt that they often knew what to say. Subject Effects: Several factors were also significant as a function of subject. Some subjects may have had an easier time using the system. There were significant differences in Mean Recognition Score (df = 10, F = 2.7, p < .02), and the frequency with which the system played Timeout Prompts as a function of subject (df = 10, F= 3.07, p < .01). We had thought that the use of Barge In might reflect learning, on the basis that as users acquired more expertise they would interrupt the system with responses to queries before the query was completed. However, there was no increase in the number of Barge Ins over task. There was a significant difference in the use of Barge In across subjects. Apparently some subjects felt more confident about interrupting Elvis. It is also clear that subjects' perceptions of the system varied. The user's perception that they knew what they could say (User Expertise) differed (df = 10, F = 3.00, p < .01), as well as whether Elvis was easy to understand (TTS Performance) (df = 10, F = 3.71, p< .005). Perceptions of whether Elvis was slow or sluggish to respond (System Response) differed (df = 10, F = 2.96, p < .02), as well as feelings about whether Interaction Pace was appropriate (df = 10, F = 4.84, p < .001). Finally, comparisons of Elvis to the touch tone interface to voice mail (Comparable Interface) varied across users (df = 8, F= 3.74, p < .01) . Given this experimental data, we draw on the PARADISE framework to estimate which factors are most significant in predicting Cumulative Satisfaction, and thus which factors might form the basis of a predictive performance function [9]. The overall structure of objectives in PARADISE that provides the basis for estimating a performance function is shown in Figure 1. Cumulative Satisfaction is the user satisfaction measure in the objectives structure in Figure 1. The efficiency measures for this experiment are User Turns, System Turns, and Elapsed Time. The qualitative measures are Barge Ins, Mean Recognition Score, Timeout Prompts, ASR Rejections and Help Requests. These qualitative measures reflect the style or the feel of the interaction. Multivariate linear regression using all the objective performance measures shows that the only significant contributors to Cumulative Satisfaction are User Turns and Mean Recognition Score. The results of a second regression with only these factors included shows that User Turns is significant at p < .03, and that Mean Recognition Score is sig- where N is a normalization function that guarantees that the magnitude of the coefficients is independent of the scales of the factors. Applying this performance function to our experimental data, independent of task, suggests that the SI strategy overall performs better. The mean performance over all subjects for SI is .214, while mean performance for MI is -0.213. However, as with the other measures, the performance of the MI strategy improves over each successive task, with performance at -0.27 for task 1, rising to 0.125 by task 3. Continuing the trend that we observe over the three trials, it seems likely that the performance of MI would outpace that of SI as users acquire more expertise. We discussed the results of an experiment comparing a mixed-initiative dialog agent with a system-initiative dialog agent, in the context of a personal agent for accessing email messages by phone. Our initial hypotheses were that the system-initiative strategy would be better for inexperienced users, but that as users gained experience with the system over successive tasks, the mixed-initiative strategy would be preferred. Our results demonstrated that user's satisfaction and ease of use with the MI strategy did improve over successive tasks. However, the overall performance function derived from the experimental data showed that the MI strategy did not surpass the SI strategy over the three tasks. Future experiments will give users more experience to test this hypothesis further.