Experiments in Intellectual Property

The other chapters in this volume approach the study of IP by observing the outcomes of behavior in the real world -whether quantitatively (natural experiments) or qualitatively (ethnographic studies). Often, however, the behaviors that are important simply are not observable by researchers. There are often no reliable sources of data on how a potential change to the structure of IP incentives would affect innovation, for example. Nor are there opportunities for measuring how people respond to IP rights by going out into the field. Much of the data about IP issues does not exist, and, if it does, it is often proprietary and inaccessible to researchers. Experimental and survey methods offer an opportunity for studying aspects of behavior that are otherwise invisible.

Moreover, experiments (but not surveys) enable researchers to study causal relationships in ways that observational studies typically cannot. The observational studies discussed in other chapters measure correlations between two phenomena-for example, between a change in procedural law and the filing of patent lawsuits. Well-identified variables and sophisticated statistical techniques allow researchers to infer that the increase in filings after the law changed was probably due to the change itself. But, with observational studies, it is impossible to rule out all other potential causes of the change (e.g., a change in markets that occurred at the same time). Because the environment is not fully controlled, other unobserved phenomena could have been the true causes of the difference. This is not the case with experiments, whose principal virtue is the control they give researchers. As Arlen and Talley (2008, xvi) explain, "Experimentalists study human decision making by placing subjects in tightly orchestrated environments and providing them with a set of experiences and/or choices whose presentation, features, and variations are all within the researchers' control." The ability to strictly control the experimental environment enables researchers to isolate the effects of individual features of that environment. For example, if a researcher is interested in studying people's judgments of musical similarity, she can artificially manipulate songs to test subjects' sensitivities to differences between versions (Lund, 2011).

This ability to control experimental environments enables researchers to study causal effects in ways that are unavailable in observational approaches. In a standard experiment, researchers design two conditions that are identical in every way except for a single variable of interest-the independent variable. Subjects are then randomly assigned to one of the two conditions, and aspects of their behavior are measured in other dependent variables. Because the two conditions are identical except for the independent variable, and because random assignment should minimize differences in the populations in each condition, any significant differences in the dependent variables are assumed to be caused by the independent variable.

For example, imagine that researchers want to test whether creators produce more and/or better work when they are given attribution for their efforts than when they are not (Fromer, 2012). The researchers can recruit subjects to participate in a creativity task and randomly assign them to either a condition in which they will receive attribution or one in which they will not receive attribution. Then, the researchers can measure how creative the subjects in each group were (ideally using measures that are "blind" to which condition the subjects were in). Because everything about the two conditions was identical, and because the subjects were randomly assigned, if the researchers find that subjects in the attribution condition were significantly more creative, they can infer that the prospect of attribution caused the increased creative output.

As the above example suggests, the enterprise of controlling experimental environments comes with important costs. Experiments involve abstracting away from "real" environments in ways that may affect the validity of the results. For example, in order for the creativity task to serve researchers' purposes, it may differ in important ways from the kinds of creativity that are most relevant to IP questions. In the final section of this chapter, we discuss some of these limitations and the challenges facing experimental researchers.

conducive to creativity than extrinsic motivation. These studies typically investigate overjustification: having subjects engage in a task that they already might like to do with the promise of extrinisic reward (Amabile, 1979). Subjects engage in a creative task, such as drawing, collage making, or puzzle solving. Some are told that they will receive a reward for performance, while others are not. In these situations, those subjects acting with reward expectation are judged to produce significantly less creative work than those acting without reward expectation (Conti et al., 1995;Hennessey, 1989;Deci, 1971). Other studies, however, suggest that extrinsic rewards do not always undermine creativity and can, in fact, enhance it. For one thing, studies by behavioral psychologists tend to find that providing reward-external motivation-increases subjects' creative performance with regard to whatever aspect the subject is being told will be judged (such as originality or fluency) (Glover and Gary, 1976).

These two strands of studies may be reconciled. The incentive studies instruct subjects specifically how (or with regard to which aspect) to perform creatively and the extrinsic reward then helps enhance creativity. In contrast, the over-justification studies do not so instruct and the extrinsic reward there then decreases creativity. Moreover, studies that control for both whether specific instructions to perform creatively are provided before engaging in a creative task and whether reward is provided confirm that guidance as to performance metrics seems to explain the difference in the effect of reward (Amabile, 1979;Eisenberger et al., 1998). These results suggest that when it is possible to specify how to perform creatively, it is worth doing so along with providing rewards. Ultimately, however, one cannot simply assume that the addition of an incentive to an already motivated person will yield more or better creative production.

Separate from the question of intrinsic and extrinsic motivation are issues about the magnitude and structure of creativity incentives. Just as classical economic theory predicts that adding external motivation to internal motivation increases total motivation, so too does it predict that higher magnitude incentives produce greater motivation and performance than lower magnitude incentives. Recently, Dan Ariely and others have studied the effects of particular magnitudes of performance-contingent rewards on task performance (Ariely et al., 2009). Ariely and his co-investigators studied subjects in both the United States and rural India performing a variety of tasks-based on creativity, cognition, memory, or motor skills-for which the magnitude of reward -i.e., the payment -varied from low to moderate to very high. For example, in one reported experiment, subjects would receive full payment if performance was very good, half payment if performance was merely good, and no payment if performance did not qualify as good. Across the various experiments, Ariely and the others found that subjects offered low to moderate levels of reward outperformed those offered the very high level of reward. They also found that the propensity to choke on a task due to increasing reward was frequently task-specific and not just based on individual characteristics. The authors suggest that these results are consistent with the idea that "beyond an optimal level of arousal for executing tasks, further increases in arousal can lead to a decrement in performance."

Relatedly, Katharina Eckhartz and others recently investigated the effect of incentives on creativity using three different incentive schemes: a flat fee, a linear payment, and a tournament (Eckhartz et al., 2015). They presented subjects with a set of letters and asked them to construct as many words as they could within five minutes. For each word, participants received a score that was more than proportionally increasing given the number of letters in the word. Contrary to nearly all of the other studies described above, they found that that the choice of incentive had no significant effect on performance for any of the three tasks; rather, they found that "performance depends almost entirely on individual characteristics of participants and can, on the aggregate level, hardly be influenced through incentives."

All in all, we face a murky picture of the relationship between incentives and creativity. A series of studies suggests that rewards-particularly higher ones-can undermine creativity, but other studies indicate that carefully designed rewards and instructions can instead enhance creativity.

In recent years, a handful of legal scholars have made reference to the social-science studies, described above, suggesting a nuanced relationship between incentives and creativity. (Cohen, 2011;Mandel, 2011;Johnson, 2012;Benkler, 2002;Tushnet, 2009) To date, however, there has been little attempt to determine whether the specific sorts of incentives provided by IP law affect behavior in the ways suggested by social science research. Recently, Buccafusco, Burns, Fromer and Sprigman have studied in an experimental setting how the structure of IP-style incentives affects creativity. (Buccafusco, Burns, Fromer and Sprigman, 2014).

The Buccafusco et al. paper begins with the observation that both copyright and patent law aim to incentivize creativity, but those incentives are structured quite differently from one another. Patent law imposes a high bar of novelty and non-obviousness; under these standards only inventions that are truly new are entitled to protection. In contrast, copyright law employs a standard of minimal "originality," which requires only that a creative work not be copied from another and have some spark of creativity, however slight. Is there a difference between exclusive rights structured with a high-threshold (patent) or a low-threshold (copyright) in terms of the amount of creativity produced (all other elements of the right held equal)?

Buccafusco et al. employ a set of creativity tasks to investigate this question. The first is a convergent thinking task involving a mathematical combinatorial optimization problem inspired by the famous 1990s "Oregon Trail" video game. The problem is designed to be very difficult, and perhaps impossible, to calculate a solution in the time allotted (90 seconds), so subjects must resort to some sort of heuristic to approach the problem. The creativity is in the choice of heuristic; the goal was to model aspects of information aggregation and convergent thinking that play significant roles in intellectual discovery and invention Subjects were recruited online and randomly assigned to one of five experimental conditions. In all conditions, subjects were paid a fee to participate. The conditions differed in the manner in which their performance in the game affected their chance of winning a bonus prize. In the first "no incentive" condition, were told that their participation in the experiment would make them eligible for a prize, to be awarded via a (random) lottery. Task performance in this condition was unrelated to payout, whereas in all other conditions payout increased along with the quality of task performance. In the second "copyright" condition, subjects were paid the same flat participation fee, and all participants who successfully completed the task would also be entered into a lottery to win a prize, with the subject's chance of winning the prize improving proportionally to his or her performance on the creativity task. The third, fourth and fifth condition were structured to imitate patent law's higher creativity thresholds for protection. Subjects in the three conditions that modeled patent thresholds were told that only a certain percentage of contestants would be eligible to win the lottery prize -the top 50% in the "patent low" condition, the top 25% in "patent mid", and the top 5% in "patent high".

The experiment, which was run with almost 800 subjects recruited online, showed that patent-style incentives -i.e., those made available only to those who had surmounted a higher creativity threshold -caused a modest albeit statistically significant positive effect on overall performance on the creativity task. On the other hand, copyright-style incentives provided no measurable increase in overall performance on the creativity task versus the no-incentive (control) condition.

Buccafusco et al. employed the same protocol with two additional creativity tasks, one measuring linguistic creativity and the other figural or visual creativity. The linguistic creativity task involved rapidly generating a list of words that bear an indirect relationship with some target word, usually in the form of a pun. In Buccafusco et al.'s version of the task, subjects were asked to come up with a list of "keys," and they were rewarded for answers that were judged to be creative. For example, "house key" would not be considered a creative answer, while "Keyshawn Johnson," "monkey," "keynote speech," or "John Maynard Keynes" would be considered creative. 3 As in the first experiment, the sample for the Keys creativity task included approximately 800 subjects recruited online. And the results were broadly consistent with the first experiment -the subjects in the patent conditions, on the whole, performed better on the creativity task. In contrast, performance of subjects in the copyright condition was statistically indistinguishable from those in the no-incentives (control) condition.

A third experiment involved a figural creativity task, where subjects were presented with a series of abstract drawings and asked to state what the figures could be. As in the Keys task, only answers judged to be creative scored points (using a judging methodology similar to that employed in the Keys task). In this experiment, neither the patent nor the copyright conditions produced performance measurably better than the no-incentive (control) group.

These results suggest that incentives that are structured to include a relatively high creativity threshold -i.e., incentives structured like patents, which are (at least in theory) made available only to inventions that represent a substantial departure from the prior art -may have a positive effect on creativity. The incentives structured with a low creativity requirement -i.e., incentives 3 In order to assess the creativity of responses, the subjects' responses were standardized and then rated by online judges. First, every entry was standardized by two independent raters and ties were broken by a third, so that spelling, plurals, and word forms were consistent. Next, raters were recruited online, and then were instructed to evaluate a sub-set of the unique entries, using their own intuitions to produce the ratings, responding "Creative," "Standard," or "I'm not familiar with this." Any answer receiving more than 50% of total votes as "Creative" was tallied as creative.

structured like copyright, which is made available to any artistic or literary work that manifests even the slightest creative spark -had no measurable effect on performance in any of the creativity tasks employed. In addition, there was no evidence that the higher threshold in the patent-style conditions undermined creative performance, in contrast to studies suggesting that high-magnitude incentives cause subjects to "choke" (Ariely et al., 2009). While far from definitive, these results suggest that policymakers should consider whether copyright's effectiveness as an incentive system might be improved by raising the creativity threshold that authors are required to surmount before gaining rights.

A recent IP experiment, conducted by Christoph Engel and Michael Kurschilgen, examines whether creators' perceptions of fairness influence the effectiveness of IP incentives (Engel and Kurschilgen, 2011). Engel and Kurschilgen are interested in an unusual provision of the German copyright law, the so-called "bestseller paragraph," under which the transferor (whether by license or sale) of a creative work has a legally enforceable right to an "appropriate" bonus in the event that the work turns out to be very valuable. The Engel and Kurschilgen experiment tests whether the German bonus provision can be justified on either efficiency or fairness grounds (e.g. that it corrects market power or other distributional issues). They designed two experimental conditions. The first was a baseline condition where subjects were randomly assigned to act either as a buyer or a seller. The subjects were each given an initial endowment of 500 "Talers," or notional monetary units. The subjects were told that they would be transacting over an unnamed commodity. The commodity did not have a certain value, but merely a probabilistic one -there was a 25% chance that it was worth 1700 Talers, but a 75% chance that it was worth only 100 Talers. With this information in hand, the subjects participated in 8 rounds of a four-stage game.

In the first stage of the experiment, buyers were given the opportunity to make an offer to purchase the commodity, based only on their knowledge of the probabilities attending the commodity's value. In the second stage, the seller is given the choice of accepting or rejecting the buyer's offer. If the seller rejects, then both buyer and seller keep their initial 500 Talers. If the seller accepts, then the buyer's offer price is transferred to the seller. In the third stage, a random device determines the value of the commodity -either 100 Talers (75%) or 1700 Talers (25%). Finally, in the fourth stage, each of the parties learn the value of the commodity, and each is given a chance to "punish" by reducing the other party's earnings.

The second experimental condition built in an ex post adjustment procedure. The experimenters did so by introducing a third player -the "umpire." The umpire was paid a fixed fee to judge the fairness of transactions, and to adjust unfair transactions ex post.

The Engel and Kirschilgen experiment produces some intriguing results. The price at which deals are struck declines by a statistically significant amount in the second condition (104 Talers) versus the baseline (129 Talers). The number of deals stayed about the same. Importantly, bid and ask prices converged in the treatment condition (seller's ask price declined more than buyers' bids), suggesting that the presence of the ex post "fairness" mechanism generated a more efficient market by making low offer prices more acceptable to sellers.

The experiment tells us not only about buyers and sellers, but about umpires -i.e., judges. A large portion of the umpires, who had the same information as the buyers and sellers, appear to have been motivated primarily by ex-post egalitarianism -i.e., they split the profits more or less evenly between buyers and sellers. Surprisingly, they did this even though from an ex ante perspective such a result clearly favors sellers (because only buyers, ex ante, faced the prospect of a real loss). Finally, the experiment shows that ex post discontent is reduced when a mechanism exists to remedy "unfair" transactions. Angry buyers used punishment more frequently in the baseline condition (21% of the time) than in the treatment condition (12%), and they spent more on punishment (10 Taler per round vs. 7 in the treatment condition). Sellers made very little use of punishment in either condition. So the result of the treatment was to reduce buyers' ex post perceptions of unfairness, while not affecting sellers, who did not perceive unfairness in the first place. This is deeply counterintuitive. It is the sellers who are presumed, in the fairness justification for the German bestseller provision and the analogous American termination provisions, to be the victims of unfairness when transactions prove to be much richer than initially anticipated. But in this experiment, it's the buyers, not the sellers, who manifest significant feelings of unfairness, and these feelings are reduced by ex post mechanisms designed to transfer value to sellers when deals prove particularly rich. The best explanation seems to be that buyers, anticipating all of this, lowered their offers in the treatment condition. Because they had accounted for the possibility of value transfer in advance, they discounted what they were willing to pay ex ante. And this, in the buyers' perspective, was fair.

Once people create new inventions and works, they can obtain IP rights in them. It is the rare case, however, where the creator is also the best-situated person to produce and distribute the creation. Copyright and patents have to be licensed and sold in order to wind up in the hands of those who can best exploit them. IP law tends to assume that markets for exploitation function smoothly, and it counts on creators to properly price access to their creations. Yet these assumptions are subject to empirical verification and should not be accepted without it.

1. IP Valuation Buccafusco andSprigman (2010, 2011) have conducted a series of experiments designed to understand how creators value works and inventions. In particular, they studied the extent to which endowment effects attend transactions in intellectual property. The label "endowment effect" is applied to an unexpected gap between prices that buyers are willing to pay (WTP) and that sellers are willingness to accept (WTA) for a good. A rational choice economic framework would posit that, in general, transactions over property whose value is well understood would feature WTP and WTA that are, on average, close to one another. For example, if both buyer and seller know that the value of a coffee mug is worth approximately $5.00, a rational buyer should be willing to take anything over that amount, and a rational seller should be willing to pay anything up to that amount. Personal valuations of the property will vary, but, on average, WTP should meet or exceed WTA about half the time, and so we should see transactions at that rate. In the endowment effects experiments we do not, for the most part, see the behavior that the rational choice model predicts. Instead, sellers demand higher compensation -often significantly higher -with the result that WTP and WTA are driven apart, and the number of instances in which WTP meets or exceeds WTA, and therefore the number of completed transactions, fall short (often dramatically) of the rational choice prediction (see Buccafusco and Sprigman 2010 for a collection of cites to endowment effects studies).

Buccafusco and Sprigman's experiments build on more than a quarter century of endowment effects literature, but they depart from that literature in two ways. First, the "goods" that are transferred in the IP experiments are not simply property (e.g., mugs, candy bars, basketball playoff tickets) with which subjects have been endowed. Rather, they are property that subjects have made (in particular, poems and paintings). Second, and just as importantly, the property transacted over in the Buccafusco and Sprigman experiments is non-rivalrous, in contrast to the rivalrous property used in all of the previous endowment effects studies. That is to say, the sale of the IP interest by sellers does not limit their ability to continue to enjoy possession of the good itself.

Interestingly, these two differences from earlier endowment effects studies create hypotheses that point in different directions. Given subjects' likely greater emotional attachment, in general, to property, like a poem or a painting, that they have created, versus property (like a mug or a candy bar) that they merely own, it may be that transactions involving creative property may manifest larger endowment effects than seen in experiments involving property with which subjects were merely endowed. On the other hand, given that transactions involving non-rival property do not involve the loss of possession of that property, it is possible that in the absence of anticipated loss we would see no endowment effects.

Buccafusco and Sprigman conducted two initial studies, the first using poems (Buccafusco and Sprigman, 2010) and the second, paintings (Buccafusco and Sprigman, 2011), designed to understand (a) whether endowment effects attend IP transactions, (b) what causes those effects, and (c) whether they differ in intensity for creators as opposed to mere owners of IP.

In the first experiment, Buccafusco and Sprigman solicited subjects from the Charlottesville, Virginia area via fliers, e-mails, and online advertisements. The experiment included three randomly assigned groups of subjects: "Authors," "Owners," and "Bidders." The Authors were instructed to write three-line haiku poems for entry into a $50 contest. Then, they were then given the opportunity to sell their chances to win the prize to another subject-a Bidder-by indicating the least amount of money that they would be willing to accept to give up their chances. The Bidders were each shown a poem and asked to indicate how much money they would be willing to pay to obtain the poem's chance to win the $50 prize. Finally, the group of Owners were told that they owned one of the poems' chances to win the prize and were asked to indicate the least amount of money that they would be willing to accept to sell it to a Bidder. The subjects were reminded that they were exchanging only the chance to win the prize and not the poem itself.

Buccafusco and Sprigman found a substantial valuation asymmetry between creators and purchasers of IP, with creators valuing their work almost twice as highly, on average, as buyers did. This result is the first demonstration that endowment effects attend intellectual property transactions, which, given the non-rival nature of the property, was in some doubt. The gap between Bidders' willingness to pay and Authors' willingness to accept was substantial, and resistant to attempts to diminish it. Importantly, the authors were unable to diminish the asymmetry either by using transaction intermediaries (the "Owners", who were not themselves authors) or by providing additional market information (allowing buyers to view all the poems, rather than only the poem on which they were assigned to bid).

Buccafusco and Sprigman were also able to isolate the cause of the valuation asymmetry their experiment measured. By asking subjects to estimate their particular poem's chance of winning the prize, the authors measured a significant difference in optimism between Authors and Bidders, with Authors' greater optimism contributing to their higher valuation. But differential optimism wasn't the only mechanism driving the valuation assymetries. Buccafusco and Sprigman find also that the asymmetry arises in part from "regret aversion" -i.e., the anticipation of regret following alienation, a regret for which the subject demands to be compensated in advance. Buccafusco and Sprigman found that Authors anticipated regret if they transferred the rights to the winning poem, but that Buyers valuations were consistent with the expected returns, adjusted by their anticipated likelihood of prevailing, and so did not contain any adjustment for anticipated regret.

There was one hypothesis that failed to hold -Authors did not exhibit significantly greater valuation than Owners, despite the Authors' presumably higher attachment to their work, which, Buccafusco and Sprigman predicted, would result in Authors reporting higher minimum willingness to accept values. Buccafusco and Sprigman predicted that the failure to find a valuation differential between Authors and Owners in the first experiment might be due to the small size of the creative effort involved -a three-line poem -in the particular experimental setting

In the second experiment, Buccafusco and Sprigman set out to test whether an experiment based on a more substantial creative act would produce a valuation difference between Authors and Owners. In their second experiment (2011), Buccafusco and Sprigman used student "Painters" who had invested substantially in their works and who were primarily internally motivated to create them. Buccafusco and Sprigman solicited undergraduate and graduate painting students from the School of the Art Institute of Chicago (SAIC) as subjects. The subjects were invited to choose a medium-sized painting for entry into a contest for a $100 prize. The experimenters received twenty submissions and held two contests of ten paintings each. Each of the Painters was paid $15 for participating. The contest was hosted in an exhibition space at SAIC, and a faculty member at SAIC judged the paintings.

When the Painters arrived at the exhibition space, they were given sheets of paper describing the contest. They were told that they would be competing with the nine other Painters in their group for a $100 prize based on the quality of the paintings as judged by an expert. They were next told that they would be matched with one of ten additional subjects, known as "Buyers," who would make them a cash offer for their chance to win the prize. The Painters were told to indicate the least amount of money that they would be willing to accept to sell their chances to win. If the Buyer's amount was equal to or higher than the amount indicated by the Painter, the Buyer would pay the Painter the amount of the Buyer's offer and receive the chance to win the $100 prize. If the Buyer's offer did not meet the Painter's WTA price, the Painter would retain her chance to win the prize. Neither party would know the other party's offer before responding. The Painters were reminded in bold that they were not transferring the paintings themselves or any rights in them other than the chance at the prize awarded to the winning painting.

In addition to the Painters, Buccafusco and Sprigman recruited forty additional subjects who were randomly assigned to the role of either Buyer or Owner. Each Buyer was told that she would be matched with one of ten Painters who had entered paintings into a $100 contest. The Buyer would be able to make an offer to purchase the Painter's chance to win the prize. She was told to indicate the most she would be willing to pay to buy the Painter's chance to win.

The Owners were placed in a similar position to the Painters with respect to the contest. They were told that ten paintings had been entered into a contest for $100 and that each had been assigned to be the owner of one of the paintings. If "her" painting were chosen as the winner, she would receive $100. The Owners were then told that a Buyer would make them a cash offer for their chance to win the prize, and that if the Buyer's offer exceeded the least amount that they would be willing to accept, they would receive the cash offer in exchange for the chance to win.

This time, Buccafusco and Sprigman found not merely a difference between Painters' and Buyers' valuations, but also a significant difference between the valuations of Painters and those of mere Owners. The Painters' mean willingness to accept was $74.53, while the Buyers' mean willingness to pay was only $17.88, thus illustrating a substantial endowment effect. The Owners' willingness to accept fell in the middle, at $40.67. The differences were statistically significant across all roles, and they again appeared to be caused primarily by differences in optimism, with Authors being far the most optimistic regarding their painting's chance of winning the prize, followed by Owners, and finally Bidders. Most importantly, the difference in valuation between Painters and mere Owners suggests that creators manifest more substantial valuation assymetries versus ordinary owners of property -a phenomenon that Buccafusco and Sprigman refer to as the creativity effect. Buccafusco and Sprigman's (2011) findings suggest that private transactions in creative goods may face significant transaction costs arising from cognitive biases. These biases in turn drive the price that creators and owners of IP are likely to demand considerably higher than buyers will, on average, be willing to pay. This discovery does not mean, of course, that transactions in IP will not take place-we see such transactions happening every day. The research suggests, however, both that IP transactions may occur at a frequency that is significantly suboptimal and that the baleful effect of cognitive and affective biases is likely to be more serious for transactions in works of relatively low commercial value or for which no well-established custom or pattern helps to inform valuation.

2. Valuing Attribution It has often been suggested that the prospect of attribution is an important force motivating the production of creative work (Fromer 2012), and indeed we see a variety of creative fieldsfor example, open source software, or the production of Wikipedia entries or the range of materials made available under some form of the Creative Commons license -in which participants work with little or no prospect of monetary compensation but expect and indeed enforce norms encouraging attribution. Sprigman, Buccafusco and Burns (2013) set out to test experimentally whether and to what extent creators value attribution, and, in particular, whether creators view attribution as a complete or partial replacement for monetary compensation such that they are willing to sacrifice pay in order to receive it (Sprigman et al 2013). The Sprigman et al. attribution experiments involved digital photographs. Subjects were recruited online and invited to upload a digital photograph of a "nature" subject in order to participate in a contest with 99 other entrants for a chance to win a $1000 prize.

Subjects were randomly assigned to one of three conditions. In the first, Contest, condition, subjects were told that their photograph would be viewed by a potential buyer before any judging would take place. The buyer would make a cash offer which, if accepted by the photographer, would result in the transfer of the opportunity to win the $1000 prize from the photographer to the buyer. The offer was not for the photograph itself, but only for the right to be paid the prize if the photograph was judged the winner. Once informed of the rules, the photographers' WTA was elicited, that is, they were each asked to specify the lowest amount they would accept to sell their photograph's contest rights.

The second, Publication, condition was the same as Contest, except that the photographers were offered the opportunity to have their photo published, uncredited, on a major website if they won the contest. But the possibility of publication would arise only if the photograph had been sold to the buyer and it won the contest. As in the Contest condition, subject were asked to specify the minimum amount of money they would be willing to accept to transfer to a willing buyer their photograph's chance of winning the prize.

The third, Attribution, condition was the same as "Publication", but if the conditions specified above were met, the photograph would be published along with the photographer's name. Again, subjects were asked to specify their minimum WTA.

Sprigman et al. structured the conditions this way in order to determine whether the photographer subjects valued publication and attribution, and whether subjects' attraction to these prospects would reduce their WTA relative to a situation in which publication and attribution were not available. If they attached a significant value to the prospect of publication, then subjects in the Publication condition should report lower WTA than those in the Contest condition. And if they attached a significant value to the prospect of attribution, subjects in the Attribution condition should report lower WTA than in both the Publication and Contest conditions.

The experiment was run with 200 subjects recruited online. The data suggests that the subjects do, in fact, value attribution. The subjects in the Attribution condition reported a significantly lower figure for willingness to accept versus subjects in the Contest or Publication conditions -that is, the two conditions where there was no prospect of attribution. The investigators re-ran the study with a sample of 77 professional and advanced amateur photographers, and found similar results, with subjects in the Attribution condition reporting a significantly lower willingness to accept. The upshot is that the prospect of attribution caused both casual snapshooters and professional and advanced amateur photographers to moderate their demands for monetary compensation.

Sprigman et al. performed a follow-up experiment designed to determine whether subjects' valuation of attribution was stable against changes in the default rule governing the availability of attribution. This inquiry is based in a real-world policy difference. There is no default right to attribution in U.S. copyright law -copyright owners must negotiate for attribution, using their copyright rights as leverage. In many others countries the situation is reversed -copyright owners start with a default rule to attribution for many uses of their work, and publishers who prefer to use work without attribution must negotiate out of the attribution default. Sprigman et al. hypothesized that subjects' valuation of attribution would be, to some extent, endogenous on the default rule. That is, they suspected that creators' preferences for attribution would be substantially affected by whether attribution was provided as a default right or not. Research in other areas had suggested that default rules often produce this sort of stickiness (Korokbin, 1997;Johnson and Goldstein, 2003). As before, subjects were recruited online, and asked to upload a photograph of a nature subject. Subjects were then randomly sorted into two conditions.

In the first Default Attribution condition, subjects were told that their photograph, along with those of four other participants, would be shown to a graphic designer who was participating in a design contest with a prize of $200. In order to participate in the design contest, the designer needed to purchase the rights to use one of the five photographs as the basis for his design. In order to use one of the images, the designer would have to strike a deal with one of the photographers. Subjects in the first condition were told that if they made a deal with the designer and the designer's creation won the contest, the finished design would appear on a major website with both the designer's and the photographer's names. The photographer would receive the agreed upon price for the photograph, but she would not be eligible to win the $200 contest prize. The subjects were then asked to indicate the least amount of money they would be willing to accept to allow the designer to use their photograph (WTA:Att).

After entering WTA:Att, the subjects were then directed to a new screen. Here they were told, "Some designers have indicated that they do not want to share credit for the design with the photographer." Subjects were then reminded of their WTA:Att. Then they were told, "Now you will be asked to indicate the least amount of money you would be willing to accept to allow the designer to use your photograph without credit. If you do not care about credit, you can put the same price." They were then asked to specify the least amount of money they would be willing to accept to allow the designer to use the photograph without attribution (WTA:NoAtt). Photographers in this Default Attribution condition were, in effect, selling their default right to attribution. Based on findings from the previous studies in the paper, the experimenters expected that subjects would demand more money for an uncredited use of their image than a credited use (WTA:NoAtt > WTA:Att).

In the second "No Default Attribution" condition, the statements regarding credit were reversed. On the first screen, subjects were told that if they licensed the photograph and the design won the contest, the final design would appear on a major website with only the designer's name. Subjects were then asked for their willingness to accept this uncredited use (WTA:NoAtt). On the next screen they were told, "Some designers have indicated that they are willing to share credit for the design with the photographer." The subjects were reminded of their WTA:NoAtt and were asked to indicate their WTA with credit (WTA:Att).

In the No Default Attribution condition, subjects were, in effect, buying attribution. Again, the previous studies indicated that since attribution has economic value for creators, they should be willing to pay something to receive it. They could manifest their willingness to pay for attribution by reducing the amount they were initially willing to accept for uncredited use of the photo (WTA:NoAtt > WTA:Att).

The results suggest that subjects' valuation of attribution is indeed strongly endogenous on the default rule. In the Default Attribution condition, subjects' initial WTA:Att for credited use of their photos averaged $40.17, while they demanded $54.94 for uncredited use of their photos (WTA:NoAtt). This difference is statistically significant. Subjects given an opportunity to "sell" their attribution right demanded almost $15 on average to do so. This is a strong confirmation of the previous experiments reported in the paper.

In the No Default Attribution condition, subjects also altered their willingness to accept. Their initial WTA:NoAtt for uncredited use averaged $42.36, yet they were willing to accept only $38.75 in order to receive credit (WTA:Att). Again, the difference is statistically significant. Subjects who were given an opportunity to "buy" attribution were willing to give up $3.61 to obtain it.

Comparing the two conditions, it is clear that subjects' valuation of attribution is highly endogenous on the legal rule. When the subjects were initially endowed with a right to attribution, they valued it four times higher than they did when they were not so endowed and had to purchase it. The Sprigman et al. findings suggest that an endowment effect attaches to the right of attribution when it is structured as a default right which creators must contemplate trading away. Legal rights that are structured as defaults are in a sense "owned", just like any other form of property, and the owners of those default legal rights will tend to be resistant to parting with them. The authors predict that the magnitude of the endowment effects attaching to attribution rights will be higher for creators than for others, and that the law will minimize transaction costs by assigning rights to parties with lower endowment biases.

Electronic copy available at: https://ssrn.com/abstract=2838317

Innovation is not a one-shot activity. In all fields of creativity, new efforts build on old ones. But the nature of sequential innovation is affected by existing IP regimes. IP rights shape the manner and timing of innovation by affecting the decisions that downstream creators make about how and when to build off of existing ideas. Most importantly, IP rights affect downstream creators' judgments of whether to borrow from existing creations by licensing IP rights or to invent around those rights to develop different, non-infringing solutions (Bechtold, Buccafusco & Sprigman 2016).

A number of experiments have focused on whether an IP regime or an "open source" or "IP-free" regime does a better job of promoting innovation over time. We review some of those studies here. Torrance and Tomlinson (2009) were the first to study sequential innovation using experiments. They created a simulation of an innovation environment involving a non-creative search task, in which subjects were given options to "patent," make, license, and "open source" their creations. Subjects who patented their creations could also enforce their patents against other players who infringed them. The simulation was played in rounds that allowed for cumulative innovation and development of existing solutions. Subjects played the game in groups of five in trials that lasted thirty minutes.

Three different conditions were used: Pure Patent, Patent/Open Source, and Pure Commons. In the middle condition, subjects could choose whether to patent their creations or not. In the last condition, all creations were open for others to develop further. The researchers find that subjects in the Pure Commons condition produced significantly more innovations and generated significantly more value than did those in the conditions where patenting was allowed.

Julia Brueggemann and co-authors (2015) have studied also experimentally the effects of IP rights on sequential innovation. Like Torrance and Tomlinson (2009), they ask whether the existence of IP rights enhances or impedes innovation and overall welfare. To do so, they employ a creativity task based on the board game Scrabble. Subjects playing in groups of four can choose on every turn whether to create a new root word using the letters available to them or to extend existing words into longer (and more valuable) words. Subjects (n = 214) were randomly assigned to one of two conditions: noIP or IP. 4 In the noIP condition, subjects could freely build upon any words that had been created. In the IP condition, once a subject created a word, she could insist on a license fee from 0-100% of the value of new words built upon hers. Others who extended her word would have to pay the fee, which was visible to all players.

Although one might have predicted that the possibility of higher returns from licensing fees in the IP condition would have produced greater innovation, the researchers instead find that innovation and welfare are impeded in the IP condition relative to the noIP condition. In the IP condition, subjects produced less valuable words than in the noIP condition. The prospect of having to pay licensing fees in the IP condition discouraged people from borrowing others' words. Instead, they either focused on creating more low-value root words or building on the root words that they had produced. Thus, under the conditions studied in the experiment, the introduction of IP rights harms innovation and social welfare shifting creator behavior away from the most valuable kinds of creativity. These findings are consistent with those of Torrance and Tomlinson (2009).

In a recent experiment, Boudreau and Lakhani (2013) set up a tournament on the TopCoder computer programming platform that involved solving a complicated algorithmic task over the course of two weeks. Subjects played for cash prizes (up to $500) and reputational enhancement within the TopCoder community. Subjects were randomly assigned into different conditions based on varying disclosure regimes. The three disclosure conditions were:

• Intermediate Disclosure -Subjects could submit solutions to the contest, and, when they did, the solutions and their scores were immediately available for other subjects in the same condition to view and copy.

• No Disclosure -Subjects' solutions to the contest were not disclosed to other subjects until the end of the two-week contest.

• Mixed -During the first week of the contest, submissions were concealed from other subjects, but, during the last week of the contest, they were open and free to copy. For the Intermediate and Mixed conditions, subjects were asked to provide attribution to other subjects' whose code they copied.

The Intermediate condition allowed free-riding by others which could diminish incentives to participate. The data supported this hypothesis: Fewer people submitted answers in the Intermediate condition than in the No Disclosure condition, and the average number of submissions and the number of self-reported hours worked were also lower by significant margins. The Mixed condition generally produced data that were between the other two conditions. Despite the lower participation rates, scores in the Intermediate condition were better than those in the other conditions because subjects could borrow from high-performing solutions.

More importantly, the data also disclosed differences in how subjects solved the problem. Subjects in the Intermediate condition tried fewer technical approaches and seemed to experiment less than did those in the No Disclosure condition. Once significant improvements were disclosed, other subjects in the Intermediate condition tended to borrow the successful code leading to path dependence. In the No Disclosure condition, by contrast, subjects tried a greater variety of approaches to solving the problem.

Boudreau and Lakhani's approach is valuable for a number of reasons. It involves actual creators in an incentivized environment. It measures tradeoffs between participation losses from free-riding against more rapid improvement in solution quality. And it offers researchers competing visions of innovation success in terms of the value of different solutions to a problem.

Although there have been a number of studies contrasting IP versus no-IP regimes for sequential innovation, few researchers have focused on the nature of sequential innovation as such. Bechtold, Buccafusco, and Sprigman (2016) have studied how creators think about cumulative innovation. In particular, they studied how creators decide whether to borrow from existing ideas and pay a license fee or whether to innovate around those ideas. In theory, creators' decisions should be based on their assessments of the relative costs and benefits of these two options. The researchers study creators' responses to two factors that should influence this cost-benefit calculus: how costly the license is and how easy it is to innovate around the existing IP. Comparing the costs and benefits of innovating versus borrowing is complex, however, and the authors hypothesized that creators may use "innovation heuristics" -mental shortcuts about innovation decisions -to determine what to do.

The experiments employ two separate creativity tasks. In one, subjects play a version of a knapsack task that requires them to select items to add to a wagon in order to maximize the wagon's value without exceeding its weight limit. They are told that another subject has begun playing the game, and that if they innovate around the other subject's solution, they can obtain more bonus points. The second task involves a Scrabble game in which subjects must create a list of six words using a series of letters provided. Again, another person has submitted a word list, and subjects gain more points by innovating around the other subject's list (defined as using two or fewer of his words). In both tasks, although choosing to innovate results in a bonus, it also constricts the subject's options. Because innovating can be a good or a bad decision, subjects need to weigh the costs and benefits of their choices.

In the first set of experiments, the researchers test whether subjects are sensitive to the cost of borrowing the other subject's solution. Subjects are randomly assigned to conditions that differ in terms of the value of the bonus offered for innovating. In some conditions, the bonus should be too small to induce rational subjects into restricting their options to play the games. In other conditions, the bonus should be far greater than is necessary to induce innovation. Despite these differences, subjects tended to innovate at almost identical rates (about 70%) in each condition in both the wagon and the scrabble games. This leads to welfare losses in both the low and high bonus conditions, because many subjects are choosing to innovate when they should be borrowing (and vice versa). The researchers find that although subjects are insensitive to objective changes in the innovation environment, their decisions are largely driven by their own subjective assessments of how easy it is to innovate.

In the second set of experiments, Bechtold, Buccafusco, and Sprigman (2016) test whether subjects are sensitive to how easy it is to innovate around the existing solutions. The better the given solution, the more likely it is that people should borrow rather than innovate. In separate studies using each of the creativity tasks, subjects were randomly assigned to conditions in which the strength of the given solution was 60%, 80%, or 100% of the best solution. They were offered a single innovation bonus that remained constant. In the Wagon version of this study, unlike in the first set of studies, subjects demonstrated some sensitivity to how good the given solution was. Subjects were more likely to borrow rather than innovate as the solution strength increased (80.8%, 69.3%, and 60.2%, respectively). This was particularly surprising because they were not told how strong the given solution was.

Yet the truly striking results appeared in the Scrabble version of the study. Between the 60% condition and the 80%, subjects chose to innovate less in the latter (46.8% vs. 40.9%), as standard theory would predict given the smaller innovation space. But subjects in the 100% condition innovated at a substantially higher rate (85.7%) than did those in the other conditions, even though innovating was an overwhelming bad option. The authors suggest that subjects in the 100% condition were influenced by an "innovation heuristic." Instead of comparing the ease of finding an innovating solution with the value of that solution relative to the value of borrowing, subjects simply asked themselves how easy it was to generate a list of non-borrowing words. Here, the given solution in the 100% condition had very challenging words, so subjects found it easy to create a non-borrowing list of words. They ignored, however, the low value associated with those words. Rather than assess the full problem, subjects seem to have focused only on a portion of it.

The results of these studies suggest that creators' responses to sequential innovation decisions are not necessarily as rational as IP law predicts. Creators are often insensitive to key aspects of the innovation environment, and their decisions may be motivated more by internal preferences from innovating versus borrowing than by objective factors. Moreover, features of the innovation environment may affect the success of creators' decisions depending on whether heuristics do a good job of proxying for the best decision. Finally, it is worth pointing out that there is also some overlap between these data and those of Brueggemann et al. (2015) suggesting that creators tend to avoid borrowing from others and paying licensing fees even when doing so is economically rational.

IP incentives are not the only ones that might matter for innovation. Some scholars have suggested that innovation prizes might do a better job of stimulating innovation than IP rewards. A working paper by Julia Brueggemann and Lukas Meub (2015) attempts to measure how the availability of prizes affects creators' innovation behavior. They use the same Scrabble-style game as in Brueggemann et al. (2015), in which competing players create words and build off of root words to create new ones. Those who create root words can insist upon royalties for subsequent words that build on their roots. This gives creators an IP right in the words that they create. Groups of four players interact over 25 rounds (total n = 144).

Groups play in one of three different conditions: 1) control (players are paid according to points scored from words and royalties); 2) bonus (scored like control but the player with the highest value reward receives an additional payment); and 3) ranking (where the top player is paid twice as much as the other three players). The first condition is modeled on a standard IP regime of rights licensing. The latter two conditions are intended to simulate innovation prizes in addition to IP rights. The researchers measure a variety of different dependent variables. The average royalty rate per condition is taken to represent the degree of cooperativeness that players show. They also study how much building and sequential innovation occurs, as well as the overall innovativeness of each group.

Their findings indicate that royalty rates are significantly higher in the prize conditions than in the control condition, indicating that subjects are less cooperative and more competitive when there is an additional bonus available. There was no difference in royalty rates between the two prize conditions. Despite the higher royalty rates in the prize conditions, however, there were no differences in degree of borrowing between the three conditions. In each condition, subjects built on existing words about two-thirds of the time, meaning that in the prize conditions, borrowing subjects were willing to pay more to compete. Moreover, the results showed no significant differences in total innovation activity between groups in different conditions. Groups produced approximately the same scores regardless of whether prizes were available.

The decreased cooperativeness without increased innovation suggests that prizes combined with IP rights might not lead to welfare improvements. Although the subjects were more competitive, this led to less sharing and, thus, greater restrictions on sequential innovation between the parties. The researchers suggest that a better system might be one in which creators are forced to choose between prizes and IP rights.

Over the last decade, scholars have argued that a complete understanding of the economics of IP requires appreciating the roles that social norms and morals play in creative communities and public discourse (Raustiala and Sprigman, this volume;Buccafusco and Fagundes, 2016). Increasingly, scholars have turned to experimental and survey evidence to study how social norms operate and moral beliefs are formed.

A number of recent studies have shown that people's intuitions about creativity, ownership, and copying are formed during infancy. For example, children apply the same principles of ownership to ideas that they apply to physical property. A study by Shaw, Li, and Olson (2012) showed that six-year-old children believe that ideas such as songs, jokes, and solutions to math problems can be owned. These children also seem to apply ownership principles about first possession, denial of permission, and non-transfer of ownership through theft to creative ideas. It seems likely that children map their intuitions about physical property (which develop earlier) onto intellectual property as they begin to learn that ideas can be valuable. Thus, by the time we enter grade school (at least in the U.S.), we already understand that ideas can be subject to individual ownership and propertization.

By a very early age, children seem to understand relationships between creating ideas, ownership, and value. Studies show that very young children will assign ownership of objects to people who invested creative effort in them rather than the initial possessor of the underlying materials, and that they will do this for themselves and for third parties (Kanngiesser, Gjersoe, and Hood, 2010;Kanngiesser and Hood, 2014). Importantly, young children also distinguish between creativity and labor, and they place more value on the former. In a series of studies, researchers found that four-and six-year-old children prefer pictures that depict their ideas over ones on which they have labored (Li, Shaw, and Olson, 2013). And six-year-olds generally assigned ownership of a picture to the person who contributed the ideas about the picture rather than to the person who contributed the labor. Even very young children understand that ideas can be created, that creativity is especially valuable, and that creativity can lead to ownership relationships with the objects that embody the creativity.

Moreover, young children also believe that taking others' ideas or objects embodying those ideas is wrong. Two-and three-year old children will object when someone attempts to take objects which they have created (although they do not yet object on behalf of third parties). In a different study, three-year-old children protested when someone threatened to destroy an object that a third party had created (Vaish, Missana, and Tomasello, 2011). In addition, by the time they are six or so, children also object to others who take ideas (Olson and Shaw, 2011). In one study, children were shown videos of people who drew unique pictures and people who copied others' pictures. The children were then asked to rate how good or bad each person was. They rated the copier significantly worse than they did the creative drawer. These children typically mentioned copying or something similar as the reason why they rated the plagiarist poorly. Interestingly, however, the copier was rated as less bad than someone who stole a piece of physical property.

Gregory Mandel (2014) has conducted experimental and survey research on adults' perceptions of various aspects of IP doctrine. Mandel surveyed 1,719 Americans on their views about four separate IP issues using vignettes about creations and inventions. For each issue, subjects either read a vignette about copyrightable authorship or patentable invention. In the first scenario, subjects were willing to award monetary damages for both copyright and patent infringement, but they were more willing to do so for the latter. The second scenario featured new but not particularly creative efforts by an author or inventor. Respondents were generally willing to give both creators IP rights, even though only the copyright author would have received them under current US doctrine. The third scenario tested whether independent creators of new works or inventions deserve IP rights. Again, a majority of respondents believed they do, consistent with copyright law but inconsistent with patent law. In the final scenario, subjects in both the copyright and patent conditions were about equally split on whether joint creators deserved IP rights. Finally, Mandel surveyed participants on the appropriate basis for granting IP rights, and he found substantial support for a natural rights approach to IP law that contradicts the common view among scholars that IP rights are consequentialist in nature.

While Mandel focused on the concurrence between people's beliefs and formal IP doctrine, other scholars have studied the emergence and enforcement of informal social norms that govern certain creative communities. Bauer, Franke, and Teurtscher (2014) analyze the norms against copying in an online community of t-shirt designers, Threadless.com. Using content analysis and survey research, they describe seven norms that govern the community. These norms include prohibitions on copying and requirements of permission and attribution. They also include affirmative duties on members to police others' behavior and report violations to the community. Bauer, Franke, and Teurtscher (2014) also replicate their survey data with a live intervention into the Threadless community. They posted copied designs and measured whether they were detected, shared, and negatively remarked on. Most copied designs were detected, and the response was consistent with the norms discovered in the survey data. Ultimately, the authors conclude that this community's norms emerge and are enforced because members believe that doing so is morally correct.

Closely related to the issue of moral norms about IP is the question of people's judgments of IP infringement. In the past several years, scholars have begun to focus on how people decide whether one work infringes the rights of another work and how people determine damages for IP infringement.

For example, Jamie Lund (2011) has examined how people apply the "lay listener test" in music composition copyright cases. According to the test, jurors should compare only the original aspects of the plaintiff's composition to the compositional elements of the defendant's work. This means that jurors should consider only the two compositions' melody, rhythm, and harmony and ignore aspects of the ways in which the works were performed, such as their style, tempo, or instrumentation.

Lund's experiment had lay subjects compare two songs to determine whether the allegedly infringing song was substantially similar to the plaintiff's song. In one condition, subjects heard the two songs played with similar style, tempo, and instrumentation. In the second condition, however, the allegedly infringing song was performed in a strikingly different style (e.g. calypso). Subjects in the first condition were substantially more likely to find the two songs to be compositionally similar (mean = 4.36 on a scale of 1-5) than were subjects in the second condition (mean = 2.23). These results held for another version of the study that included a more formal jury instruction. Lund also ran the study with a second pair of songs with the same differences between conditions, but the results were not as strong. For this pair, subjects were less influenced by performance and more sensitive to jury instructions about what to compare. Nonetheless, this study casts considerable doubt on the ability of lay jurors to make reliably and valid determinations about copyright infringement in music composition cases. Shyamkrishna Balganesh, Irina Manta, and Tess Wilkinson-Ryan (2014) have also studied how people apply copyright law's substantial similarity doctrine. As in Lund's (2011) study, Balganesh et al. were interested in the extent to which extraneous information might influence subjects' assessments of similarity. Here they studied whether knowledge that the defendant copied from the plaintiff and information about the plaintiff's level of effort inappropriately affect similarity determinations. In their first study, subjects recruited from Amazon Mechanical Turk were shown a series of six pairs of works and asked to assess their similarity on a scale of 1-7. In one condition, subjects were told that one of the works copied from the other work, while in the other condition, subjects were told that the works were created entirely independently of one another. This difference should be irrelevant, but the authors found a significant difference in the aggregate similarity judgments for works in the "copied" condition compared to those in the "not copied" condition. Thus, when people know that a work was copied from another work, they tend to consider them to be more similar to each other. In a second set of studies reported in the paper, Balganesh et al. (2014) used a similar method to determine whether information about the creator's level of effort and the economic impact of infringement affected similarity judgments. In on comparison, subjects were told either that the work took about two months to create or that it took about ten minutes to create. 5 In another comparison, subjects were told either that the second work significantly impacted the market share of the first or that it had no effect on its market share. Here, the experimenters found a significant difference only for the first comparison but not for the second. In the labor comparison, the extraneous information affected subjects' judgments, with those in the highlabor condition judging the works to be significantly more similar than those in the low-labor condition. But there was no difference in similarity judgments between the groups who were given different information about market effects.

The authors suggest that the effects in the studies might arise from either attentional biases (where subjects divert their attention towards similarities when they know of copying) or due to motivated reasoning. They argue that the latter is more likely in their labor study, where subjects who believe that copying another's creative labor is bad tried to find greater similarities between the two works in order to punish the wrongdoer. Interestingly, though, subjects would also likely have thought that causing substantial market harm is bad, but the differences in market harm did not seem to lead to higher levels of similarity judgments.

Experimental studies of patent infringement issues also exist. For example, Mandel (2006) has examined the role of hindsight bias in non-obviousness judgments, and David Schwartz and Christopher Seaman (2013) have measured the effects of standards of proof on patent invalidity determinations. Because these studies are discussed elsewhere in this volume (chapters X and Y, respectively) we will not repeat those discussions here.

Although experimental research increases investigators' ability to control environments and study causality, these features come at a cost. Laboratory settings differ from real-world creative environments in many ways that might affect the validity of experimental findings. In this section, we discuss a variety of issues in experimental IP research that impact validity. We also discuss opportunities for future research that could address these concerns.

Inventing a new pharmaceutical or producing a movie takes years of intellectual effort. The creativity that researchers can study in the laboratory, however, may involve minutes or even seconds. Moreover, some IP experiments do not even involve tasks that require creativity or innovation. Instead, the tasks simply require algorithmic search. It is difficult to know to what extent these differences affect studies' findings.

Research suggests that tasks involving creativity are different from those involving pure effort or search (Amabile, 1979). Compared to effort or search tasks, creativity tasks may generate more intrinsic motivation, which may be differentially affected by incentives and scoring. In addition, people tend to value things that they created more than things that they own, which may affect important aspects of IP markets that researchers want to study (Buccafusco and Sprigman, 2011). For these reasons, researchers should attempt to use tasks that involve creativity, and not simply algorithmic tasks, whenever possible.

Of course, not all creativity tasks are the same, and different research questions might be better studied with some tasks rather than others. The creativity involved in copyrighted works is often open-ended, so it is best studied with divergent creativity tasks. Patented innovations might be more similar to convergent creativity tasks, where there are one or only a few appropriate answers.

A related issue of validity arises with the populations recruited to study IP issues. Traditionally, many social scientists have relied upon available students to serve as experimental subjects. This can cause problems, however, where students differ from the population of interest in systematic ways. Recently, many researchers, including some studying IP, have used subjects recruited from Amazon Mechanical Turk (AMT). AMT subjects are often more demographically diverse than undergraduate student populations, but they still are not representative of the U.S. population. AMT can be used with previously arranged panels of subjects to more closely match populations of interest (e.g. Buccafusco, Heald, and Bu, 2016). This will be particularly valuable when studying questions of liability or other issues that juries face.

Researchers should also explore opportunities to study real-world creativity experimentally. Doing so can minimize validity concerns arising from creativity tasks and subject pools. Academic researchers may have access to students at their universities who are engaged in creative writing, design, architecture, and engineering projects. Admittedly, the classroom environment is different from the environments in which creativity is exercised outside the educational setting, just as creativity in the lab is different from creativity in real life. But diversifying the settings in which creativity is studied experimentally will, at very least, help us understand whether and when findings are driven, in whole or in part, by the experimental setting. There are also possibilities for studying creativity experimentally in natural settings, including in online communities such as Kickstarter and Topcoder. Using subjects drawn from these communities could improve the ecological validity of experimental research, although designing and running experiments in natural settings raises the related possibility that behavior in the natural setting may be altered when subjects understand they are being studied.

The validity of experiments can also be affected by the use of incentive-compatible elicitation methods. Researchers often worry that subjects who have no financial motivation to respond truthfully may not do so. Fortunately, almost all of the creativity studies discussed above use monetary incentives to encourage subjects to respond appropriately. The same is not necessarily true of all survey research, which can bias results in unpredictable ways. Survey researchers should consider options for encouraging subjects to respond truthfully.

Another, and important, validity issue involves the focus on individual rather than group decision-making in IP experiments. Almost all of the experiments described above involve individual actors, as creators, mock jurors, or survey participants. Many situations that interest IP researchers, however, involve group decision-making. This is a limitation not simply of IP experiments but of most behavioral research. It is difficult to predict how the different contexts might affect behavior. Would a group of creators tend to act more rationally than the average of its members or would an extreme member further bias the group (Sunstein & Hastie 2008)? In fact, both of these outcomes are likely in certain circumstances. Subsequent IP experiments should address the effects of creating in teams, how agents shape creators behavior, and how a jury's decisions may differ from those of individual jurors.

Finally, too many publications in this field report only a single experiment or survey and make no attempt to replicate results with additional tests. In the social sciences, multiple experiments are usually included within the same paper to assess the replicability and robustness of results. Running multiple studies also allows researchers to understand how changes in context or procedure may affect their results. For example, do results found using a convergent creativity test remain the same for a divergent creativity test? Given the early state of the literature, with few existing studies and methods to draw upon, researchers should attempt to provide multiple tests of their research questions whenever possible.

Finally, one of the principal challenges of experimental work involves the difficulty of matching experimental variables with real world phenomena. This can be particularly difficult when the phenomenon being studied is creativity or innovation. These concepts do not have well settled meanings either in the social science or in the legal literatures. Nonetheless, experimentalists should attempt to clarify the conceptual account of creativity or innovation that they are using.

For example, consider a hypothetical study with two conditions. The data show that condition 1 produces better innovations in terms of absolute quality versus condition 2. But, the data also show that condition 2 produces more innovations than condition 1. Determining which treatment is more conducive to innovation is a difficult question and one that relies on more fundamental normative questions about the kinds of innovation that IP law is supposed to promote. What counts as valuable for creative production could include a variety of different goals, including absolute creativity, speed of creativity, and distribution of creativity.

As yet, however, researchers have done an insufficient job of clarifying the aspects of creative or innovative production that they are studying. We would like to see future studies report a variety of creativity measures. This would allow for a fuller normative assessment of the data. Alternatively, if researchers focus on only one aspect of creative performance, they should justify that decision.

Experiments have begun to provide valuable information about some of the foundational questions in IP law, and we expect that the experimental IP law and economics scholarship will continue to expand. Experiments afford IP scholars with an opportunity to investigate questions for which no data exists, or for which access to data held in private hands is restricted. Like all empirical scholarship, experiments raise issues of methodology, including the overarching question regarding the extent to which an experiment, which is necessarily highly stylized, can produce findings that are suitable to use in the assessment of IP policies. While it is rare, and perhaps impossible, for one experiment to definitively address a policy question, we believe that over time a variety of experimental approaches to a particular question -supplemented, if possible, by observational quantitative studies and qualitative or ethnographic studies -can add substantially to our ability to assess the effectiveness of IP policies.