AI Systems as State Actors

Advocates and experts are increasingly concerned about the rapid introduction of artificial intelligence (AI) systems in government services, from facial recognition and autonomous weapons to criminal risk COLUMBIA LAW REVIEW [Vol. 119:1941 assessments and public benefits administration. 1 Every month, more algorithmic and predictive technologies are being applied in domains such as healthcare, education, criminal justice, and beyond. 2 A range of "advocates, academics, and policymakers have raised serious concerns over the use of such systems, which are often deployed without adequate assessment, safeguards, [or] oversight." 3 This is due, in part, to the fact that government agencies commonly outsource the development-and sometimes the implementation-of these systems to third-party vendors. 4 This outsourcing often leaves public officials and employees without any real understanding of those systems' inner workings or, more importantly, the variety of risks they might pose. Such risks range from discrimination and disparate treatment to lack of due process, discontinuance of essential services, and harmful misrepresentations. 5 These risks are neither hypothetical nor intangible. Today, AI systems help governments decide everything from whom to release on bail, 6 1. See Litigating Algorithms, AI Now Inst., (Sept. 24, 2018), https://ainowinstitute.org/ announcements/litigating-algorithms.html [https://perma.cc/683L-NSBJ] [hereinafter Litigating Algorithms Announcement]; infra section I.A. The term "artificial intelligence" has taken on man y meanings, especially in conversations about law and policy. For this Essay, we will use it as a broad umbrella term, covering any computational system that utilizes machine learning, including deep learning and reinforcement learning; neural networks and algorithmic decisionmaking; and other similar techniques to generate predictions, classifications, or determinations about individuals or groups. We choose this definition in part because, while some of the systems we discuss may not actively incorporate the most modern AI techniques, they are designed with the same objectives in mind and aim to usher in AI capabilities as soon as they are feasible or available.

3. Litigating Algorithms Announcement, supra note 1. 4. See, e.g., AI Now Inst., Litigating Algorithms: Challenging Government Use of Algorithmic Decision Systems 7 (2018), https://ainowinstitute.org/litigatingalgorithms.pdf [https://perma.cc/FSG5-JBHT] [hereinafter Litigating Algorithms] ("Government agencies adopting these systems commonly enter into contracts with third-party vendors that handle everything.").

5. For a survey of these risks and concerns, see generally Solon Barocas & Andrew D. Selbst, Big Data's Disparate Impact, 104 Calif. L. Rev. 671 (2016) (using the lens of antidiscrimination law to explore bias arising from data mining); Danielle Keats Citron & Frank Pasquale, The Scored Society: Due Process for Automated Predictions, 89 Wash. L. Rev. 1 (2014) (warning that additional procedural safeguards are necessary for automated prediction systems); Danielle Keats Citron, Technological Due Process, 85 Wash. U. L. Rev. 1249 (2008) (proposing a "technological due process" model to vindicate procedural values in an era of automation); Kate Crawford & Jason Schultz, Big Data and Due Process: Toward a Framework to Redress Predictive Privacy Harms, 55 B.C. L. Rev. 93 (2014) (arguing that procedural due process provides a framework for the regulation of big data); David Gray & Danielle Citron, The Right to Quantitative Privacy, 98 Minn. L. Rev. 62 (2013) (raising concerns over the use of algorithmic systems to establish probable cause for law enforcement searches or arrests).

6. See Julia Angwin, Jeff Larson, Surya Mattu & Lauren Kirchner, Machine Bias, ProPublica (May 23, 2016), https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing [https://perma.cc/BH93-NG7C].

to how many hours of care disabled individuals will receive, 7 to which employees should be hired, fired, or promoted. 8 Yet as decisionmaking shifts from human-only to a mixture of human and algorithm, questions of how to allocate constitutional liability have remained largely unanswered.

The majority of solutions to these concerns have focused on technological or regulatory oversight to address bias, fairness, and due process. 9 However, to date, few if any of these approaches have succeeded in providing adequate accountability frameworks, either because they have failed to address the larger social and structural aspects of the problems or because there is a lack of political will to implement them. 10 As such, it is time to consider new paradigms for accountability, especially for potential constitutional violations.

One underexplored approach is the possibility of holding AI vendors accountable for constitutional violations under the state action doctrine. Although state actors are typically governmental employees, a private party may be deemed a state actor if (1) the private party performs a function that is traditionally and exclusively performed by the state, (2) the state directs or compels the private party's conduct, or (3) the private party acts jointly with the government. 11 This Essay explores this approach to AI accountability in three parts. Part I outlines the current state of play for government use of AI systems, especially those involved in key governmental decisionmaking processes. Part II reviews the relevant case law and literature on the state action doctrine, focusing on the public function, compulsion, and joint participation theories, and how these theories might apply to vendors of AI systems that government uses. Finally, Part III discusses the normative arguments in favor of applying the state action doctrine to close the AI accountability gap. Specifically, this Essay argues that-unlike traditional technology vendors that supply government actors with primarily functional tools, such as a computer operating system, word processing program, or web browser-AI vendors provide government with tools that assist or supply the core logic, justification, or action that is the source of the constitutional harm. Thus, much like other private parties whose conduct is fairly attributable to the state, vendors who build AI systems may also subject themselves to constitutional liability.

To date, there is no comprehensive map or even agreed-upon methodology for tracking government use of AI in the United States, although some efforts are currently underway at the federal, state, and city levels. 12 Until the existence, design, and functions of these systems can be successfully documented, assessing their impact on constitutional accountability will be challenging. 13 In particular, there have been two main challenges to public scrutiny of AI: (1) lack of clear public accountability and oversight processes; and (2) objections from vendors that any real insights into their technology would reveal trade secrets or other confidential information. 14 12. For some preliminary attempts, see, e.g., Frank Pasquale, The Black Box Society: The Secret Algorithms that Control Money and Information 18, 140-58 (2015) (explaining that meaningful reform "mean[s] focusing less on trying to control the collection of data up front, and more on its use-how companies and governments are actually deploying it"); AI Now AIA Report, supra note 10, at 16 (proposing a self-assessment process that provides "an opportunity for agencies to develop expertise when commissioning and purchasing automated decision systems"); Robert Brauneis & Ellen P. Goodman, Algorithmic Transparency for the Smart City, 20 Yale J.L. & Tech. 103, 109 (2018) (testing the opacity of six predictive algorithms used by different local governments).

13. See, e.g., AI Now AIA Report, supra note 10, at 6 (explaining that algorithmic impact assessments can "provide communities with information that can help determine whether those systems are appropriate"); Press Release, Admin. Conference of the U.S., ACUS Announces New Initiatives on the Use of Artificial Intelligence in the Federal Administrative Process (Nov. 28, 2018), https://www.acus.gov/newsroom/news/acus-announces-new-initia-tives-use-artificial-intelligence-federal-administrative [https://perma.cc/4UUB-YECL] (announcing a collaboration with leading scholars to produce a report on AI's application to administrative adjudication and rulemaking); Thus, while some information can be gleaned via public processes, investigative reporting, or open records, this information is often generalized and lacking in useful detail. 15 For example, at the federal level, the few glimpses into the state of AI have come through self-initiated processes, such as the Obama Administration's AI policy process. 16 Recently, the Administrative Conference of the United States announced its own process and forthcoming report on "existing and potential uses of artificial intelligence to improve administrative adjudication, rulemaking, and other regulatory activities throughout the federal government." 17 At the state and local levels, several commissions and task forces have begun to look into these questions. . Note that one of the authors of this Essay, Jason Schultz, was a member of the Office of Science and Technology Policy AI Policy Team during this period and contributed to some of the Team's reports. The Trump Administration has also flagged some of these issues in the President's recent executive order on artificial intelligence. See Exec. Order No. 13,859, 84 Fed. Reg. 3967 (Feb. 11, 2019). However, it is unclear exactly what efforts will be made as the order does not provide any direction, resources, or other guidance on how to proceed. 17. Admin. Conference of the U.S., supra note 13. 18. See, e.g., Idaho Code § 19-1910 (2019) (establishing that all pretrial risk assessment tools should be transparent and open to public assessment); DJ Pangburn, How to Lift the Veil off Hidden Algorithms, Fast Company (Jan. 28, 2019), https://www.fastcompany.com/90292210/transparency-government-software-algorithms [https://perma.cc/2DAN-G4EV] (noting algorithmic accountability efforts in Washington State; Santa Clara County, California; Oakland, California; Berkeley, California; New York City; and Seattle); New York City Automated Decisions Task Force, NYC.gov, https://www1.nyc.gov/site/adstaskforce/index.page [https://perma.cc/64H5-PNDM] (last visited July 29, 2019) (describing the composition and purpose of New York City's task force to evaluate the implications of automation in the city's decisionmaking).

Behind the scenes, however, government use of privately designed algorithmic systems is increasing. For example, there have been reports-but little meaningful public disclosure-on the development of various systems within Immigration and Customs Enforcement that raise constitutional concerns, including at least one system provided by the well-known data analytics firm Palantir. 19 In particular, evidence that has recently come to light suggests that Palantir helped provide the "intelligence" to identify and separate undocumented immigrant children from their families. 20 Another example is the Trump Administration's recent budget request to Congress, which included a section from the Social Security Administration (SSA) explaining that it would study "whether to expand the use of social media networks in disability determinations, partly to help identify fraud." 21 While the current budget proposal discusses human monitoring, 22 a separate SSA document notes that it is developing an "Anti-Fraud Enterprise Solution," which will "integrate data from multiple sources and use industry-proven predictive analytics software to identify high-risk transactions for further review." 23 The Trump Administration's 2019 executive order on artificial intelligence also lays the path for all federal agencies to adopt AI systems moving forward. 24 As we slowly learn more about these systems, it is becoming clear that they represent a range of public-private configurations. Some are developed entirely "in-house" by government, 25 others by contractors or as a licensed service, 26 or even as a "donation," 27 which may impede oversight. But few publicly available documents note specifically how constitutional accountability is allocated in each system, especially within joint public-private endeavors.

In mapping AI accountability, one often overlooked resource is the courtroom, "where evidence, expert testimony, and judicial scrutiny reveal new insights into the current state of AI systems." 28 Recently, we spearheaded an effort via the AI Now Institute, in partnership with the New York University School of Law's Center on Race, Inequality, and the Law and the Electronic Frontier Foundation, "to conduct an examination of current United States courtroom litigation where the use of algorithms by government was central to the rights and liberties at issue in the case." 29 Our first report focused primarily on three case studies in which AI decisionmaking systems were already prevalent in government:

(1) Medicaid and disability benefits, 30 (2) public teacher employment evaluations, 31 and (3) criminal risk assessment. 32 At our most recent workshop, we learned about new litigation involving the use of algorithmic decision systems in unemployment benefits. 33 1. Medicaid and Disability Benefits. -Our first session began with the story of Tammy Dobbs, who has cerebral palsy. In 2008, Tammy moved from Missouri to Arkansas, where she was able to sign up for a state Medicaid waiver program to pay for a caretaker. 34 Tammy uses a wheelchair and does not have full use of her hands, so she needs help with many basic daily tasks, such as going to the bathroom and bathing. 35 The initial nurse that assessed Tammy under the program decided that she should have fifty-six hours of home care per week, the maximum allowable. 36 This all changed in 2016 when Tammy's annual assessment came with a new decisionmaker-an algorithm on a laptop computer. 37 The human assessor asked similar questions to those asked in previous years, but after entering the answers into the algorithmic system, Tammy's allowable care hours were reduced to thirty-two hours per week from fifty-six. 38 Worse yet, the computer system provided no explanation or opportunity to discuss the change, let alone guidance on how Tammy would be able to adjust to the reduction in home care hours. 39 The human who accompanied the computerized decisionmaker couldn't help either, nor would the state when Tammy complained. 40 Unfortunately, the story of Tammy Dobbs was not an isolated one. Hundreds of disabled Arkansans saw their care hours suddenly and drastically cut. They began to complain to the state, and later to attorneys, such as Legal Aid's Kevin De Liban, about the systematic and unexplained cuts in their disability benefits. 41 These cuts were all apparently the result of a new algorithmic system-an early form of AI-that the state had adopted as a cost-saving measure in an era when budgets were tight and healthcare costs continued to rise. 42 And Arkansas wasn't alone. A similar situation had also developed in Idaho under their Medicaid program. 43 Ultimately, both De Liban and the ACLU of Idaho sued in their respective states, claiming that the "faulty algorithmic decision systems improperly diminished or terminated benefits and services to individuals with intellectual, developmental, and physical disabilities." 44 The plaintiffs prevailed in both of these lawsuits, using a combination of constitutional and statutory claims to enjoin the use of these programs. The failure of the state to provide adequate notice of the change-or an explanation of how the new algorithmic system would work-was central 45. K.W., 180 F. Supp. 3d at 719-20 (outlining the due process violation arising from the lack of notice given to beneficiaries regarding their benefit calculations); Ark. Dep't of Human Servs. v. Ledgerwood, 571 S.W.3d 911, 913 (Ark. 2019) (explaining that, on remand, the Arkansas circuit court issued a permanent injunction on the use of the challenged program); Class Action Settlement Agreement at 2-3, K.W., 180 F. Supp. 3d (No. 1:12-cv-00022-BLW) (outlining the plaintiffs' due process, equal protection, and statutory claims). [Vol. 119:1941 to each decision, rendering the deployment of the systems illegal. 46 Notably, the question of how to address the individualized deprivations for each plaintiff remained undetermined upon remand to the respective trial courts. 47 These cases, however, were not situations in which negligent or vindictive government officials actively sought to deprive beneficiaries of their entitlements. Rather, these cases emerged when agencies attempted to implement complex yet archaic algorithmic formulas in computer systems to govern benefit assessment and disbursal. These AI systems were implemented without meaningful training, support, or oversight, and without any specific protections for recipients. 48 This was due in part to the fact that they were adopted to produce cost savings and standardization under a monolithic technology-procurement model, which rarely takes constitutional liability concerns into account. 49 Instead, "these systems typically target populations that are considered the 'most expensive,' which often include the most politically, socially, and economically marginalized communities, who, because of their status, are more likely to need greater levels of support." 50 Thus, an algorithmic system itself, optimized to cut costs without consideration of legal or policy concerns, created the core constitutional problems that ultimately decided the lawsuits.

These problems were also exacerbated as the result of a pattern that has emerged in which AI systems are adopted from state to state through a pattern of software contractor migration, by which AI vendors-like traveling sales representatives-usher the system from one state to another, training it on one state's historical data and then applying it to the new population. 51 Through this migration, patterns of bias or 46. Ledgerwood, 530 S.W.3d at 341; see also K.W., 180 F. Supp. 3d at 720 ("Crucial here is that the notice provide the reasons for the budget reduction so that the participant can challenge the reduction, and this requires the IAP to explain what she relied upon.").

47. See, e.g., K.W., 180 F. Supp. 3d at 722 (explaining that "[t]here are simply too many questions to rule as a matter of law" on either party's motion for summary judgment regarding the plaintiffs' individual claims).

48. See Litigating Algorithms, supra note 4, at 7; see also Class Action Settlement Agreement, supra note 45, at 6-9.

49. See Litigating Algorithms, supra note 4, at 7. 50. Id. 51. See id. ("Many states simply pick an assessment tool used by another state, trained on that other state's historical data, and then apply it to the new population . . . ."). Furthermore, "there are frequent flaws and errors in how these assessment systems are implemented and how they calculate the need for care." Id. Government agencies adopting these systems commonly enter into contracts with third-party vendors that handle everything. See supra note 26. The agency, particularly frontline staff that are most familiar with the Medicaid program and its challenges, has little to no involvement in how the tool analyzes data and produces calculations. See Litigating Algorithms, supra note 4, at 7-8. Because these tools are often based on private systems licensed to government agencies, the design specifications and particularities of the technical system are considered trade secrets of the vendor and are not publicly available. Id. at 8. discrimination can proliferate technologically outside of the actions or intentions of any individual state employee. 52 In terms of litigation outcomes, a key finding was that these cases involved claims against the government agencies alone and not the thirdparty AI vendors. 53 In bringing their claims against traditional government actors, plaintiffs were able to succeed, in part, on constitutional due process and administrative law theories that challenged the lack of notice, explanation, and ability to comment on or contest the changes to public benefit systems. 54 This was especially relevant for the plaintiffs, who were individuals with intellectual or developmental disabilities. Notably, procedural due process claims were able to overcome some of the arguments that disclosure of the technical workings of the systems would violate trade secrecy laws, especially when central to the question of how various public benefits determinations were made. 55 However, these victories offered the plaintiffs and their advocates neither accountability for the core violations they experienced nor any real sense of protection against future harms from similar AI systems. In these cases, the court was willing to rule against the government's use of an AI system when it was deployed without constitutionally proper notice or when it produced discriminatory or otherwise inaccurate determinations. 56 But the claims and the court's jurisdiction were limited solely to the government agency, which had little to no actual involvement in the design, training, implementation, or testing of the system. In a sense, the state was merely a shell to house unconstitutional activity, not the primary 54. In one case, "a court found that the state's automated Medicaid budgeting system was so unreliable that it 'arbitrarily deprive[d] participants of their property rights and hence violate[d] due process.'" Brief for the AI Now Institute, supra note 14, at 19 (quoting K.W., 180 F. Supp. 3d at 718). In the same case, the court found that the state's refusal to provide a manual for a disability scoring tool furnished by a private company frustrated patients' ability to appeal. K.W., 180 F. Supp. 3d at 717. 55. Another finding was the extent to which discovery of errors in the AI's software design or implementation was connected to direct constitutional liability. Such connections were predicated on having access to technical information about the system and access to experts who have the ability to review and interpret the system, both of which can be difficult to obtain. For example, in the Arkansas case, the AI system allocating home healthcare to Medicaid patients failed to accurately understand the care needs of patients with conditions like cerebral palsy or multiple sclerosis. See Ledgerwood, 530 S.W.3d at 339-40, 343. Yet this was only discovered during the course of litigation, and only after the system's code and its associated technical documentation had been carefully examined. See Letter from Kevin De Liban, Legal Aid of Ark., to Becky Murphy, Office of Policy Coordination & Promulgation 5-6 (July 31, 2018) (on file with the Columbia Law Review); Marci Manley, Working 4 You: A Formula for Care, Finding a Solution, KARK (Nov. 17, 2017), https://www.kark.com/news/working-4-you-a-formula-for-care-finding-a-solution-2/ [https://perma.cc/B7K5-URBK].

actor responsible for perpetrating it. At the end of the day, the plaintiffs still had very little understanding of exactly how and why the AI system had reduced their benefits, and even less of an opportunity to hold accountable the private technology vendors who were primarily responsible for the harm. Constitutional accountability mechanisms in the courts inherently involve core judicial concepts such as access to the evidence of the harm 57 and invocation of the court's appropriate remedial and prophylactic powers. 58 In the Arkansas and Idaho litigation, as well as their sister cases throughout the country, constitutional accountability for the creators of the AI systems responsible for the harms has been entirely absent.

2. Public Teacher Employment Evaluations. -The second case study explored similar themes in one of the few successful cases challenging the use of proprietary algorithms to evaluate the performance of public employees. In this case, a school district in Texas implemented a "datadriven" teacher-evaluation model through privately developed third-party software that purported to compare the results of a teacher's students to classroom statistics across the state and within the teacher's prior performance record. 59 The teachers sued the district through their union, arguing that the software was fundamentally inscrutable and that there was no way for teachers to know whether the software was accurately assessing their job performance. 60 The court agreed, holding that the "teachers have no meaningful way to ensure correct calculation of their [evaluation] scores, and as a result are unfairly subject to mistaken deprivation of constitutionally protected property interests in their jobs." 61 The court based its holding on procedural due process, finding that the teachers could proceed to trial on this constitutional issue. 62 The school district soon settled the case and stopped using the software. 63 Again, this case demonstrates one of the challenges of litigating AI claims when the entire algorithmic process is under the control of private third parties. Here, the challenged action was even more remote from the state than in the disability benefits cases discussed above. In this case, the state did not even house the AI system; instead, the system was built, trained, housed, and maintained entirely by a third-party software company, SAS Institute, Inc. 64 SAS fought to keep its source code, training data, and design as secret as possible, initially refusing to let the plaintiffs' experts see any of it and ultimately agreeing only to allow one expert to review the system under extreme constraints: only in the vendor's company office, only on a vendor laptop, and only with a pad of paper and a pen for note-taking. 65 While this lack of access ultimately supported the procedural due process ruling in favor of the teachers against the state, 66 it failed to provide any accountability mechanism against SAS that might have allowed the union to challenge the broader substantive concerns in the case, such as the union's equal protection claim or the claim that the system's determinations were arbitrary.

3. Criminal Risk Assessment. -The third case study focused on a juvenile sentencing hearing in Washington, D.C., in which the presiding judge declined to admit evidence from a long-standing "Violence Risk Assessment" system that had not been properly validated. 67 While the risk assessment in this case had not been implemented as part of an AI system, judges and other state actors at all levels of the criminal justice system rely on algorithmic tools to make decisions about detention and release. 68 In the case discussed at the workshop, participants noted: [Vol. 119:1941

[T]hese violence risk assessment systems have a powerful influence over criminal sentencing outcomes, especially for children. . . . [A] "high risk" finding on one of these algorithmic assessments can result in [a juvenile offender] being sent to a psychiatric hospital or a secured detention facility, separating them from their family and drastically changing the course of their life. Moreover, young people often plead guilty to violent offenses on the condition that they will be eligible for probation rather than incarceration, if they comply with certain court requirements including algorithmic risk assessments. When the risk assessment produces a high risk score, that score changes the sentencing outcome and can remove probation from the menu of sentencing options the judge is willing to consider.

In examining these systems, many advocates have raised significant concerns about embedded racial bias. For example, most assessment systems include several risk factors that function as proxies for race. One risk factor that is often used is "parental criminality" which, given the long and well-documented history of racial bias in law enforcement, including the overpolicing of communities of color, can easily skew "high risk" ratings on the basis of a proxy for race. "Community disorganization" is another influential risk factor if an individual lives in a neighborhood considered to be "violent" or near gang activity, which given the long and well-documented history of private and public housing discrimination, could skew "high risk" ratings on the basis of a proxy for race. 69 Even though defense attorneys were able to convince the judge to find the risk assessment inadmissible in that case, the ruling has not barred that particular assessment system or others from being used in subsequent cases in that juvenile court or in other courts and law enforcement contexts across the country. 70 As numerous AI vendors continue to license these tools, they will continue to evade broader accountability in courtrooms if their systems must be challenged on a case-by-case basis and the remedy is limited to the exclusion of the tool from specific cases.

4. Unemployment Benefits. -A final case study comes from our most recent workshop, which was held in June 2019. 71 In 2013, Michigan governor Rick Snyder launched the Michigan Integrated Data Automated System (MiDAS), a $47 million attempt to utilize the state's vast internal accuracy, and due process concerns require that "use of a COMPAS risk assessment must be subject to certain cautions").

69. Litigating Algorithms, supra note 4, at 13. 70. See Beth Schwartzapfel, Can Racist Algorithms Be Fixed?, The Marshall Project (July 1, 2019), https://www.themarshallproject.org/2019/07/01/can-racist-algorithms-be-fixed [https://perma.cc/KN46-J77Q] (noting that some criminal justice advocacy groups encourage judges to use risk assessment algorithms "in context-as part of a larger decision-making framework that's sensitive to issues of racial justice").

databases to detect and "robo-determin[e]" findings of fraud among recipients of unemployment benefits. 72 Specifically, after cross-checking data with employers, other state agencies, and the federal government, MiDAS "searched for discrepancies in the records of unemployment compensation recipients" and, if it found any, alerted the state Unemployment Insurance Agency (UIA) so that the claimant's file would be flagged as a potential case of misrepresentation. 73 When a file was flagged, MiDAS would initiate an automated process that attempted to transmit a multiple-choice questionnaire to the claimant, requiring a response within ten days. 74 However, because of the system's configuration, many questionnaires never arrived. 75 Others went to dormant accounts or to accounts of individuals whose benefits had already expired. 76 The questionnaire attempted to ask the recipient the following: Did you intentionally provide false information to obtain benefits you were not entitle[d] to receive? Yes No Why did you believe you were entitled to benefits?

1. I needed the money 2. I had not received payment when I reported for benefits 3. I reported the net dollar amount instead of the gross dollar amount paid 4. I did not understand how to report my earnings or separation reason 5. I thought my employer reported my earnings for me 6. Someone else certified (reported) for me 7. Someone else filed my claim for me 8. Other 77 The system did not provide any other means of notice or response, and failure to respond to the questionnaire or any affirmative answer to even one question would result in a default determination that "the claimant knowingly and intentionally misrepresented or concealed information to unlawfully receive benefits." 78 Once the default determination was made, the UIA combined the MiDAS determination with its Enterprise Fraud Detection Software (provided by third-party vendor SAS Institute, Inc.) and sent the claimant a letter demanding repayment and assessing penalties plus interest, without any opportunity to appeal or otherwise contest the finding. 79 The penalties for nonpayment included "interception of the claimant's state and federal income tax refunds, garnishment of wages, and legal collection activity." 80 Unfortunately for Snyder, the State of Michigan, and many of the recipients of its unemployment benefits, the system adjudicated over 22,000 fraud determinations with an error rate of 93%. 81 According to Steve Gray, the former director of Michigan Law's Unemployment Insurance Clinic and the current head of the UIA, those wrongly accused of fraud were subjected to "highest-in-the-nation quadruple penalties" and often weren't given sufficient notice of the adjudications to allow for proper appeals before the thirty-day deadline. 82 This resulted in an estimated tens of thousands of dollars per person in penalties, interest, and lost wages. 83 A subsequent class action lawsuit was brought against the State of Michigan over MiDAS on behalf of the class of recipients who had been wrongly accused. 84 In that case, the court found that the flaws in the MiDAS system had damaged plaintiffs and "eventually approved a settlement agreement in which the State agreed, among other things, to suspend all [MiDAS] collection activity." 85 78. Id. (quoting Plaintiff's First Amended Class Action Complaint and Jury Demand at 17-18, Cahoo v. SAS Inst. Inc., 322 F. Supp. 3d 772 (E.D. Mich. 2018) (No. 17-10657), 2017 WL 3405195). It is worth noting that this "matching" approach is quite similar to those that were used in Florida and Georgia to attempt to detect "voter fraud" with an "exact match" data-driven system prior to the 2018 elections. See generally Complaint, Ga. Coal. for the Peoples' Agenda, Inc. v. Kemp, 347 F. Supp. 3d 1251 (N.D. Ga. 2018) (No. 1:18-cv-04727-ELR) (alleging that Georgia's "Enet" system rejected registrations for voters if there was any mismatch between records on file with the Georgia Department of Drivers Services or Social Security Administration, even if the mismatch resulted from a government employee's typographical error during data entry).

79 However, as that case only addressed prospective relief against the state, plaintiffs brought a second class action lawsuit against both the individual state actors responsible for directly operating the system and the software companies that designed and implemented it for the state. In that case, Cahoo v. SAS Institute Inc., the complaint alleged that three separate technology vendors-FAST Enterprises, LLC; SAS Institute, Inc.; and CSG Government Solutions-"designed, created, implemented, or maintained the automated system employed by the UIA in adjudicating fraud determinations." 87 The plaintiffs alleged that these companies were state actors and thus were liable under 42 U.S.C. § 1983 for "the deprivation of a right secured by the Constitution or laws of the United States . . . caused by a person acting under the color of state law." 88 In its decision on the defendants' motions to dismiss, the court specifically held that "these contracted companies and individuals, working alongside state officials, played some role in implementing a defective system that placed a significant financial burden on unemployment beneficiaries, and they acted under color of state law when doing so." In other words, at least at the motion to dismiss stage of the civil case against them, the court found that the AI vendors were state actors. Exactly how and why is discussed in further detail below. 90

While the above case studies show that pathways exist, to some degree, for holding governments accountable for how they use AI systems, they also highlight the stark fact that, until the Cahoo case, none of the third-party AI providers faced any liability for the constitutional harms their technology imposed. As shown below, this is largely because constitutional liability doctrines, including liability under 42 U.S.C. § 1983, have traditionally focused on the activities of public actors, such as government agencies or officials. 91 These doctrines operate under the assumption that government actors have both the greatest power and responsibility for upholding those rights and protections, and should therefore be held to the highest levels of accountability. 92 Meanwhile, private actors, such as corporations or citizens, need only be held accountable under traditional tort or regulatory approaches. 93 scholar puts it, "governmental power is, in general, more to be feared than nongovernmental power." 94 However, when private actors wield the power of the state, or "act under color of state law," courts have sought to hold them as accountable as the state. 95 The state action doctrine mediates the border between private actors whose conduct is "fairly attributable to the state" and those whose conduct is seen as unrelated or external-a distinction that, despite its theoretical and formalistic dichotomy, has become increasingly difficult to maintain, if it even existed to begin with. 96 In particular, historical attempts to arbitrage constitutional protections through private sector "outsourcing" and the complex intertwining of public-private partnerships in the modern economy have challenged this separation as a sensible division in many arenas. 97 This is particularly problematic for AI systems, as the power and responsibility bestowed upon AI vendors to provide the functions of government is increasing dramatically. Thus, the applicability of the state action doctrine to AI vendors and their systems will be a central question for AI accountability going forward. 98 To deal with this legacy and complexity, courts have been forced to evolve in their interpretation of the state action doctrine. 99 98. While some vendors may voluntarily attempt to provide versions of transparency or accountability-either for internal ethical reasons or because of external market pressures-the lack of any legal accountability remains a concern, especially in situations in which government actors have little or no incentive to impose accountability on vendors through the contractual or procurement processes. 99. See Edmonson v. Leesville Concrete Co., 500 U.S. 614, 620 (1991) ("Although the conduct of private parties lies beyond the Constitution's scope in most instances, governmental authority may dominate an activity to such an extent that its participants constitutional liability for private parties under the state action doctrine, courts have generally applied three tests: (1) the public function test, which asks whether the private entity performed a function traditionally and exclusively performed by government; 100 (2) the compulsion test, which asks whether the state significantly encouraged or exercised coercive power over the private entity's actions; 101 and (3) the joint participation test, which asks whether the role of private actors was "pervasively entwined" with public institutions and officials. 102 Despite this seemingly well-articulated approach, Supreme Court cases on the subject of state action have "not been a model of consistency," 103 and therefore courts generally have "no single test to identify state actions and state actors." 104 Courts often look to "a host of facts that can bear on the fairness of an attribution of a challenged action to the State." 105 Thus, the fundamental question under each test is whether the must be deemed to act with the authority of the government and, as a result, be subject to constitutional constraints."); Blum v. Yaretsky, 457 U.S. 991, 1002-05 (1982) ("[O]ur precedents indicate that a State normally can be held responsible for a private decision only when it has exercised coercive power or has provided such significant encouragement, either overt or covert, that the choice must in law be deemed to be that of the State."); Lugar, 457 U.S. at 936-37 (noting that "[i]t is a fundamental fact of our political order" that federal law limits constitutional liability to actions "fairly attributable to the State"). Note that "state action" has generally been found to be synonymous with "under color of law" in the Fourteenth Amendment context.

The first test focuses on whether the private actor is engaged in a core governmental function that has been exclusively and traditionally performed by the state. 107 As the Supreme Court noted this term in Manhattan Community Access Corp. v. Halleck, very few "functions" remain exclusive to the state in the modern era of public-private partnerships and competition. 108 Rather, many functions are shared or more diversely administered-such as "administering insurance payments, operating nursing homes, providing special education, . . . supplying electricity," 109 or-as the majority found in Halleck-"operating public access channels on a cable system." 110 When a traditional and exclusive public function is "outsourced" to a private entity, however, it still may fall within the scope of the state action doctrine's purview. 111 For example, in West v. Atkins, the Supreme Court unanimously held that a private medical provider that contracted to run a clinic for a North Carolina prison engaged in state action by treating inmates. 112 There, the Court ruled that "the State was constitutionally obligated to provide medical treatment to injured inmates, and the delegation of that traditionally exclusive public function to a private physician gave rise to a finding of state action." 113 In holding the private medical provider liable as a state actor, the Court reasoned that "[c]ontracting out prison medical care does not relieve the State of its constitutional duty to provide adequate medical treatment to those in its custody, and it does not deprive the State's prisoners of the means to 106. Rendell-Baker, 457 U.S. at 838 (internal quotation marks omitted) (quoting Lugar v. Edmondson Oil Co., 457 U.S. 922, 937 (1982)); see also Am. Mfrs. Mut. Ins. Co., 526 U.S. at 50 (requiring the consideration of "whether the allegedly unconstitutional conduct is fairly attributable to the State"); Burton v. Wilmington Parking Auth., 365 U.S. 715, 725-26 (1961) ("Owing to the very 'largeness' of government, a multitude of relationships might appear to some to fall within the Amendment's embrace, but that, it must be remembered, can be determined only in the framework of the peculiar facts or circumstances present.").

vindicate their Eighth Amendment rights." 114 Even Justice Scalia in his concurrence in part agreed that "a physician who acts on behalf of the State to provide needed medical attention to a person involuntarily in state custody (in prison or elsewhere) and prevented from otherwise obtaining it," is constitutionally liable when the physician "causes physical harm to such a person by deliberate indifference." 115 Two key considerations emerge from this analysis. First, the exclusivity of the function is not defined by competition-there are many medical providers in the world-but rather by the voluntary ability of the plaintiff to access (or obtain) the benefits of that functionality elsewhere. In West, because the plaintiff was a prisoner of the state, no such alternative access existed. 116 Second, the completely private status of the actor in question has little to do with constitutional liability. Instead, it is the role that actor plays in the administration of the state's function that governs.

As a result, the public function theory often goes beyond the formalism of identifying functions by type and instead looks more deeply at the stakes of delegating the specific function at issue to private actors. For example, in Giron v. Corrections Corp. of America, the district court applied the doctrine to a private management company that ran a state prison. There, the court reasoned that "[i]f a state government must satisfy certain constitutional obligations when carrying out its functions, it cannot avoid those obligations and deprive individuals of their constitutionally protected rights by delegating governmental functions to the private sector. . . . The delegation of the function must carry with it a delegation of constitutional responsibilities." 118 Similarly, in DeBauche v. Trani, the Fourth Circuit found that a private party is a state actor "when the state has sought to evade a clear constitutional duty through delegation to a private actor . . . [or] delegated a traditionally and exclusively public function to a private actor." 119 In fact, the Supreme Court later went on to frame the public function approach as one almost imposing a legal duty of care on the government in the context of incarceration, explaining in DeShaney v. Winnebago County Department of Social Services that "when the State takes a person into its custody and holds him there against his will, the Constitution imposes upon it a corresponding duty to assume some responsibility for his safety and general well-being." Otherwise, the state would "be free to contract out all services which it is constitutionally obligated to provide and leave its citizens with no means for vindication of those rights." 121 This approach to constitutional responsibility has also been extended to private transportation companies that serve prisons, as well as private residential treatment centers for children under the state's custodial care. 122 Such questions of function and responsibility played out in the recent Halleck case before the Supreme Court. Justice Kavanaugh, writing for the majority, framed the role of the private telecommunications corporation as merely providing and operating the forum of public access television, which is a nonexclusive function. 123 Justice Sotomayor, writing for the dissenters, argued that the corporation did much more than that, having accepted the state's delegation of responsibility for administration and decisionmaking that was so central to the function of a public forum that constitutional liability was appropriate. 124 Thus, when private AI vendors provide their software to governments to fulfill duties that are specifically tied to a state's overall public and constitutional obligations, the possibility of the vendor being held a state actor becomes a reality. For example, when governments use privately provided AI systems to support determinations of criminal propensity 125 or child welfare interventions, 126 the traditional and exclusive nature of those functions, along with their constitutional obligations, puts the AI provider in a similar role to the physician in West. The key question, then, is whether AI vendors-and the systems they create-are merely tools that government employees use to perform state functions, contracts to provide medical care to prisoners undertakes 'freely, and for consideration, responsibility for a specific portion of the state's overall [constitutional] obligation to provide medical care for incarcerated persons.'" (alteration in original) (quoting Rodriguez v. Plymouth Ambulance Serv., 577 F.3d 816, 827 (7th Cir. 2009))).

121. West, 487 U.S. at 56 n.14 (internal quotation marks omitted) (quoting West v. Atkins, 815 F.2d 993, 998 (4th Cir. 1987) (Winter, C.J., concurring and dissenting)).

122. See Lemoine v. New Horizons Ranch & Ctr., 990 F. Supp. 498, 501-02 (N.D. Tex. 1998) (finding that where the state provides housing, food, medical, and educational services to children involuntarily committed to private juvenile residential treatment centers, the owners and employees of those centers may be considered state actors, even when the state has delegated full authority to administer care to the private actor); see also Nguyen v. Prisoner Transp. Servs., No. 3:18-cv-00871, 2019 WL 429678, at *4 (M.D. Tenn. Feb. 4, 2019) (collecting cases of prison transport companies held to be state actors because they could not have performed those services without state authorization).

123. See Manhattan Cmty. Access Corp. v. Halleck, 139 S. Ct. 1921, 1929-31 (2019) ("[M]erely hosting speech by others is not a traditional, exclusive public function and does not alone transform private entities into state actors . . . .").

124. See id. at 1940 (Sotomayor, J., dissenting) ("When a government (1) makes a choice that triggers constitutional obligations, and then (2) contracts out those constitutional responsibilities to a private entity, that entity-in agreeing to take on the jobbecomes a state actor for purposes of § 1983.").

125. The COMPAS system provides one example. See Angwin et al., supra note 6. 126. See Hurley, supra note 23.

or whether the vendor systems perform the functions themselves. 127 If one views the provision of AI as simply the latest technological "tool"like the provision of a hammer-to the state, then the private vendors are outside the definition of state action. On the other hand, when the purpose of the AI is to support or take on a role in the decisionmaking functions of a government official, one could easily imagine a court finding that work fitting within the public function test, more similar to the West example of the private medical professional using his professional judgment in deciding which medical services to provide as opposed to simply providing a scalpel or X-ray machine to a prison hospital.

In fact, many AI vendors specifically optimize their systems to attempt to approximate what a human actor would decide in a similar situation. 128 Take, for example, the Allegheny Family Screening Tool (AFST), an AI system that attempts to forecast child abuse and neglect so that child welfare workers can intervene preemptively and prevent "increased occurrences of drug and alcohol abuse, suicide attempts, and depression" among children in abusive or neglectful situations. 129 The tool, created by a team from Auckland University of Technology, purports to rank the danger of a child's situation from "a green 1 (lowest risk) at the bottom to a red 20 (highest risk) on top." 130 According to one report, AFST was based on a statistical analysis of prior child welfare calls, including "100 criteria maintained in eight databases for jails, psychiatric services, public-welfare benefits, drug and alcohol treatment centers." 131 When vendors supply AI systems to government agencies, the results of the decisions and actions taken are not only attributable to the state but also effectuated through it. In particular, when AI systems are designed specifically for use in governmental domains, such as criminal justice, benefits determinations, or public employment, the conclusion that 127. Note that we discuss AI systems here because, while vendors provide them currently, they may at some point become so sophisticated that constitutional liability may need to apply to these systems themselves. Of course, therein lies a problem with remedies. See Mark Lemley & Bryan Casey, Remedies for Robots 3 (Stanford Law & Econ. Olin Working Paper No. 523, 2018), https://ssrn.com/abstract=3223621 (on file with the Columbia Law Review) ("[I]t turns out to be much harder for a judge to 'order' a robot, rather than a human, to engage in or refrain from certain conduct.").

128. See Ke Li, Learning to Optimize with Reinforcement Learning, Berkeley Artificial Intelligence Research (Sept. 12, 2017), https://bair.berkeley.edu/blog/2017/09/12/learning-to-optimize-with-rl/ [https://perma.cc/P2LT-8UHX] (describing optimization techniques that attempt to replicate in "concrete algorithms" the process by which "humans not only reason, but also reason about their own process of reasoning").

129. Virginia Eubanks, A Child Abuse Prediction Model Fails Poor Families, WIRED ( Jan. 15, 2018), https://www.wired.com/story/excerpt-from-automating-inequality/ [https:// perma.cc/6QLL-HE69].

130. Hurley, supra note 23. 131. Id. [Vol. 119:1941 their design is a core public function is not difficult to imagine. 132 For example, the MiDAS and Enterprise Fraud Detection Software (EFDS) tools in the Cahoo case 133 were specifically designed, developed, and implemented to automate the determination of which unemployment beneficiaries to investigate and penalize, a set of decisions previouslyand, after the litigation, subsequently-initiated, conducted, and supervised by government employees. 134 All of the case studies above demonstrate situations in which algorithmic and AI systems are performing traditional public functions. In the disability cases, it is the function of assessing and recommending public benefit eligibility. 135 In the public employment context, it is the function of assessing and recommending human resources actions. 136 In the criminal justice context, it is the function of evaluating the dangerousness of a defendant. 137 And in the unemployment benefits context, it is the function of investigating and enforcing antifraud regulations. 138 Thus, in many contexts, the case for considering AI vendors as performing public functions could be quite strong.

The second test asks whether the action taken by private entities was encouraged, controlled, or compelled by the state, rather than being done with the "mere approval or acquiescence of the State," 139 or, as it was continually framed in the recent oral argument in Manhattan Community Access Corp. v. Halleck, the extent to which the private entity has discretion to make substantive choices that impact constitutional concerns. 140 For government use of AI, the determination would be quite fact dependent, but to the extent that any allegations of constitutional liability were based on inputs or designs given to the vendor by the state, this could qualify. For example, in Cahoo, the complaint alleged that CSG, the software company that ran and administered the UIA Project Control Office, was charged to do so by the State of Michigan and "received significant encouragement from the State when it implemented, configured, administered and maintained the defective and unconstitutional fraud detection system." 141 Moreover, the contract between CSG and the State of Michigan allegedly delegated managerial authority over the entire project to CSG. 142 In this sense, the state could be seen to have compelled CSG to take actions subject to constitutional liability.

In the other case studies, there are similar elements of compulsion. For example, in the Houston Federation of Teachers case, the key algorithmic inputs-student test scores-were provided entirely by the state, and the federal government required the "value-added" model upon which the AI system was based as a condition to receive $4.35 billion in Race to the Top funds. 143 In the disability benefits cases, much of the logic of the systems and the classification of conditions stems from mandatory state and federal regulations. 144 On the other hand, in criminal risk assessment cases in which the judge ultimately retains the discretionary authority to impose detention and other sentencing conditions, the doctrine might be less applicable. 145 Thus, for AI, the question of who controls the decisions for the design and implementation of the systems, including who provides the data to train and test the system, is relevant. 146 141. Cahoo I, 322 F. Supp ). This has also emerged as a basis for finding state action within the context of the Fourth Amendment and private party searches when the private party acts as "an agent or instrument of the [g]overnment." Gray & Citron, supra note 5, at 135-36 (internal quotation marks omitted) (quoting Skinner v. Ry. Labor Execs.' Ass'n, 489 U.S. 602, 614-15 (1989)). [Vol. 119:1941

The third test for state action asks whether the government was significantly involved in the challenged action that is alleged to have caused the constitutional harm, so much so that the two entities can be considered joint participants. 147 For example, in Brentwood Academy v. Tennessee Secondary School Athletic Ass'n, the Supreme Court held that the private association was a state actor in part because a substantial majority of its members were public schools and public school officials "overwhelmingly perform all but the purely ministerial acts by which the Association exists and functions." 148 By contrast, in Blum v. Yaretsky, the Court considered "whether the decision of nursing homes to transfer or discharge patients constituted state action in light of the state's requirement that physicians certify the medical necessity of nursing home services on a 'long term care placement form' devised by the state." 149 The Court found no state action, holding that even though the state had created the form for evaluating patients, "the physicians, and not the forms, make the decision about whether the patient's care is medically necessary." 150 Instead, a plaintiff must show that the state actor and the private party acted jointly-for example, by carrying out "a deliberate, previously agreed upon plan" or engaging in activity constituting "a conspiracy or meeting of the minds." 151 Examples such as Brentwood Academy and Blum attempt to define the contours of the joint participation test in terms of specific human and organizational activities. 152 If the government were merely involved through standard setting but not active decisionmaking, no joint participation exists. However, as the Brentwood Court acknowledges, "What is fairly attributable [to the state] is a matter of normative judgment, and the criteria lack rigid simplicity." 153 In Cahoo, the court found that two software companies were so entangled in allegedly unconstitutional conduct that they could potentially be found liable as state actors. 154 Government Solutions, the court cited allegations that CSG had directly participated in the administration of unemployment benefits, "a power traditionally exclusively reserved to the State." 155 It also found that the contract between the State of Michigan and CSG had delegated "managerial authority" over the development of MiDAS to CSG as the state's application development and implementation vendor responsible for "the timely delivery of quality information technology services for all stakeholders of the [MiDAS] project." 156 The court also cited the allegations that CSG was responsible for "utilizing and mentoring" state employees on the project. 157 Thus, the court found that these allegations supported a theory that CSG, "acting in concert with the State, was 'entwined' with the UIA in administering and maintaining the robo-fraudadjudication system that deprived the plaintiffs of their constitutional rights." 158 As to SAS, the court found that plaintiffs had successfully pled a § 1983 claim because they had alleged that SAS, acting under the color of state law, "designed, created, implemented, maintained, configured and controlled" the EFDS, which UIA used "to make unemployment insurance fraud determinations." 159 According to the contract between the state and SAS, SAS agreed that the EFDS would "utilize[] data from the Department of Technology, Management, and Budget (DTMB)'s Data Warehouse in the development of UIA Benefit and Tax fraud detection analysis, and the results of that analysis would be integrated with MiDAS." 160 Under the terms of the contract, SAS's responsibilities included "requirements definition, functional design, configuration, testing, implementation, warranty, and maintenance." 161 SAS responded by claiming "that it was merely an independent contractor that provided software to the State." 162 But the court found that the contract provided for more. 163 First, the court agreed with the plaintiffs' allegation that SAS did more than provide the system; it "implemented and maintained" the system. 164 Second, the court found, based on the contract, that SAS was entwined with the state because it played "a non-negligible role in the automated system." 165 For example, the contract required SAS to "schedule, coordinate, and perform all testing activities to validate that the product will operate in its intended environment, satisfies all user requirements, and is supported with complete and accurate operating documentation." 166 The court also highlighted the fact that "SAS was responsible for correcting defects discovered during testing and collaborating with the State to improve the system . . . [and] for providing EDFS performance tuning and defect repair." 167 Thus, the court held that, if the allegations were correct, SAS could plausibly be considered a state actor and be held constitutionally liable for the harm that MiDAS and EDFS had inflicted on the plaintiffs. 168 While the facts in Cahoo make a particularly persuasive case for state action, one could also see the same "entwinement" theory applying in the disability benefits and public employment case studies. 169 In both sets of cases, the government agencies worked hand in hand with private software contractors to design, implement, and-at least in theory-test the AI system that was directly responsible for the constitutional harms involved. On the other hand, as noted above, in the criminal risk assessment context, it is less clear that providers of risk assessments are as "entwined" as CSG and SAS were in Michigan. 170 More facts would need to be known about the level of engagement and involvement in the creation, implementation, and testing of the systems.

Above we have outlined how, under each of the three theories of state action, courts could find private AI providers responsible for the constitutional harm they cause. In this section, we discuss if and when courts should do so.

One instance in which the normative argument for constitutional accountability for private AI system vendors is the strongest is when state accountability is the weakest. For example, in the Arkansas disability 166. Id. 167. Id. 168. Id. The court did, however, dismiss the individual employees of SAS, FAST, and CSG from the suit as the plaintiffs could not show that any of them had specifically and personally been involved in the actions that caused the harm. Id. This raises interesting questions about the role of humans within software engineering. Perhaps the court felt that holding the companies liable was enough and did not want to inflict too much constitutional liability on individuals, but this is hard to distinguish from West, especially because some humans must have been involved in the design, implementation, and testing that the court found sufficient to potentially hold CSG and SAS liable as state actors.

169. See supra sections I.A.1-.2. 170. See supra section I.A.3. case, 171 the state relied on private contractors almost wholesale to design and implement the system that caused the constitutional harm. 172 While the plaintiffs were able to bring claims against the government to stop the ongoing deployment of the AI-driven program, the state agency lacked the capacity to address most of the specific causes of harm directly. The state had very little knowledge of how the AI software code had been written, where the mistakes were made, what data had been used to train and test it, or what means were required to mitigate the concerns raised in the case. 173 The same is true for the Houston Federation of Teachers case, in which not a single employee of the school district could explain, let alone remedy, the methods or outputs of the proprietary AI at the heart of the constitutional liability concerns. 174 This is strikingly similar in many ways to the prison doctors in West, to whom the state had "outsourced" its constitutional obligation to provide responsible healthcare. 175 Of course, one could argue that this is often the case with government vendors and that holding the state accountable is often sufficient because the state can simply demand that the vendor provide the appropriate remedy or it can switch vendors. However, in West, we see that the Court was concerned not only with the specific harm to the single inmate in the case but also with the potential for the state to systematically avoid constitutional accountability by outsourcing potential liability to unaccountable private actors. 176 Holding private doctors personally liable ensures that they too have incentives to mitigate constitutional harms. Applying similar incentives to software vendors that sell AI systems to government agencies would accomplish similar goals. Moreover, while government actors can be assessed for damages when their AI systems violate individual rights, the primary remedy against state actors in such contexts is injunctive relief. 177 Unless vendors are subject to the court's jurisdiction, the court cannot assert any real oversight or impose any specific injunctive relief on that party, 178 even if it is in the best position to fix errors in how the AI performed.

Another normative argument in favor of finding state action applies when AI vendors are underregulated. Currently, regulatory approaches to AI are under discussion, but almost no jurisdiction has enacted rigorous regulatory approaches to ensure accountability, especially for constitutional concerns. 179 When such private accountability gaps exist, state action remedies make more normative sense. Were these gaps to be filled to allow regulators or harmed plaintiffs to sue private actors separately under alternative laws, there would be less justification for designating AI vendors with state actor status. While this was never part of the explicit holdings in previous state action cases, one can see hints of this in several of the key decisions. 180 Again, Cahoo provides an excellent example. There, in the same decision upholding federal civil rights claims against CSG and SAS as state actors, the court also simultaneously dismissed all state tort claims against them. The court concluded that under Michigan's product liability, negligence, and civil conspiracy laws, neither company could potentially be held liable for its actions. 181 Therefore, the only viable claims of relief were procedural due process, equal protection, and freedom from unreasonable seizure of property. 182

A third situation in which normative values argue in favor of state action is where trade secrecy or third-party information is at the heart of the constitutional liability question. For example, in the Houston Federation of Teachers case, none of the school district employees could provide any answers to the core substantive questions concerning constitutional liability in the case. 183 Instead, all of those answers were within the technical and legal power of the vendor. 184 In such cases, considering the vendor a state actor would allow courts access to the necessary information to decide cases while also directly addressing vendor trade secrecy concerns. As parties, technology companies litigate their technologies every day in courts. Allowing those who have been constitutionally harmed to sue the vendors directly would allow plaintiffs and courts to access all relevant information about the AI system, its function, and the role of the vendor in the alleged constitutional violation. Vendors, of course, would have all rights to object or limit discovery under standard civil procedure provisions, including invocation of protective orders. Moreover, many courts have developed specific provisions to narrow and vet claims of trade secrecy. 185

The state action doctrine should be considered as a potential pathway to providing greater accountability for the government use of AI systems. As Professor Gillian Metzger argues, "State action doctrine remains the primary tool courts use to ensure that private actors do not wield government power outside of constitutional constraints." 186 As discussions of AI regulation move forward, the state action doctrine should form part of the landscape of the reasonable and appropriate regime that is ultimately devised. In particular, as AI systems rely more on deep learning, potentially becoming more autonomous and inscrutable, the accountability gap for constitutional violations threatens to become broader and deeper. This may result in both state and private human employees having less knowledge or direct involvement in the specific decisions that cause harm. For example, a new proposed rule from the U.S. Department of Housing and Urban Development creates a complete defense to a prima facie case of housing discrimination when the defendant uses an industry-standard algorithmic model to make its housing decisions. 187 This rule, if adopted, would encourage many actors in the housing industry to use AI systems, knowing that they could avoid liability by blaming the AI itself, even if there was overwhelming evidence that they knew the use of the system would have a disparate discriminatory impact. 188 No doubt there will be many attempts, such as the proposed HUD rule, to allow AI systems to be used as accountability-avoidance mechanisms when companies cause constitutional violations. This is why the state action doctrine must remain a powerful and flexible common law approach for courts to use to redress this gap as it widens. This will be particularly necessary if legislation or agency regulation is slow to materialize or inadequate for the complex task that AI will present in the coming years. 188. Cf. Logiodice v. Trs. of MCI, 296 F.3d 22, 27 (1st Cir. 2002) (holding private entity MCI was not a state actor even though it ran part of the Maine public school system because the alleged constitutional violation-illegally disciplining a student-was carried out entirely and internally by MCI and had no joint participation elements). But see Patrick v. Success Acad., 354 F. Supp. 3d 185, 209 n.24 (2018) (noting that private operators of charter schools are state actors, despite extensive internalization of decisionmaking).