# The Legal Imitation Game: Generative AI’s Incompatibility with Clinical Legal Education

**Authors:** Jake Karr, Jason M. Schultz
**Citation:** "The Legal Imitation Game: Generative AI’s Incompatibility with Clinical Legal Education," 92 *Fordham L. Rev.* 1867 (2024) (with Jake Karr)
**Source:** https://fordhamlawreview.org/wp-content/uploads/2024/03/Vol.-92_Schultz-Karr-1867-1886.pdf

## INTRODUCTION

*p. 1*
Legal practitioners are currently in the midst of a technological maelstrom. Generative artificial intelligence ("GenAI"), and specifically large language models (LLMs), are taking the legal world by storm, and GenAI evangelists and skeptics are furiously debating the potential impacts of the technology. Will the introduction of GenAI turn lawyers into "prompt engineers"? 1 Will it entirely eliminate the need for human lawyers, at least for certain repetitive legal tasks and work product? 2 Or are we living through yet another "Big Tech" hype cycle? 3 Legal educators are engaged in a similar debate. 4 Many in the legal academy-in particular, clinicians and skills instructors who straddle both practice and pedagogy-are asking whether they should teach GenAI tools to law students, how they should teach these tools, and whether to allow students to use the technology in client casework or coursework. 5 These questions are further complicated by the rapid development and deployment of GenAI tools, which have created an atmosphere of not only overwhelming urgency but also perceived inevitability. 6 Legal educators are left between a rock and a hard place. On the one hand, legal employers are racing to embrace GenAI, putting pressure on educators to prepare students for any number of possible AI futures. 7 On the other hand, the technology is novel, uncertain, and risky. 8 Educators must choose between either speeding up to incorporate GenAI tools into their curricula without sufficient evidence and reasons to do so or slowing down to thoroughly understand, assess, and test these tools at the risk of undermining their students' employability in an unforgiving job market. All the while, technology vendors and platform providers are attempting to integrate GenAI into everything, including legal research, writing, and practice tools. 9 They are directly marketing these products to students, 10 often, in our experience, with little consultation with the legal academy and without an agreed-upon framework for evaluating risk or utility.

*p. 3*
So how should those of us in the field of legal education proceed? To answer this question, we must examine the tangible student learning opportunities that we are hoping to create and evaluate whether, where, and how integrating GenAI into law school curricula might help or hinder those opportunities.

*p. 3*
In this Essay, we briefly describe key aspects of the technology that are particularly relevant to, and raise particular risks for, its potential use by lawyers and law students. We then identify three foundational goals of clinical legal education that provide useful frameworks for evaluating technological tools like GenAI: (1) practice readiness, (2) justice readiness, and (3) client-centered lawyering. First is "practice readiness," which is about ensuring that students have the baseline abilities, knowledge, and skills to practice law upon graduation. 11 Second is "justice readiness," a concept proposed by Professor Jane Aiken, which is about teaching law students to critically assess the social and political implications of legal work and the legal system, as well as making space for students to confront systemic injustices and the role of lawyers in perpetuating them. 12 Third is "client-centered lawyering," which at its root is about client empowerment and autonomy, teaching students to recognize the power imbalances present in the attorney-client relationship and the importance of ensuring client agency in decision-making. 13 Although these are by no means the only goals of clinical education, they provide key perspectives and criteria for GenAI assessment.

*p. 3*
Finally, we examine whether GenAI is pedagogically compatible with each of these three goals. We conclude that although GenAI does present some de minimis learning opportunities for practice readiness, it is largely incompatible with justice readiness and client-centered lawyering, especially when considering the serious concerns that the development, deployment, and use of GenAI raise for those clinical programs with public interest missions. [Vol. 92

## I. THE LEGAL IMITATION GAME BEGINS

*p. 4*
In 1950, iconic computer scientist Alan Turing first proposed a test to determine whether a computer could imitate human behavior with sufficient "intelligence" to deceive a discerning human judge who asked it questions. 14 The goal of the test was not for the computer to provide the correct answers but rather for its answers merely to resemble human answers-to imitate the appearance of human thinking rather than to produce actual thinking. 15 Turing therefore called his test "The Imitation Game." 16 Today, as explained below, the legal profession is playing a similar game with GenAI, raising two significant pedagogical and ethical risks: (1) the risk that AI outputs will appear to imitate competent and ethical human lawyering but ultimately fall short of the real thing and (2) the risk that AI users will suffer from automation bias-the presumption that AI outputs are by nature accurate.

*p. 4*
As to the risk of imitation, it is important to recognize that GenAI systems like ChatGPT or CoCounsel have a single objective: recognizing patterns in data to simulate human language and interaction. 17 The key here is simulation-the imitation of a thing as opposed to the thing itself: accurate and competent legal outputs. 18 In his groundbreaking 1950 paper describing The Imitation Game, Turing noted that this preference for imitation over accuracy is a feature of AI and not a bug, as one of the challenges in convincing humans that machines can think is that their answers to human questions may be too accurate:

*p. 4*
It is claimed that the interrogator could distinguish the machine from the man simply by setting them a number of problems in arithmetic. The machine would be unmasked because of its deadly accuracy. The reply to this is simple. The machine (programmed for playing the game) would not attempt to give the right answers to the arithmetic problems. It would deliberately introduce mistakes in a manner calculated to confuse the interrogator. 19 In other words, producing errors becomes a tactical mechanism for AI to "sound" more human. 20 Or as the science fiction writer Ted Chiang noted in his New Yorker essay on the subject, this means that our interactions with such tools provide, at best, a blurry or distorted reflection of "reality" that is both fascinating and entertaining without necessarily being accurate, ethical, or useful. 21 Much like lawyers who feel compelled or pressured to agree with their clients and/or downplay unfavorable analyses, these systems are purposely built to provide responses that please us by imitating the responses we are most likely to believe. 22 Even if those responses are speculative or wrong, GenAI neural network architecture may calculate that wrong answers are more likely to pass as human conversation than correct ones. 23 This, in part, is the reason why they produce "hallucinated" citations. 24 Lawyers are often desperately searching for the "perfect" case to defeat an opponent or reassure their client. That desperation drives their prompts, in turn driving GenAI's responses. 25 To the system, a hallucinated answer is better than no answer at all.

*p. 5*
Unlike most other legal research or educational tools, GenAI is not optimized to help humans become better lawyers-it is optimized to imitate how a human lawyer, good or bad, might respond. 26 Its goal is to feign intelligence and convince as many users as possible that it knows and understands things. 27 Although it sometimes uses accurate and insightful language, that accuracy and insight is a mere side product employed toward the ultimate goal: successful performance. 28 This is similar to the goal of many magicians-to convince us that they can bend the rules of physics and perform amazing feats, despite our knowledge that they cannot. To do so, magicians often will demonstrate that the laws of physics still exist by inviting an audience member up to the stage to test their equipment in order to lower our skepticism and encourage us to believe in the subsequent illusion. GenAI is designed to push us to accept that it has capacities and authorities beyond our knowledge and understanding-and that is exactly why it is so risky to use.

*p. 5*
In addition, GenAI leverages automation bias to convince audiences that these programs understand complex concepts and contain knowledge, when in fact they rarely have the capacity to do either. 29 Automation bias is part of a long history of the Human Computer Interaction (HCI) field in computer science and concerns situations in which human audiences are convinced to believe the outputs of automated systems because those systems appear to be more objective and lacking the biases of humans. 30 In reality, they were created by the very humans who have the biases the audience seeks to avoid. 31 A famous case of this was when Professor Joseph Weizenbaum, an early AI pioneer, created "ELIZA," the first chatbot, in 1964. 32 ELIZA shocked Professor Weizenbaum himself when he saw how quickly many users would believe that ELIZA really was a remote psychologist providing them with qualified mental health information via a text interface, even though it was merely scripted code designed to mirror user responses. 33 We see similar "automation bias" problems with misinformation online and automated decision-making in areas such as criminal justice, health care, education, and employment. 34 So, what does this mean in terms of legal education? It means that law students who seek answers to legal problems from GenAI may very well ask legal questions in their prompts and believe, due to automation bias, that they are receiving high-quality responses. 35 But the truth is that we can never know. Although law students and lawyers can verify the text of a specific case or statute, there is no existing mechanism for auditing or interrogating the logic behind GenAI responses. 36 Much like the magician, they never reveal their secrets.

*p. 6*
AI today is no closer to understanding lawyering than Professor Weizenbaum's ELIZA was to understanding human psychology in 1964. Yet, like ELIZA, it might well pretend to engage with legal issues to convince law students and practicing lawyers that it can. Thus, GenAI tools may imitate some basic forms of human lawyering, but they provide only surface-level learning experiences for students while raising serious ethical concerns. 37 As we discuss below, this leaves GenAI tools generally incompatible with the goals of clinical and, more broadly, legal education. 30. See id. at 616. 31. See id. ("We find that the mix of human biases and seemingly coherent language heightens the potential for automation bias, deliberate misuse, and amplification of a hegemonic worldview.").

*p. 6*
32. See JOSEPH WEIZENBAUM, COMPUTER POWER AND HUMAN REASON: FROM JUDGMENT TO CALCULATION 2-3 (1976)

## A. The Goal of Practice Readiness

*p. 7*
Practice readiness is often considered the traditional goal of clinical legal education, and perhaps the one most commonly recognized outside of the legal academy. 38 Scholars have traced the notion of practice readiness to the clinical legal movement of the 1960s and 1970s. 39 In opposition to the perception that law schools historically failed their graduates by offering little in the way of practical training, practice readiness came to prominence as a call to provide law students with the experiences and skills that they need to adequately and ethically serve their clients when they graduate. 40 Despite its longstanding and widespread use in clinical literature, law school promotional materials, and bar association debates, 41 practice readiness is largely undefined in its particulars, sometimes criticized as "more slogan than idea." 42 Nevertheless, some high-level benchmarks have emerged for evaluating the types of experiences and skills that help train practice-ready graduates. The American Bar Association (ABA), for example, outlines expected learning outcomes for law students in its Standards and Rules of Procedure for Approval of Law Schools, Standard 302: (c) Exercise of proper professional and ethical responsibilities to clients and the legal system; and (d) Other professional skills needed for competent and ethical participation as a member of the legal profession. 43 More specifically, in its interpretation of the standard, the ABA states that "professional skills" may include "interviewing, counseling, negotiation, fact development and analysis, trial practice, document drafting, conflict resolution, organization and management of legal work, collaboration, cultural competency, and self-evaluation." 44 In another recent prominent effort to define and refine practice readiness, the Institute for the Advancement of the American Legal System (IAALS) conducted fifty focus groups of new lawyers and supervisors of new lawyers "to gather data about the knowledge and skills new lawyers need to practice competently." 45 The study suggested that "minimum competence consists of 12 interlocking components-or 'building blocks'" 46 :

*p. 8*
[1] The ability to act professionally and in accordance with the rules of professional conduct; [2] An understanding of legal processes and sources of law;

*p. 8*
[3] An understanding of threshold concepts in many subjects;

*p. 8*
[5] The ability to interact effectively with clients;

*p. 8*
[9] The ability to see the "big picture" of client matters;

*p. 8*
[10] The ability to manage a law-related workload responsibly; [11] The ability to cope with the stresses of legal practice; and [12] The ability to pursue self-directed learning. 47 Against this backdrop, scholars have debated how emerging legal technologies impact what it means to be practice ready. There is growing recognition that practice readiness today includes some level of technical training in, or at least an understanding of, the technologies that attorneys use to serve their clients. 48 The ABA and multiple states have now explicitly incorporated an expanded duty of technological competence into their rules of professional conduct, advising lawyers to "keep abreast of changes in the

## B. GenAI Is Minimally Compatible with Practice Readiness

*p. 9*
GenAI raises new questions as to what it means for law students to be practice ready and what that means in turn for clinicians committed to the goal of practice readiness. Although the technology is fast evolving and its impact on legal practice remains uncertain, current GenAI tools do not appear to have much to offer in the way of practice readiness.

*p. 9*
Evangelists may argue that GenAI "prompt engineering" 50 is the future of legal practice and thus should be recognized as a new skill that law students need to learn to be practice ready. 51 There are obvious reasons to believe that GenAI will lead to further automation of certain segments of the legal profession and certain legal services-processes that have been ongoing for quite some time 52 -and this will require legal practitioners to continue adapting to best serve their clients. If GenAI tools as they presently operate become a routine aspect of legal practice, perhaps prompt engineering becomes a skill that many lawyers feel as though they must learn in order to compete in the legal market or competently represent their clients. 53 Yet this possibility does not on its own demand a reimagining of a clinical curriculum committed to training practice-ready graduates. There are plenty of experiences and skills that clinical programs do not emphasize or prioritize and many useful or popular technologies that are widely employed in legal practice but are not systematically taught in law schools. 54 54. The ABA explicitly acknowledges that law schools should "determine[]" for themselves which "professional skills" to teach their students. AM. BAR ASS'N, STANDARDS AND RULES OF PROCEDURE FOR APPROVAL OF LAW SCHOOLS 2023-2024, Interpretation 302-1 (2023). [Vol. 92 them independently or receive training in them later in their careers. 55 Moreover, there are already signs that prompt engineering is headed for obsolescence. 56 All of this leads us to conclude that the teaching of prompt engineering for prompt engineering's sake may very well be a waste of the limited time that clinicians have with their students.

*p. 10*
The question still remains, however, whether the use of current GenAI tools can significantly assist students in learning other important skills or knowledge that are considered minimum baselines for legal practice. At the moment, we do not believe so. These authors have not seen compelling evidence, for example, that learning engineer "good" prompts and review GenAI outputs translates to material gains in the traditional domains of practice readiness, whether they be any of the "professional skills" contemplated by the ABA 57 or the twelve "building blocks" proposed by the IAALS. 58 Evangelists are excited by GenAI's potential to improve the speed of everyday tasks like legal research and writing and to produce simulacra of legal work product. 59 But helping law students learn to become more efficient does not automatically help them learn to become-or how to belawyers. Even if students can use GenAI tools to produce quality and quick outputs that outwardly meet the standards of legal competence, it is not clear what students can genuinely learn about practicing law through the process of using them. A student who passively prompts a GenAI tool to output a legal brief does not necessarily learn the "legal analysis and reasoning," "problem-solving," or written communication skills that go into producing such a brief, let alone the lessons to be drawn from iteratively drafting and discussing it with colleagues. 60 They are not honing their "ability to interpret legal materials" or "identify legal issues." 61 From this perspective, GenAI use may in fact stifle a student's intellectual and professional growth rather than foster it.

*p. 11*
To be sure, many students have started turning to GenAI tools as study or research aids in seemingly productive ways. Prompting a GenAI tool for a case summary or restatement of law arguably can deepen students' "knowledge and understanding of substantive and procedural law," 62 their "understanding of legal processes and sources of law," 63 and their "understanding of threshold concepts in many subjects." 64 In doing so, though, they are using GenAI tools just as they would any other reference material at their disposal, from casebooks and hornbooks to Westlaw and Wikipedia. This limited pedagogical compatibility between GenAI and practice readiness does not suggest to us that the technology merits special attention or consideration from law school clinics.

## A. The Goal of Justice Readiness

*p. 11*
The concept of justice readiness seeks to expand the goals of clinical pedagogy beyond the value-neutral skills instruction of practice readiness, at least as traditionally conceived. 65 Aiken writes forcefully against the ways in which legal education inculcates the "'hired gun' approach to what it means to be a lawyer." 66 "Clinics must teach skills," of course, "but they should also challenge the conception of law inculcated by law schools. . . [and] work toward inspiring students to bring about a more just society with their legal skills." 67 A clinician committed to justice readiness is a "provocateur for justice" who "actively imbues her students with a lifelong learning about justice, prompts them to name injustice, to recognize the role they may play in the perpetuation of injustice and to work toward a legal solution to that injustice." 68 Preparing students to be justice ready can be particularly helpful in providing them with a foundation to engage in various modes of lawyering focused on "empowering those without power and fighting for justice and equality," including political lawyering, rebellious lawyering, community lawyering, and movement lawyering. 69 Justice readiness encourages clinicians to "pull[] back the curtain and dethrone[] neutrality" by teaching students "how to reflect on their experience, place it in a social justice context, glimpse the strong relationship between knowledge, culture and power, and recognize the role they play in either unearthing hierarchical and oppressive systems of power or challenging such structures." 70 To do this, according to Aiken, clinicians must first engage students in effective critical thinking that transcends "the idea that there is a right and wrong answer to every legal problem." 71 Lifting this veil reveals "the role values play in the justice system." 72 Rather than allow students to "feel powerless" or incapacitated by law's inherent uncertainty, 73 however, clinicians should inspire students to seize upon that uncertainty, "assert [their] own values," and "become proactive in shaping legal disputes with an eye toward social justice." 74 Clinicians committed to justice readiness should "communicate the importance of social justice, the opportunity to make a difference that [a] law degree creates, and the responsibility that [students] bear as lawyers for the delivery of justice in our society." 75 As Professor Deborah Archer notes, "[t]he primary tool" that Aiken "promotes to help students achieve justice readiness is 'disorienting moments' where students have experiences that surprise them because the experience challenges the student's established way of viewing the world." 76 Clinicians must create the conditions that allow for students to experience these disorienting moments: "To find these disorienting moments, clinical professors must engage students in a moral and ethical discourse about the choices they make during . . . representation, and help them become more able to identify injustice." 77 Moments of disorientation and critical reflection can manifest in a variety of contexts. As an example, Aiken suggests selecting "cases that require creative solutions to clients' problems"-cases in which "there is no 'outside authority' from which to draw a remedy" and thus "may force the student to draw from her own knowledge base and to draw connections based on context." 78 Through such critical reflection, students can begin to learn how they may be able to act as "agents of change," rather than "agents of stasis." 79

## B. GenAI Is Largely Incompatible with Justice Readiness

*p. 13*
GenAI triggers acute cultural, ethical, political, and social concerns that appear wholly incompatible with the clinical mission of justice readiness. On one level, legal GenAI tools pose risks to our justice system, including worsening the already unequal access to legal information and services. 80 Public interest lawyers and scholars have long fought to break down barriers to access. 81 Evangelists now argue that the expediency of GenAI tools can help democratize the availability of legal information and increase the scale of legal services. 82 In reality, GenAI will likely undermine these efforts. 83 The capital investment needed to develop GenAI systems, which ranges into the billions of dollars, 84 means that only a handful of powerful and wealthy corporations will be able to provide GenAI tools for legal uses, all of which are also proprietary as to the data sources used to train or fine-tune their models. There are no low-cost or transparent alternatives. Nor is there likely to be one any time soon. To the extent that we teach our students to use and rely on GenAI tools, we further concentrate legal information in the hands of a select few gatekeepers, replicating the legal information asymmetries wrought by previous advances in technology. that each AI-driven interaction uses more than ten times the energy used in a traditional web search. 93 Another found that ChatGPT costs the energy equivalent of 33,000 U.S. households per day. 94 As one study argued, the total electricity demand of our computational infrastructures could reach 20 percent of global electrical demand by 2030, from 1.5 percent today. 95 GenAI systems similarly require enormous amounts of freshwater to cool their energy-intensive, heat-producing graphics processing units. 96 One study made a rough calculation that every exchange with ChatGPT of twenty-five to fifty questions is the equivalent of pouring out a half-liter of freshwater on the ground. 97 Google and Microsoft have also reported major spikes in their water usage-increases of 20 percent and 34 percent respectively in just one year. 98 Unjust labor practices further serve to prop up GenAI systems. Recently, it was reported that OpenAI had hired Kenyan "click" workers for approximately one dollar an hour to remove toxic results from ChatGPT, such as sexually graphic materials and other explicit scenes. 99 The University of Oxford's Fairwork project, which studies the labor conditions for "cloudwork" platforms that support AI providers, has given many scores of less than three out of ten for fairness. 100 Thus, LLMs are by their very nature "soldiers for the status quo." 101 In their accelerating and widespread development and deployment across the legal system and elsewhere, they could potentially reinforce existing economic and power structures as well as the resulting injustices. In their outputs, far from being "vehicle[s] for justice," GenAI tools are stare decisis machines. 102 They consume, replicate, and make predictions based on existing data and depictions of the world, using data as precedent to produce endless variations of nothing more than what has come before. 103 In short, GenAI development and deployment may exacerbate existing injustices because their outputs may reflect them. If justice readiness requires us to "deviate from system-reinforcing behaviors," 104 discouraging the teaching and use of legal GenAI tools may seem eminently reasonable, if not ethically and pedagogically imperative.

*p. 16*
At the same time, justice readiness requires confronting the harsh realities of the law and legal practice, not shying away from them. With good reason, many clinicians may now wish to confront GenAI in a similar fashion and incorporate it into curricula in ways that are at least consistent with this pedagogical goal. Clinicians committed to justice readiness could press students to interrogate how these tools are built and operate, 105 to "pull[] back the curtain and dethrone[] [their] neutrality," 106 to investigate the wide-ranging ethical implications that they pose for the justice system and society, and to recognize the role that lawyers who use these tools may play in causing or reinforcing harm-especially in situations in which they might consider using GenAI to be in a client's best interest. By prompting students "to tease out or hunt down assumptions" 107 that underlie GenAI and to engage in critical reflection and dialogue around its potential uses in legal practice, clinicians may even help lead students into disorienting moments around technology, power, and justice.

*p. 16*
Indeed, clinicians so committed should not stand by as their students resign themselves to the inevitability of any particular GenAI future for the legal profession. We should help our students make informed, value-based, and justice-ready decisions about the technology and their own use of it. this work, and the learning opportunities that may flow from it, are achievable simply by teaching students about these tools. Integrating GenAI conversations and critique into justice-ready coursework does not necessarily require integrating GenAI use into client casework. It is not clear what more could be learned about justice readiness in contemporary legal practice by formally teaching students how to use GenAI tools as well. Thus, we believe that using GenAI tools in law school clinics would do little to serve the goal of training justice-ready graduates.

## A. The Goal of Client-Centered Lawyering

*p. 17*
Following its inception in the 1970s, client-centered lawyering (CCL) has become a centerpiece of law school clinical education. 109 CCL arose in response to perceived flaws in the traditional lawyer-client relationship, which typically involved arms-length interactions, with clients remaining relatively passive after stating their ultimate goals. 110 CCL instead focuses on dynamically engaging with clients and offering them more autonomy and agency as a partner in the legal decision-making process. 111 To do so, CCL requires "a commitment to looking at problems from clients' perspectives, of seeing the diverse nature of the problems, and of making clients true partners in the resolution of their problems." 112 Scholars have since debated and strengthened the concept of CCL by addressing perceived tensions within it. 113 For example, in her influential essay, Am I My Client?: The Role Confusion of a Lawyer Activist, Professor Nancy Polikoff explores what it means to work as a client-centered lawyer for political activists and members of vulnerable populations while also identifying with those movements and populations herself. 114 As part of this exploration, she discusses many of the issues that arise with these identifications, both positive and negative, such as the "insider/outsider" dilemma. 115 As someone who is familiar with the values of a movement, Polikoff discusses how she is able to build trust with clients who share those values (being an "insider"). 116 At the same time, as a lawyer who is expected to conform to the norms and rules of the legal profession, she has been forced into situations in which the disobedient actions and decisions of her clients pose a threat to her professional credibility, even though they align with her political objectives (being an "outsider"). 117 Ultimately, Polikoff concludes that resolving such tensions requires explicitly recognizing and integrating them into her CCL approach. 118 Critiques of bias and discrimination have also been instructive for improving the use of CCL in legal education. As Professor Michelle Jacobs argues, well-intentioned "race neutral" applications of CCL can often disserve clients of color by failing to adequately consider their culturally specific needs, especially when their lawyers do not share the same lived experience with the legal system. 119 This has helped highlight the importance of incorporating discussions of bias and discrimination into clinical education through mechanisms like cross-cultural training within the CCL framework. 120

## B. GenAI Is Pedagogically Incompatible with Client-Centered Lawyering

*p. 18*
As noted above, CCL requires a commitment from lawyers to see problems from clients' perspectives, including the diverse nature of the problems, and to make clients true partners in the resolution of their matter. 121 It also requires considering a range of socioeconomic contexts for client work, including bias, discrimination, and the connection between legal advocacy and greater movements for social change. 122 Although human lawyers and law students may never perfect these approaches, they can at least conceptualize them and aspire to learn how to pursue them as pedagogical goals. In other words, in our opinion, they can approach experiential education from a client-centered perspective by planning their approach, acting on it, and then reflecting on the results through the CCL lens. This can be done through a range of learning opportunities, skills, and experiences centered around listening, dialog, empathy, and other human relational skills and traits. 123 GenAI systems, on the other hand, teach away from CCL-oriented pedagogy. Rather than following the CCL model of dynamically engaging with clients as partners, in our experience using such tools, they instead imitate the traditional arm's length attorney-client relationship that CCL rejects. For example, when students prompt GenAI for the answer to a legal question, they remain relatively passive beyond stating the prompt. Conversely, we have seen that GenAI outputs are not dynamic interactions; rather they are static responses to each query without inquiry, dialog, feedback, or reflection. In our experience, GenAI systems make no attempt to ask the prompter for more information on how to see the problem from the client's point of view, how to understand the complex nature of the problem, or how to partner with the client on decisions or next steps. Nor do they provide any suggestions for how a lawyer or student attorney might do so. The prompter is also prohibited from gaining any insights into how GenAI systems determine what response to give, including any of the ideologies, logics, data, or sources of authority that the system relied on in doing so. 124 Like the black box of traditional lawyering, experience shows us that GenAI outputs and explanations are given didactically and often dogmatically with zero transparency. Authority is assumed. Certainty and confidence are projected. This is the opposite of the CCL approach. 125 Moreover, as noted above, GenAI systems are optimized to simulate the appearance of human lawyering over the accuracy or adeptness of their outputs. 126 Thus, if a GenAI system calculates that a more traditional lawyering approach (one that frames clients as passive or uninvolved in legal decision-making) sounds more "human" than CCL-driven ones, it will take that approach in framing its output. 127 GenAI systems also fail to engage with issues of subjectivity or systemic bias, such as those raised by Polikoff and Jacobs. Polikoff identifies tensions inherent to human social relationships, identities, and experiences. 128 GenAI systems are incapable of relating to these human experiences or the quandaries that they present. 129 Jacobs' concerns over racial bias are even more problematic for AI systems, which numerous scholars have shown contain inherent and problematic data based on myriad forms of discrimination that no amount of cross-cultural training can correct. 130 Simply put, concepts such as seeing a problem from a certain perspective, understanding the diverse nature of a problem, or creating a true partnership are beyond the model of any GenAI system, making them incompatible with CCL.

## CONCLUSION

*p. 20*
Law schools are not meant to train students to imitate lawyers-they are meant to train students to become lawyers. Although GenAI tools may be able to help students imitate some forms of human lawyering, it is unlikely that they can serve the educational goals of law schools and especially those of law school clinics. Unless and until GenAI evangelists can prove the pedagogical value and ethical integrity of these technologies, we will remain skeptical.

## Footnotes

> . 33. See id. at 3-7. 34. See, e.g., Kathleen L. Mosier, Linda J. Skitka & Susan T. Heers, 37. See infra Parts II.B, III.B.