Engelberg Center Corpus

Comments of the Knowing Machines Research Project to the United States Copyright Office Notice of Inquiry on Artificial Intelligence and Copyright

Melodi Dincer, Jason M. Schultz, Christo Buschek
Briefs & Testimony
Knowing Machines Rsch. Project, Comment Letter on Notice of Inquiry on Artificial Intelligence and Copyright, 88 Fed. Reg. 59,942 (Oct. 30, 2023), https://knowingmachines.org/docs/legal_knowing_machines/Knowing_Machines_-_USCO-Comment_10-30-23.pdf
This is an author copy made available for research purposes. Publisher version →
PDF Markdown All publications
p. 3

A portion of the NOI also concerns transparency and recordkeeping for dataset creators and users. Knowing Machines recommends that USCO further study and advocate for funding to support best practices for training dataset creation and maintenance. This includes dataset transparency and recording requirements to facilitate keeping track of the vast, internet-scale datasets that power GenAI, as well as guidelines for preventing the use of depreciated datasets to train these systems.

p. 3

We understand the difficulties of gaining a deep understanding of these training datasets firsthand. We need new investigatory methods to uncover the hidden problems inscribed in machine learning processes. Because dataset creators and AI developers lack standardized ex ante dataset transparency and recordkeeping requirements, we now rely almost exclusively on ex post data investigative tools for research, often unable to identify all the necessary information we need to understand datasets, especially in a copyright context.

p. 3

Although it is challenging, we urge the USCO to support evidence-based research concerning the nature of training datasets and their role in GenAI outputs, minimizing the influence of conjecture in the policy-making process. The NOI asks several important questions regarding training AI models. 17 To adequately answer these questions, however, it helps to have a basic understanding of training datasets and the values that motivate their creators. This is Knowing Machines' core mission. Our research unpacks what training datasets contain, where they come from, and how they are used to train AI systems, including recent GenAI. We combine STS research methodologies with technical data investigations to render these vast datasets more legible. Our expertise comes from years of research into the contents of these datasets, their influence on dominant AI systems, and the motivations of their creators.

I. AI Researchers

p. 3

A dataset can be any set of collected, curated, and interrelated data. The term "dataset" often refers to large collections of data used in computation, especially in machine learning. 18 4

p. 4

But a dataset could have numerous versions all up for use in training AI models, and this is especially so for earlier datasets that have become influential through decades of developers relying on them to train their models. For example, ImageNet is one of the most influential datasets that, since its release in 2009, has been used to "train and evaluate nearly every AI model in the object recognition task." But despite its popularity in the field, "no single 'ImageNet' dataset exists." There are nine versions of the dataset, some designed for specific tasks while others resulted from filtering and moderation meant to improve the perceived fairness of the original dataset. 21 When developers rely on influential datasets like ImageNet that have been used to train numerous AI systems, they are not always specific about which version(s) they used to train their models and why. This is in part because there are no consistent legal requirements for dataset recordkeeping and transparency.

p. 4

Like the data they contain, datasets are often constructed by people and will always reflect the contexts of their creation, including technical constraints of the tools and methods used to construct the dataset, the intentions and perceptions of their creators, and the ability to afford data training, storage, and transmission equipment. Indiscriminate scraping of the internet-without permission or consent from data subjects-is a common technique among dataset creators. 28 The training datasets that constitute "ground truth" for AI developers are less reflections of reality and more "jumble[s] of

p. 5

[data] scraped from whatever various online resources were available." 29 To our knowledge, the technical methods used to scrape data to build such vast, internet-sized datasets do not include mechanisms to determine the copyright status of a piece of data before including it as a candidate. Currently, most dataset creators appear to operate under the default assumption that everything on the public internet is fair game unless told otherwise. How copyright owners or data subjects "tell" dataset creators not to include or to train on their data is also an area of law that needs clarification. Until more recently, many in the machine learning community operated under a "train first, ask questions later" approach advocated by the creators of influential datasets in the field. 30 As more rules and responsibilities around datasets emerge, 25 Crawford and Paglen, supra note 21. The next step in creating training datasets is curating the candidates to determine whether a piece of data will be included or not. In open-access datasets, this is mostly an automated process modeled after a statistical process. In the case of the LAION-5B dataset, which was used to train popular text-to-image GenAI like Stable Diffusion, 31 an automated process examines each candidate image to determine if it is "similar" enough to its caption text to be included, based on an algorithmically-set threshold of similarity. By contrast, there is no published process for algorithmically determining the copyright status of a given image. Thus, while image similarity has been both automated and benchmarked to scale across billions of images, image copyright status has not. Even watermark detection processes, while helpful in flagging some copyright-protected images, have not been developed or benchmarked sufficiently to serve as a reliable proxy for copyright status itself. Whether copyright status processes can be automated and included as part of dataset creation is a key question for policymakers to consider and a key area for researchers to continue investigating. 32

p. 6

Dataset creators wield immense downstream influence in this initial step of creating and curating training datasets by defining the parameters of what will be exposed to AI models in training. For example, the nonprofit that created the Common Crawl dataset determined the parameters of the data included, setting up downstream developers to reinscribe those choices when they use Common Crawl's dataset to train their models without significant filtering. 60%

p. 6

of the training data used to train OpenAI's GPT-3 language model was "derived from raw The NOI also asks questions about the provenance of training data, 34 but these questions are often difficult to answer as AI developers generally choose to withhold information about which datasets they use to train their proprietary models. 35 Their secrecy makes it difficult to know how and where they acquire training datasets, 36 as well as the specific types of data they access to train their models. dataset record in their "license field" appears to be whether a piece of data has a Creative Commons (CC) license and, if so, which type. 42 As a result, while LAION-5B includes a "license" field for each of its 5.85 billion data points, 43 this field lacks any licensing information for 99.7% of the data. And even for the 0.3% of data which are supposedly CC licensed, the results are not verified. 44 One recent investigation traced over 1,800 text datasets used in supervised natural language processing systems by developing tools and standards to trace their provenance. 45 watermarks, we could instead analyze a collection of a few thousand watermarked images and gain the same insights about their prevalence, sources, metadata, and other features.

p. 12

See:Set is not meant to be a centralized, comprehensive tool. It helps build a "foundational information architecture to store and query data," which pushes researchers to readjust their assumptions and ask different questions based on the insights that their searches unearth. 56 We describe See:Set here to suggest that data investigations are a crucial element to building a fact-based regulatory approach to GenAI, because they help bolster or challenge our initial assumptions about the content and provenance of training data. or data describing the actual data, such as license information; and (3) the process that compiles the list of candidates and curates the dataset (as described supra in Section I.). To understand key aspects related to the construction and outputs of AI systems, all three of these areas must be knowable. 60 Transparency mechanisms should also apply to AI developers so they actively disclose which datasets they used in training, which portions of those datasets were used, why those datasets were used, and whether AI developers consulted the datasets' transparency records before use.

Footnotes

This Section responds primarily to Questions 6.1.-3.
See NOI Questions 6.-14.
Datasets Critical Field Guide, supra note 8.
Sasha Luccioni, Knowing Machines Res. Project, 9 Ways to See a Dataset: Investigating Imagenet (2023), https://knowingmachines.org/9-ways-to-see/ImageNet.
Id. See also Kate Crawford and Trevor Paglen, Excavating AI: The Politics of Training Sets for Machine Learning (Sept. 19, 2019), https://excavating.ai/ (describing problematic image labels in ImageNet).
See Datasets Critical Field Guide, supra note 8. See also Bernard Koch et al., Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research (2021), https://openreview.net/forum?id=zNQBIBKJRkd (finding that the majority of widely-used benchmark datasets come from just a handful of elite institutions, including Stanford, Microsoft, Princeton, Google, CUHK, AT&T, NYU, Georgia Tech, Berkeley, and Facebook).
Datasets Critical Field Guide, supra note 8.
Buschek, supra note 8.
Buschek, supra note 8.
22 A dataset can contain specific types of data or a mixture of different types; for example, computer vision datasets include thousands of image or video files, natural language processing datasets contain millions of bytes of text, and increasingly more datasets contain multimodal data.23 Training data refers specifically to a portion of the full dataset used to create a specific machine learning model. In supervised machine learning processes, where algorithms are trained to produce desired outcomes based on labeled data, training data includes results that are like the desired results the model will generate.24 Similarly with GenAI models, training involves presenting data in an iterative process and frequently adjusting its parameters until generated outputs match the
38 Training data thus comes from a mixture of corporate datasets and internet-scale, publicly-scraped datasets. Some consider this to be "data laundering," where private corporations attempt to avoid copyright and data protection issues by training on nonprofit and/or academic datasets, which presumably are more likely exempt from liability.39 Thus it is important to increase transparency in the supply chain of AI systems and to understand the exact role that each actor within the chain plays, what their commercial interest is (if any), and what their specific organizational policies are regarding dataset use for training. Such facts are essential to any copyright analysis, including whether fair use should apply to a given actor's conduct.40 In this mishmash of datasets, issues of provenance become murkier and murkier, often confusing AI researchers as to what legal constraints, if any, apply.41 For example, our data investigations have shown that the only licensing information the creators of the LAION-5B 34 NOI Questions 6.1.-3. (where AI developers acquire training data, to what extent copyrighted works are licensed for use as training data, and to what extent non-copyrighted materials are used to train AI, respectively). 35 See Tiernan Ray, With GPT-4, OpenAI Opts for Secrecy Versus Disclosure, ZDNET (Mar. 16, 2023), https://www.zdnet.com/article/with-gpt-4-openai-opts-for-secrecy-versus-disclosure/. 36 NOI Question 6.1. 37 NOI Questions 6.2.-3. 38 Orr and Crawford, supra note 8. In their article, the authors use the term "dataset creator" to refer to the individuals "who had substantive first-hand experience producing datasets." Id. 39 Andy Baio, AI Data Laundering: How Academic and Nonprofit Researchers Shield Tech Companies from Accountability, WAXY.ORG (Sept. 30, 2022), https://waxy.org/2022/09/ai-data-laundering-how-academic-and-nonprofitresearchers-shield-tech-companies-from-accountability/. 40 NOI Question 8.3 41 NOI Question 6.3.
images; by 2010, the ImageNet database approached 14 million images. That is a 140,000% increase in scale in seven years. Twelve years later in 2022, LAION-5B had over 5 billion webscraped image-text pairs, and by April 2023, CommonPool launched with 12.8 billion webscraped image-text pairs.48 In just under a year, we have witnessed a 256% increase in scale with a starting point already in the billions. The difference between datasets built just a decade ago and now is notable. It would take a person four and a half years to look at each image in ImageNet for ten seconds; it would take the same person 4,000 years to glance at each data point in CommonPool-about fifty lifetimes.49 And this is just at the level of training data. The 52The internet-scale datasets that power GenAI require new data investigation tools and research methods to inform regulatory choices. The unfathomable scale of more recent training datasets should motivate researchers, regulators, and policymakers alike to develop novel investigative methods that make these invisible building blocks of GenAI more visible. Such tools must offer new ways of seeing data by enabling investigators to focus on specific slices of training data, search and probe the data and metadata directly, and draw broader conclusions through statistical analyses. These investigations should go beyond itemizing each piece of data in a dataset, uncovering whose specific worldviews training datasets encode, when that encoding occurs during training, and to what ends.53 To develop targeted and effective policy recommendations, researchers, regulators, and policymakers must be able to explore the allow us to understand the implications of an AI model even when the model itself is unexplainable, or it is impossible to identify whether an AI model was trained on a particular piece of training material.51 See Crawford, supra note 8.52 This Section responds to NOI Question 34.53 See Crawford, supra note 8. information shapes our results. But while designing investigatory tools can be the first step in understanding datasets, focusing too heavily on the tools used can distract researchers from the real goal-gaining insights and facts about what these datasets contain and where those data come from. The tools must follow the research questions, not the other way around. When developing an investigatory tool for training datasets, the research question is relatively straightforward: How can we explore their contents and make them comparable to one another? For copyright researchers, regulators, and policymakers, the research questions might include several NOI questions concerning the copyright status of training data within dominant training datasets in the field. Datasets are built for machine consumption, so investigatory tools must render them human-readable. 55 1. Knowing Machines' See:Set Tool Provides Useful Insights into Creating Investigatory Tools for Training Datasets.trained to interpret reality. From the beginning, a core design choice was to use a search function to parse through dataset contents. See:Set was designed to foster collaborative investigations among Knowing Machines' members, so it includes two features to maximize collaborations. The first feature enables an investigator to export data discovered within See:Set using an open data format (CSV in this case), which allows them to piece together insights from various tools, including See:Set, without requiring See:Set to implement a long list of features that other tools are better suited for. The second feature is having each piece of data individually addressable with a direct link. This link feature is crucial because it allows investigators to "name" a piece of data, and this naming gives collaborating investigators a shared language to discuss individual pieces of data. While data is often viewed as useful in the aggregate, individual data points can also contain useful insights for regulators (for example, if a data point contains racist labeling or clearly infringes on copyright-protected or licensed work). One year into See:Set's development, we added another big feature-curating collections of data.
To aid USCO's consideration of these issues with an eye towards the ultimate report and further recommendations to Congress, Knowing Machines urges USCO to recommend further study and funding to develop specific best practices for training datasets. These best practices should include (1) developing data investigatory tools for use by researchers, regulators, and policymakers to inform regulatory agenda; (2) endorsing standardized, industry-wide dataset transparency and recordkeeping requirements to facilitate data investigations; and (3) adopting guidelines to prevent or minimize the use of depreciated datasets in training AI models.
64 Transparency records should also be made available to the public to promote trust, responsibility, and accountability around AI systems. It is in the public interest that any curious individual can understand the composition of datasets and AI models at the level of data, metadata, and process. While everyone should have the ability to access this knowledge, researchers, journalists, and civil society organizations are often the aggregate stakeholders of this public interest and have direct interests in accessing this information.65 Finally, USCO should consider the prevalence of depreciated data in popular training datasets today. Datasets are frequently depreciated for many reasons, including practical 61 Timnit Gebru et al., Datasheets for Datasets 2 (2018), https://arxiv.org/pdf/1803.09010.pdf.62 Id. at 1.63 Proposal for a67 Id. at 9 (internal quotations omitted).68 Id. at 2.69 Id.70 Id. at 5.71 See id. at 13.