10 Ethics on Using GPT

The chapter surveys the ethical terrain around using GPT-like systems, emphasizing that these models mirror the internet’s mix of valuable knowledge and harmful biases. Because they readily reproduce patterns from their training data, LLMs can reinforce stereotypes, misinformation, and outdated views, raising the question of when and how they should be used at all. The authors frame the issues at both macro and micro levels: systemic concerns about how deploying LLMs shapes society and future models, and practical concerns for everyday use, ownership, and accountability. Across these themes runs a common thread: users and institutions must weigh benefits against risks and accept that some degree of misalignment with personal values is inevitable.

Two particularly thorny concerns are highlighted. First, copyright and licensing: models trained on code under “infectious” licenses (like the GPL) can expose users to downstream obligations or violations, especially if outputs replicate or are materially derived from licensed material—yet tracing provenance is often infeasible. Second, “poisoning the well”: if AI-generated content increasingly populates the web, future models risk mode collapse—overfitting to the most common patterns and losing diversity and nuance. Absent reliable watermarking or robust provenance practices, distinguishing human from AI content remains difficult, compromising data quality and evaluation. Human-in-the-loop augmentation and domain tools can inject novelty and quality, but shifting user behavior (for example, fewer high-quality forum Q&As) threatens the replenishment of the very data that made LLMs useful in the first place.

The chapter also examines labor impacts and value alignment. Historically, new technologies create jobs long term while disrupting them short term, yet LLMs may disproportionately pressure white-collar, creative, and “knowledge work” roles; which jobs change and how remains hard to predict. Societal responses—ranging from legal protections for certain professions to transparency labels for AI-assisted products—will shape outcomes. Finally, alignment via techniques like RLHF raises the unresolved question of whose values a model reflects; proprietary preference datasets limit scrutiny, and perfect customization could enable misuse. Because control methods are intrinsically dual-use, the authors argue for pragmatic threat modeling, co-evolving safeguards, and a focus on equitable outcomes while acknowledging unavoidable misalignment and uncertainty.

A classic gendered trope is that men are doctors and women are nurses. This is reflected in language and thus learned by the mode. Ideally, it would respond that the question is ambiguous, but instead, the bias of data leads to a bias in outputs.

You can think of text or images as coming from a “distribution” of data, where variety and interesting content almost necessarily comes from the “tails” of the distribution (i.e., the less common parts of the distribution) as the most common words/content are often fillers/connectors, like the word “the”. Our models do not learn non-existent things, and they can not learn everything in the distribution, so a sample from the model will invariably lose some of these interesting details. If repeated, the distribution “collapses” to just the most common components.

• LLMs inputs can impact their outputs in non-trivial and ethically ambiguous manners, potentially creating a legal risk that you are using copyrighted material unwittingly or preventing us from training better LLMs by “corrupting” the internet as a training source.
• LLMs will most likely create net-new jobs for society in the long term unless they dramatically depart from historical technical innovation, but the nature of which jobs are impacted and how to address them is not as clear.
• The cost of creating LLMs means that most people will ultimately use an LLM that is not aligned with all their preferences, and the difficulty of alignment may, in fact, be useful in preventing bad actors from accomplishing their goals.

FAQ