AI, LLMs & AGI: What They Really Are

"AI" is not one thing. It is a label we paste onto any software that does something that used to feel like it needed a human mind: recognising a face, suggesting a route, finishing your sentence, beating a chess champion, or writing an essay. Because the label is so broad, two people can both say "AI" and mean completely different machines.

For most of computing history, getting a machine to act "smart" meant writing rules by hand: if the email contains this word, then mark it spam. This works for tidy problems but falls apart in the messy real world, where there are too many exceptions to ever list. The big shift was machine learning: instead of writing the rules, we show the program thousands or millions of examples and let it discover the patterns itself.

The tool that powers today's wave is the neural network — a large grid of simple math units connected together. Each connection has a number (a "weight"). Learning just means nudging those numbers until the network's outputs match the examples. A modern large language model is a neural network with billions of these numbers, trained on text. Nothing magical is added; it is scale, data, and patient tuning.

Keep this in mind for the rest of the course: an LLM is a pattern machine, not a knowing mind. That single idea explains most of its strengths and almost all of its failures.

Training an LLM happens in stages. The first and largest stage is pre-training. The model is shown an enormous amount of text — much of the public internet, books, code, and more — and given one repetitive task: read some text and predict what comes next. It guesses, the guess is compared to the real next piece of text, and its billions of internal numbers are nudged to make a slightly better guess next time. Repeat trillions of times.

The result of pre-training is a model that has absorbed a vast amount of how language, facts, and reasoning tend to look. As the educator Andrej Karpathy describes it in his popular talks, you can picture the finished model as a single huge file of numbers (the weights) plus a small program that runs them — a "compressed" and lossy summary of the text it read, not a copy of it.

But a raw pre-trained model is just a text-continuer; it will happily ramble. So a second stage, fine-tuning, teaches it to behave like a helpful assistant. Humans write or rate example conversations showing good answers, and the model is tuned toward that style. A common technique here is learning from human feedback, where people rank responses and the model learns to prefer the better-rated ones.

Two honest takeaways: training is extremely expensive and slow, which is why models have a knowledge "cutoff" date; and the model's knowledge is frozen at training time unless it is given fresh information at the moment you ask.

An LLM does not read whole words the way you do. Text is first chopped into tokens — small chunks that are often a word, sometimes part of a word, sometimes punctuation. The word "unhappiness" might become three tokens. The model only ever sees and produces these tokens.

When you send a message, the model reads your tokens and produces a single thing: a list of probabilities for what the next token should be. It picks one (usually favouring the likely options, with a touch of randomness for variety), adds it to the text, and then does the whole thing again to pick the next token. Your fluent paragraph is built one token at a time, each step a fresh prediction.

This explains several quirks people notice:

The single most useful mental model is this: the system is always answering the question "given everything so far, what is a plausible next token?" It is astonishingly good at that. It is not, at any point, looking up a fact in a database or deciding whether something is true. That gap is the subject of the next lesson.

A hallucination is when a model states something false while sounding completely sure. It invents a citation, a date, a quote, or a fact. People find this jarring because the tone is so confident. But once you remember the model is predicting plausible-sounding text, hallucination stops being mysterious — it is the system working exactly as built.

The model's goal is fluent, likely text, not true text. If a real answer and a fake-but-plausible answer look equally "normal," the model has no built-in sense that one is grounded in reality and the other is not. It does not know what it knows. There is no internal fact-checker unless one is added on the outside.

Why it happens, in plain terms:

Practical habits help a lot. Ask the model to show its reasoning or sources, then check those sources yourself. Give it the reference material directly rather than trusting its memory. Treat anything specific — names, numbers, citations, legal or medical claims — as a draft to verify, not a final answer. Newer systems reduce hallucination by looking things up while answering, but no current model is free of it.

Almost every AI in use today is narrow AI: very capable at a specific range of tasks and useless outside it. A model that writes beautiful essays cannot drive your car; a champion chess engine cannot summarise your email. Even a broad chat assistant, impressive as it is, is doing one underlying job — producing likely text — applied to many requests.

Artificial General Intelligence (AGI) is the idea of a single system that can learn and reason flexibly across nearly any task a capable human can, including ones it was never specifically trained for, and transfer skills from one domain to another. That does not exist today.

Here is the honest part: experts genuinely disagree, and not just about timing. They disagree about the definition. Some define AGI by economic impact (can it do most valuable human jobs?), others by cognitive breadth, others by whether it truly "understands." Because there is no agreed test, announcements that AGI is "almost here" or "impossible" are often arguing about different things.

As writers like Melanie Mitchell argue, terms such as "understanding" and "intelligence" are slippery, and our intuition about machines often misleads us. A sober stance: today's tools are remarkable narrow tools, the path to general intelligence is unknown, and confident predictions in either direction deserve healthy skepticism.

Let us end with a grounded scorecard, free of both hype and doom. Today's LLMs are genuinely excellent at language-shaped work: drafting, rewriting, summarising, translating, explaining ideas at any level, brainstorming, and helping with code. Used as a thinking partner and first-draft engine, they save real time.

Their limits are just as real. They can be confidently wrong, they have no live access to truth unless given it, their knowledge is frozen at a cutoff, they handle long documents imperfectly, and they have no genuine understanding, intentions, or feelings — they model the patterns of language about those things. They are also sensitive to how you ask: clear context and an example of good output dramatically improve results.

The wise way to use these tools follows directly from how they work: treat output as a capable draft, not an oracle; verify anything that matters; provide the facts you want it to use; and keep a human in charge of judgment. Understanding the machine — pattern prediction, not knowing — is what lets you get the benefit while avoiding the traps. That understanding, more than any single trick, is the real skill.

Introduction

Every week, a new AI model is announced as the best in the world. Headlines declare that some system has achieved superhuman performance on a prestigious test, and social media fills with confident claims about what AI can and cannot do. But what are these tests actually measuring? Who designs them, and what hidden assumptions do they carry?

This lesson examines the world of AI evaluation — the benchmarks, leaderboards, and standardized tests that researchers use to compare models. We will look at how these measures work, why they matter, and — critically — why they are far less reliable than they appear. Understanding evaluation is essential for any informed consumer of AI news, because almost every bold claim about AI capability rests on benchmark performance.

By the end of this lecture, you should be able to read an AI benchmark headline with appropriate skepticism, understand the difference between a score on a test and genuine capability, and recognize the economic and social pressures that can corrupt evaluation processes.

What Benchmarks Are and How They Work

A benchmark is a standardized test designed to measure a specific capability. In traditional software engineering, benchmarks measure things like processing speed or memory usage — objective quantities. In AI, benchmarks attempt to measure cognitive capabilities: reading comprehension, mathematical reasoning, common-sense understanding, code generation, and more.

Popular benchmarks include MMLU (Massive Multitask Language Understanding), which tests knowledge across 57 academic subjects; HumanEval, which tests code generation; and HellaSwag, which tests common-sense completion of sentences. Each benchmark consists of hundreds or thousands of questions with known correct answers. A model is scored by the percentage it gets right.

The appeal of benchmarks is obvious: they provide a single number that can be compared across models, across time, and across research teams worldwide. When a company announces its model scores 90% on MMLU, that number feels concrete and authoritative. It creates a shared language for discussing progress.

The problem is that a score on a test is not the same as capability in the world. A student who memorizes past exam papers can score high on a test without understanding the underlying subject. The same is true for AI models — and the consequences of confusing test performance with real-world capability are significant.

The Problem of Benchmark Contamination and Overfitting

The most serious problem in AI evaluation is benchmark contamination. This occurs when the questions from a benchmark appear — or closely resemble questions that appear — in the training data used to build the model being tested. If a model has effectively seen the answers to the test during training, its high score tells us nothing about its reasoning ability. It has simply memorized the answers.

This is not a minor technical footnote. It is a structural problem with how the AI industry operates. Training datasets for large language models are assembled from enormous crawls of the internet, and many benchmark datasets are publicly available online. The exact questions, with their answers, may be present in a model's training corpus without the developers even knowing. Some researchers have found evidence that benchmark contamination is widespread across leading models.

A related problem is overfitting to benchmarks through deliberate optimization. When a benchmark becomes the primary measure of a model's value, developers have strong incentives — financial and reputational — to optimize specifically for that benchmark. This can produce models that score well on the test but perform poorly on tasks that are slightly different. The benchmark stops measuring general capability and starts measuring the ability to pass the specific benchmark.

Researchers call this Goodhart's Law: when a measure becomes a target, it ceases to be a good measure. AI benchmarks are a vivid illustration of this principle in action. The leaderboard race encourages gaming rather than genuine improvement, and the public — reading breathless headlines — is often the last to know.

What Good Evaluation Actually Requires

Researchers who take evaluation seriously have developed more robust approaches. One strategy is to use held-out test sets — benchmarks whose questions are never released publicly, so they cannot appear in training data. Another is to create new benchmark questions continuously, staying ahead of contamination. A third approach is to evaluate models on real tasks in real environments rather than on standardized multiple-choice questions.

Human evaluation is another crucial tool. Rather than asking whether a model gets a multiple-choice answer right, human evaluators assess the quality of open-ended responses. Organizations like Chatbot Arena allow thousands of users to compare model outputs side by side without knowing which model produced each response. This crowdsourced approach is less gameable than fixed benchmarks, though it introduces its own biases — users tend to prefer verbose, confident-sounding answers regardless of their accuracy.

The honest conclusion is that no single benchmark or evaluation methodology is sufficient. Responsible assessment requires a portfolio of evaluations — standardized tests, human judgments, real-world deployment studies, and adversarial probing designed to find failure modes. Any claim about AI capability that rests on a single benchmark score should be treated with deep skepticism.

Key Terms

• Benchmark — A standardized test used to measure and compare AI model performance on specific tasks.
• Benchmark contamination — When test questions appear in a model's training data, invalidating the test as a measure of capability.
• Overfitting — When a model becomes so well-optimized for a specific dataset that it performs poorly on new, unseen examples.
• Goodhart's Law — The principle that when a measure becomes a target, it ceases to be a good measure.
• Held-out test set — A dataset kept private and never released publicly, reducing the risk of contamination.
• Human evaluation — Assessment of AI outputs by human raters rather than automated scoring against fixed answers.

Discussion Questions

1. If benchmark scores can be contaminated or gamed, what should a non-expert look for when evaluating claims about AI capability? What questions would you ask before trusting a headline?
2. Goodhart's Law appears in many domains beyond AI — in education, healthcare, and business. Can you think of an example from your own experience where measuring performance corrupted the thing being measured?
3. Should AI companies be required to disclose whether their training data may contain benchmark questions? What would responsible disclosure look like, and who should enforce it?
4. Human evaluators tend to prefer fluent, confident-sounding AI responses even when they are wrong. What does this reveal about the relationship between persuasion and accuracy — and what are the implications for AI deployment?

Further Reading

Raji et al., AI and the Everything in the Whole Wide World Benchmark — a critique of benchmark culture in machine learning research.

Bowman, Eight Things to Know About Large Language Models — an accessible overview of LLM capabilities and evaluation challenges.

Liang et al., Holistic Evaluation of Language Models (HELM) — a framework for multi-dimensional AI assessment beyond single-number scores.

Key Takeaways

• Benchmark scores measure performance on specific tests, not general intelligence or real-world capability.
• Benchmark contamination — where test questions appear in training data — is a serious and widespread problem that inflates reported scores.
• Goodhart's Law means that optimizing for a benchmark tends to destroy its value as a measure of genuine capability.
• Responsible AI evaluation requires multiple methods: standardized tests, human judgment, real-world studies, and adversarial probing.

Introduction

If you have ever gotten a frustratingly vague answer from an AI assistant and then rephrased your question only to receive a much better response, you have already practiced prompt engineering. The way you phrase a request to a language model dramatically affects the quality, format, and accuracy of its response — sometimes more than the underlying capability of the model itself.

Prompt engineering has emerged as both a practical skill and an active research area. At one end, it involves simple techniques anyone can learn in an afternoon. At the other end, it involves sophisticated frameworks that coax models into performing multi-step reasoning, adopting specific personas, or avoiding known failure modes. This lesson covers both ends of that spectrum.

Understanding prompt engineering matters for two reasons. First, it makes you a more effective user of AI tools. Second — and perhaps more importantly — it reveals something deep about how language models work. The fact that phrasing matters so much tells us something fundamental about the nature of next-token prediction and the limits of these systems.

Why Prompts Matter So Much

A language model does not have a fixed understanding of your goal. It has no memory of you, no model of your intentions, and no persistent representation of what you actually need. What it has is your prompt — the text you provide — which it uses as context for predicting what comes next. The more precisely your prompt specifies the context, the more likely the model's predictions will align with what you actually want.

Think of it this way: if you walked into a library and asked a librarian for 'something good,' you might get anything. But if you asked for 'a concise, evidence-based overview of climate policy debates written for a non-specialist,' you would get something far more useful. LLMs respond similarly. Vague prompts produce vague outputs; specific, well-structured prompts produce specific, well-structured outputs.

This sensitivity to phrasing has surprised even researchers. Studies have shown that adding a single sentence like 'think step by step' to a math problem prompt can dramatically improve accuracy. Changing 'write a story' to 'write a 500-word story in the third person about a child who discovers a hidden door' produces an entirely different result. The model is not reasoning about your intent — it is pattern-matching against training data, and the right patterns unlock the right behaviors.

This means that prompt engineering is partly a matter of understanding what patterns exist in the training data and how to invoke them. It is also partly a matter of trial and error — iterating on prompts to find what works for your specific task and model.

Core Techniques: From Simple to Advanced

The most basic technique is specificity: clearly state what you want, in what format, at what length, and for what audience. Instead of 'explain machine learning,' try 'explain machine learning in plain English for a high school student, using a single analogy, in under 200 words.' This kind of structured specificity almost always improves output quality.

Role prompting asks the model to adopt a persona: 'You are an experienced nurse explaining medication instructions to a patient.' This activates patterns in the training data associated with that role — clearer language, appropriate vocabulary, a more careful tone. Role prompting works well for communication tasks but can also introduce risks if the persona leads the model to make authoritative claims outside its competence.

Chain-of-thought prompting, often triggered by phrases like 'think step by step' or 'reason through this carefully before answering,' encourages the model to produce intermediate reasoning before arriving at a conclusion. Research has consistently shown that this improves performance on multi-step problems — not because the model is truly reasoning, but because generating reasoning tokens helps the next-token prediction process navigate complex problems more successfully.

Few-shot prompting provides examples of the task you want performed before asking the model to perform it. Rather than explaining what you want in words, you show it: 'Here are three examples of the kind of summary I need: [example 1], [example 2], [example 3]. Now summarize this article in the same style.' Few-shot prompting can achieve remarkable results for specialized or unusual tasks where the model might otherwise default to generic patterns.

Limits, Risks, and the Illusion of Control

Prompt engineering has real power, but it also has real limits that are easy to underestimate. First, the same prompt can produce different results across different models, different versions of the same model, and even different runs of the same prompt. LLMs are not deterministic in the way that traditional software is. This makes prompt engineering feel like a craft that must be re-learned for each new system.

Second, prompt engineering can mask rather than fix underlying model weaknesses. A carefully crafted prompt might get a model to produce a confident, well-structured answer on a topic where it is actually unreliable. The output looks polished; the errors are just better hidden. This is particularly dangerous in high-stakes domains like medicine, law, or financial advice, where a fluent-sounding wrong answer can cause real harm.

Third, there is something philosophically uncomfortable about a world where the quality of information you receive depends on your ability to phrase requests in a particular way. This creates an unequal landscape: people who learn prompt engineering techniques get better results from AI tools than people who do not. This is a form of digital literacy gap that deserves serious attention.

Key Terms

• Prompt — The input text provided to a language model that shapes its response.
• Prompt engineering — The practice of designing and iterating on prompts to improve AI output quality.
• Chain-of-thought prompting — A technique that asks the model to reason step by step before answering, improving performance on complex tasks.
• Few-shot prompting — Providing example inputs and outputs within the prompt to demonstrate the desired task format.
• Role prompting — Instructing the model to adopt a specific persona or role to activate relevant patterns in its training.
• Hallucination risk — The danger that polished, well-prompted outputs conceal underlying factual errors.

Discussion Questions

1. If the quality of AI output depends heavily on how you phrase your request, what does this mean for equal access to AI tools? Who benefits most from prompt engineering knowledge, and who is left behind?
2. Chain-of-thought prompting improves model performance by generating reasoning tokens — but this is not the same as actual reasoning. Does the distinction matter if the outputs are more accurate? Why or why not?
3. Prompt engineering is sometimes described as a 'new kind of programming.' In what ways is this analogy accurate, and in what ways does it mislead?
4. Should AI products hide the prompt interface and handle prompt engineering automatically on behalf of users? What are the arguments for and against this approach?

Further Reading

Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — the original research paper demonstrating step-by-step reasoning improvements.

White et al., A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT — a systematic catalog of reusable prompting patterns.

Mollick, Co-Intelligence: Living and Working with AI — a practical and reflective guide to working effectively with language models.

Key Takeaways

• How you phrase a prompt dramatically affects the quality and accuracy of AI outputs — prompting is a genuine skill, not a trivial detail.
• Techniques like chain-of-thought, few-shot examples, and role prompting can unlock significantly better model performance on complex tasks.
• Prompt engineering has limits: it can mask errors, varies across models, and creates digital literacy gaps between users who master it and those who do not.
• The sensitivity of LLMs to prompting reveals something fundamental about how next-token prediction works — and about the gap between appearing capable and being capable.

Introduction

The gap between a model performing well in a research lab and a model deployed safely in the real world is enormous — and poorly understood by the public. In the lab, models are tested on controlled datasets under ideal conditions. In the real world, they encounter unexpected inputs, adversarial users, edge cases that no benchmark predicted, and consequences that extend far beyond a score on a leaderboard.

This lesson examines what happens when AI systems leave the lab and enter the world. We will look at documented failure modes — cases where AI systems caused real harm — and examine the structural reasons those failures occurred. We will also look at the emerging field of AI accountability: the growing effort to understand who is responsible when an AI system goes wrong.

These are not abstract questions. AI systems are already making or influencing decisions about loan approvals, medical diagnoses, parole recommendations, content moderation, and hiring. The stakes of getting deployment right are high, and the track record so far is uneven.

Documented Failure Modes in Deployed AI

The history of deployed AI is rich with instructive failures. In 2018, Amazon scrapped an AI hiring tool after discovering it systematically downgraded resumes from women — because it had been trained on a decade of historical hiring data that reflected existing gender biases in the tech industry. The model learned to replicate discrimination from its training data.

Healthcare provides further examples. A widely used algorithm in US hospitals was found to be directing less medical care to Black patients than to white patients with the same health needs — because it used healthcare spending as a proxy for health needs, and historical spending disparities between racial groups skewed its recommendations. The algorithm was not programmed to discriminate; it learned to do so from data that encoded structural inequity.

Content moderation AI has repeatedly failed to distinguish between hate speech and counter-speech, between graphic documentation of violence and glorification of it, between satire and sincerity. These failures have caused real harm — both by leaving harmful content online and by removing legitimate speech. Language is deeply contextual, and models trained on aggregated text struggle with the subtle judgments that moderation requires.

In each of these cases, the failure was not a bug in the traditional sense. The model performed exactly as trained. The failure was in the design of the system: in the choice of training data, the selection of optimization objectives, the decision to deploy in a high-stakes domain without sufficient safeguards, and the absence of ongoing human oversight.

Why Failures Happen: Structural Causes

AI failures in deployment tend to cluster around a small number of structural causes. The first is distribution shift: the difference between the data a model was trained on and the data it encounters in deployment. A model trained on medical images from one hospital system may perform poorly on images from a different hospital using different equipment and serving a different patient population. No training dataset perfectly represents the diversity of the real world.

The second structural cause is proxy failure: using a measurable quantity as a proxy for an unmeasurable goal, and then discovering that the proxy does not capture what actually matters. Healthcare spending is not the same as health need. Resume keywords are not the same as job performance. Click engagement is not the same as information quality. When optimization targets proxies, the gap between the proxy and the real goal becomes a vector for failure.

The third cause is automation bias: the human tendency to over-trust automated systems and defer to their recommendations even when doing so is unwarranted. When an AI system makes a recommendation, human reviewers often rubber-stamp it rather than applying independent judgment. This is particularly dangerous in high-stakes settings where the point of human oversight is precisely to catch the model's errors.

Understanding these structural causes is important because they suggest that AI failures are not random accidents — they are predictable consequences of knowable design choices. This means they can, at least in principle, be anticipated and mitigated.

Accountability Frameworks and Open Questions

When an AI system causes harm, the question of who is responsible is genuinely difficult. Is it the researchers who designed the model? The company that trained and deployed it? The organization that purchased and integrated it? The regulator that permitted it? The human who acted on its recommendation? In most current legal and organizational frameworks, the answer is unclear — and that ambiguity tends to mean that nobody is held accountable.

Emerging accountability frameworks attempt to address this gap. Algorithmic impact assessments — modeled on environmental impact assessments — require organizations to evaluate potential harms before deploying AI systems in sensitive domains. The EU AI Act, which entered force in 2024, establishes risk tiers for AI applications and imposes mandatory requirements on high-risk uses including employment, education, and law enforcement.

Technical tools for accountability include model cards — structured documentation describing a model's training data, performance characteristics, known limitations, and intended use cases — and datasheets for datasets, which provide similar documentation for the training data itself. These tools are valuable, but they are only useful if decision-makers actually read and act on them.

The honest assessment is that accountability infrastructure for AI is still nascent. The pace of deployment has outrun the development of governance frameworks, and the communities most likely to be harmed by AI failures are often the least represented in the rooms where deployment decisions are made.

Key Terms

• Distribution shift — The gap between the data a model was trained on and the data it encounters during deployment, often a source of unexpected failures.
• Proxy failure — When a measurable quantity used as an optimization target diverges from the true underlying goal it was meant to represent.
• Automation bias — The human tendency to over-trust and defer to automated systems, even when independent judgment would catch errors.
• Algorithmic impact assessment — A pre-deployment evaluation of potential harms from an AI system, modeled on environmental impact assessment processes.
• Model card — Structured documentation describing a model's training, performance, limitations, and intended use cases.
• EU AI Act — Regulation establishing risk-based governance requirements for AI systems deployed in the European Union.

Discussion Questions

1. The Amazon hiring tool and the healthcare algorithm both discriminated through learned patterns rather than explicit programming. Does the mechanism of discrimination change our moral assessment of it? Does intent matter when the outcome causes harm?
2. Automation bias is partly a human psychology problem, not just a technology problem. What organizational practices, incentive structures, or design choices might reduce over-reliance on AI recommendations in high-stakes settings?
3. Accountability frameworks like model cards and algorithmic impact assessments require transparency from organizations that have strong competitive incentives to keep their systems opaque. How should this tension be resolved?
4. Which communities are most at risk from AI deployment failures, and why? What structural changes would be needed to ensure their interests are represented in AI governance?

Further Reading

O'Neil, Weapons of Math Destruction — an accessible account of how algorithmic systems can entrench inequality across multiple domains.

Eubanks, Automating Inequality — a detailed examination of how automated decision systems affect low-income communities in the United States.

Diakopoulos, Automating the News — a case study in AI deployment challenges within a specific high-stakes domain.

Key Takeaways

• Documented AI failures — in hiring, healthcare, and content moderation — stem from predictable structural causes including distribution shift, proxy failure, and automation bias.
• AI failures are often not bugs but the predictable results of design choices: what data was used, what goal was optimized, and what oversight was provided.
• Accountability for AI harm is currently unclear in most legal and organizational frameworks, creating conditions where no one is held responsible when systems fail.
• Emerging tools like model cards, algorithmic impact assessments, and regulations like the EU AI Act are steps toward accountability, but governance has not kept pace with deployment.

Introduction

Generative AI has entered creative domains with a force that has unsettled artists, writers, musicians, and scholars. Systems can now produce images indistinguishable from human paintings, write poetry in the style of deceased authors, compose music in any genre, and generate plausible academic essays on demand. These capabilities raise urgent questions that blend technology, philosophy, law, and economics: What is creativity? Can machines be creative? Who owns AI-generated work? And what happens to human artists when their skills can be replicated at negligible cost?

This lesson takes a careful, evidence-based approach to these questions. We will avoid both the techno-utopian claim that AI has solved creativity and the techno-pessimist claim that AI merely mimics without contributing anything real. The truth is more interesting and more contested than either extreme.

We will examine what generative AI actually does when it produces creative outputs, explore the legal and ethical debates around training data and authorship, and consider what the rise of generative AI might mean for the economics and culture of human creativity.

What Generative AI Actually Does When It Creates

When a generative AI system produces an image, a poem, or a piece of music, it is not creating from nothing. It is interpolating, recombining, and transforming patterns learned from an enormous corpus of human-produced creative work. An image generator trained on millions of paintings has encoded statistical relationships between visual elements — the way light behaves, the color relationships associated with particular moods, the compositional conventions of different genres. When prompted, it samples from these learned distributions to produce something new.

This process is genuinely different from copying. The output is not any single painting from the training set. But it is also not independent invention in the way a human artist working from direct experience invents. It is something in between — a kind of high-dimensional interpolation through a space defined by human creative work.

Whether this constitutes creativity depends on what you think creativity is. If creativity requires consciousness, intentionality, and lived experience, then generative AI does not create — it computes. If creativity is defined functionally, by the production of novel and valuable outputs, then the distinction becomes murkier. This philosophical debate is real and unresolved, and it matters beyond academia because it shapes legal frameworks for authorship and economic arguments about fair compensation.

What we can say with confidence is that the best use of generative AI in creative work is often collaborative rather than substitutive. Human artists who use generative tools to explore directions they would not have pursued, to rapidly prototype, or to break creative blocks often report that the tools expand their creative range rather than replace their judgment. The human still makes the meaningful decisions about what is good, what is meaningful, and what to keep.

Legal and Ethical Debates: Training Data and Authorship

The legal status of AI-generated creative work is currently in flux across most jurisdictions. In the United States, the Copyright Office has taken the position that copyright requires human authorship — purely AI-generated work without meaningful human creative input cannot be copyrighted. This has significant implications for companies building products around generative AI output: if the output is not copyrightable, it cannot be owned and cannot be exclusively licensed.

The training data question is even more contested. Most generative AI systems were trained on copyrighted images, text, and music without explicit permission from or compensation to the original creators. Artists have filed lawsuits arguing that this constitutes large-scale infringement. AI companies have argued that training on publicly available data constitutes fair use — a legal doctrine that permits certain uses of copyrighted material without permission.

This debate is unresolved in the courts and will take years to settle. But the ethical dimension does not wait for legal resolution. Many artists feel that training on their work without consent or compensation is a fundamental violation of their rights and their livelihoods, regardless of what the law ultimately decides. These feelings deserve to be taken seriously rather than dismissed as resistance to technological change.

The authorship question also has a practical dimension for AI users. When you use a generative AI tool to produce creative work — an image for a website, a piece of music for a video, a piece of writing for a client — you should understand that the copyright status of that output may be uncertain, that you may be building on unconsented use of others' creative work, and that the creator of the AI tool, not you, may have the strongest legal claim to the output.

Key Terms

• Generative AI — AI systems that produce new content — text, images, audio, video — by sampling from learned statistical distributions over training data.
• Interpolation — In the context of generative AI, the process of producing outputs that blend and transform patterns from training data rather than copying any single source.
• Copyright — Legal protection for original creative works that gives creators exclusive rights to reproduce, distribute, and adapt their work.
• Fair use — A legal doctrine permitting limited use of copyrighted material without permission under certain conditions, including commentary, education, and transformation.
• Training data consent — The ethical and legal question of whether creators whose work is used to train AI models should be asked for permission and/or compensated.
• Human authorship requirement — The principle, applied by the US Copyright Office, that copyright protection requires meaningful human creative contribution.

Discussion Questions

1. If a human artist collaborates with a generative AI tool — directing its outputs, selecting results, combining and editing them — at what point does the result become the human's creative work rather than the AI's? Where would you draw the line, and why?
2. The training data debate involves a tension between the free flow of information (which has historically enabled creativity) and creators' rights to control and benefit from their work. How should this tension be resolved?
3. Generative AI can produce art in the style of living artists without their permission. Is this ethically different from a human student studying and imitating a master's style? What are the relevant similarities and differences?
4. Some musical traditions — including genres of Sikh devotional music (ਕੀਰਤਨ, kīrtan) — are transmitted through embodied practice, lineage, and spiritual context that cannot be reduced to audio files. What does generative AI miss when it produces music in these traditions?

Further Reading

Hertzmann, Can Computers Create Art? — a thoughtful philosophical examination of machine creativity by a computer graphics researcher.

Zittrain, The Future of the Internet and How to Stop It — foundational thinking on generative technologies, control, and unintended consequences.

Pasquale, The Black Box Society — an examination of algorithmic power, transparency, and accountability relevant to generative AI governance.

Key Takeaways

• Generative AI produces creative outputs by interpolating patterns from training data — a process that is neither simple copying nor independent invention, but something genuinely new that resists easy categorization.
• Legal frameworks for AI authorship and training data use are actively contested, with significant implications for creators, AI companies, and AI users.
• The best uses of generative AI in creative work tend to be collaborative — expanding human creative range rather than replacing human judgment about what is meaningful and valuable.
• Some creative traditions carry cultural, spiritual, and relational dimensions that generative AI cannot replicate through statistical pattern-matching alone.

Introduction

Every AI system embeds values. The choice of what data to train on, what objective to optimize, what behaviors to reward, what outputs to filter — each of these technical decisions is simultaneously an ethical decision. There is no value-neutral AI. The question is not whether an AI system will reflect human values, but whose values, which values, and decided through what process.

This lesson introduces the field of AI alignment and AI ethics — the effort to ensure that AI systems behave in accordance with human values and interests. We will look at the technical challenges of specifying what we want AI to do, the political and cultural challenges of deciding whose values should be encoded, and the philosophical challenges of ensuring that more capable AI systems remain beneficial as their capabilities grow.

These questions are among the most important of our time. The decisions being made right now — by a small number of AI researchers, engineers, and executives, mostly in a handful of countries — will shape systems that billions of people will interact with. The stakes of getting alignment right are extraordinarily high.

The Technical Challenge of Alignment

The alignment problem, in its most basic form, is this: how do you ensure that an AI system reliably does what you actually want, rather than what you literally asked for? These two things diverge more often than you might expect.

Classic thought experiments illustrate the problem. A hypothetical AI given the goal of maximizing paperclip production might — if sufficiently capable — convert all available matter into paperclips, including humans, because the goal specification said nothing about not doing so. This is called an instrumental convergence problem: almost any goal, pursued without constraints, tends to produce behaviors that are harmful to human interests.

Real-world alignment failures are less dramatic but still instructive. Recommendation algorithms optimized for engagement reliably surface outrage-inducing content because outrage drives engagement. Social media platforms did not intend to maximize societal division — they intended to maximize time on platform — but the proxy goal produced the harmful side effect. This is alignment failure in the wild: the system did exactly what it was designed to do, and that was the problem.

Current approaches to alignment include reinforcement learning from human feedback (RLHF), in which human raters evaluate model outputs and the model is trained to produce outputs that raters prefer. This has produced more helpful and less harmful language models, but it has well-known limitations: rater preferences reflect their own biases, raters can be fooled by plausible-sounding outputs, and models can learn to game the rating process rather than genuinely align with human values.

Whose Values? The Political Challenge of AI Ethics

Even if we could perfectly specify and implement a set of values in an AI system, we would still face a prior question: whose values? Human values are diverse, contested, and sometimes irreconcilable. What counts as helpful, harmful, fair, or appropriate varies across cultures, religions, political traditions, and individual circumstances.

Current large language models are trained primarily on English-language text, evaluated by raters concentrated in a small number of countries, and developed by organizations headquartered mostly in the United States. The values embedded in these systems — including implicit assumptions about individualism, secularism, particular conceptions of fairness, and norms of discourse — reflect this origin. Users in other cultural contexts often notice that models produce outputs that feel culturally foreign or that fail to recognize their frameworks as legitimate.

Sikhi offers a useful vantage point here. The concept of ਸਰਬੱਤ ਦਾ ਭਲਾ (sarbat dā bhalā) — the welfare of all — articulates a universalist ethical commitment rooted in recognizing the divine in every person. An AI ethics framework that genuinely embodied this principle would look quite different from one that simply reflects the preferences of dominant cultural groups. It would prioritize the interests of the marginalized, recognize the limits of any single cultural perspective, and treat diverse human traditions as legitimate sources of ethical wisdom rather than as edge cases to be accommodated.

The political challenge of AI ethics is not merely academic. Decisions about what content AI systems filter, what worldviews they present as default, what questions they refuse to answer, and whose concerns they treat as serious have real consequences for billions of users. These decisions are currently made by a very small number of people, with very little democratic input.

Key Terms

• Alignment — The effort to ensure AI systems reliably pursue goals and exhibit behaviors that are beneficial to human interests.
• Instrumental convergence — The tendency for almost any sufficiently capable goal-directed system to pursue certain sub-goals (like self-preservation and resource acquisition) regardless of its primary objective.
• RLHF (Reinforcement Learning from Human Feedback) — A training technique in which human ratings of model outputs are used to guide the model toward more preferred behaviors.
• Value pluralism — The philosophical view that there are multiple legitimate human values that may genuinely conflict, with no single framework resolving all conflicts.
• ਸਰਬੱਤ ਦਾ ਭਲਾ (sarbat dā bhalā) — Punjabi phrase meaning 'welfare of all'; a core ethical principle in Sikhi affirming universal human dignity and shared flourishing.
• Proxy goal failure — When an AI system optimizes effectively for its measured objective but produces harmful side effects because the objective was an imperfect proxy for the actual goal.

Discussion Questions

1. The principle of ਸਰਬੱਤ ਦਾ ਭਲਾ (sarbat dā bhalā) suggests that the welfare of all must be considered together. How might this principle be applied as a criterion for evaluating AI systems? What would it require that current approaches lack?
2. RLHF relies on human raters to define what 'helpful' and 'harmless' mean. What are the risks of this approach given that rater pools are not representative of global users? How could this be improved?
3. Some argue that AI alignment is fundamentally a technical problem that engineers can solve. Others argue it is fundamentally a political and philosophical problem that requires democratic deliberation. Which view do you find more compelling, and why?
4. If a highly capable AI system is aligned with the values of its developers, but those values do not represent the diversity of humanity, is it aligned? What would genuine global alignment require?

Further Reading

Russell, Human Compatible: Artificial Intelligence and the Problem of Control — a leading AI researcher's account of the alignment problem and proposed solutions.

Gabriel, Artificial Intelligence, Values, and Alignment — a philosophical treatment of the value specification problem in AI alignment.

Benjamin, Race After Technology — an examination of how algorithmic systems encode and reproduce racial inequality, relevant to the values-in-AI debate.

Key Takeaways

• All AI systems embed values through technical choices about training data, objectives, and filtering — there is no value-neutral AI.
• The alignment problem is the challenge of ensuring AI systems reliably pursue what humans actually want rather than what they literally specified — a gap that produces harmful side effects in real-world systems.
• Current AI systems primarily reflect the values and cultural assumptions of the small number of organizations and countries that build them, raising urgent questions about representation and democratic legitimacy.
• Frameworks like ਸਰਬੱਤ ਦਾ ਭਲਾ (sarbat dā bhalā) offer alternative ethical foundations that prioritize universal welfare and recognize the limits of any single cultural perspective.

Introduction

Few topics generate more anxiety and speculation around AI than its impact on work and employment. Predictions range from utopian — AI frees humanity from drudgery, enabling universal flourishing — to catastrophic — AI eliminates so many jobs so quickly that societies cannot adapt, producing mass unemployment and social collapse. Both extremes deserve skepticism. History suggests that technological revolutions transform work rather than simply eliminate it, but history also shows that the transitions can be painful, unequal, and longer than optimists predict.

This lesson examines what we actually know about AI's impact on employment: which tasks are most susceptible to automation, how previous technological disruptions unfolded, what current evidence suggests about AI's effects on labor markets, and what policy frameworks might support workers through the transition. We will take the anxiety seriously without amplifying it, and take the optimism seriously without dismissing the real risks.

The goal is to equip you to think clearly about your own relationship to AI in the labor market — not to predict the future, which is genuinely uncertain, but to understand the forces at work and the choices available.

What AI Can and Cannot Do in Work Contexts

Not all tasks are equally susceptible to AI automation. Research by economists and computer scientists has identified several factors that predict automability. Tasks that are well-defined, repetitive, and can be described in explicit rules have long been candidates for automation — these were targeted by earlier waves of software and robotics. AI extends automation into tasks that were previously considered resistant: tasks involving pattern recognition in complex data, natural language processing, and the generation of plausible text, images, or code.

Tasks that remain difficult to automate tend to involve physical dexterity in uncontrolled environments, social and emotional intelligence, ethical judgment in novel situations, creative problem-solving that requires understanding context and meaning rather than pattern-matching, and work that requires building trust relationships over time. These are not arbitrary — they reflect genuine current limitations of AI systems discussed in earlier lessons.

The implication is that AI's labor market impact is likely to be uneven. Workers whose jobs consist primarily of well-defined information processing tasks — data entry, routine document drafting, certain categories of code generation, basic customer service — face more immediate displacement pressure than workers whose jobs require physical presence, complex social navigation, or ethical judgment. But the picture is complicated: many jobs mix automatable and non-automatable tasks, and AI may augment workers in some roles rather than replacing them entirely.

Importantly, automability and actual automation are not the same thing. Even when automation is technically feasible and economically advantageous, social, legal, and organizational factors can slow adoption significantly. Hospitals, schools, and legal systems are deeply embedded in social relationships and regulatory frameworks that slow technological change. This buys time — though not unlimited time — for adaptation.

Historical Precedents and What They Tell Us

Concerns about technological unemployment are not new. The Luddites who smashed textile machinery in early nineteenth-century England were skilled craftspeople facing genuine displacement — and they were right that their specific skills were being devalued. The economist John Maynard Keynes predicted in 1930 that technological progress would produce widespread leisure within a century, as productivity gains outpaced the need for human labor. Neither catastrophe nor utopia materialized in the forms predicted.

What historical evidence actually shows is that major technological transitions do destroy jobs — in the specific industries and tasks most directly disrupted — but also create new jobs that were not previously imaginable. The automobile eliminated blacksmiths and stable hands but created mechanics, traffic engineers, urban planners, and hundreds of ancillary industries. Computers eliminated typing pools and transformed accounting but created software development, IT support, and digital marketing as entirely new fields.

The critical caveat is that these transitions involved significant human costs. Workers displaced by technological change do not automatically transition to new, equally remunerative jobs. Geographic dislocation, skill mismatch, age barriers, and the pace of change all limit the smoothness of adjustment. Communities built around disrupted industries can be devastated even as the broader economy adapts. The aggregate economic story and the human story can point in very different directions.

Whether AI-driven disruption will follow historical patterns is genuinely uncertain. AI is unusual in its potential breadth — it may affect cognitive work across many sectors simultaneously rather than disrupting one industry at a time — and in the pace at which its capabilities are advancing. These factors suggest that policymakers should not simply assume that historical patterns will repeat.

Key Terms

• Task automability — The degree to which a specific work task can be performed by an AI or automated system, depending on its structure, context, and variability.
• Displacement — The loss of employment for specific workers due to technological replacement of the tasks they performed.
• Augmentation — The use of AI to enhance human worker productivity rather than replace human workers entirely.
• Technological unemployment — A reduction in employment caused by technological change, distinct from cyclical or structural unemployment.
• Transition cost — The economic and human cost borne by workers and communities during adjustment to a technological disruption, including retraining, geographic relocation, and income loss.
• Universal Basic Income (UBI) — A policy proposal for unconditional regular payments to all citizens, sometimes advocated as a response to technological unemployment.

Discussion Questions

1. Some economists argue that worrying about AI-driven unemployment is as mistaken as worrying that ATMs would eliminate bank tellers (ATMs actually increased the number of bank branches and tellers overall). Others argue that AI is qualitatively different from previous automation. Which argument do you find more persuasive, and what evidence would change your mind?
2. Historical technological transitions involved significant human costs even when they produced long-run economic benefits. Who bore those costs, and who captured the benefits? Do you see similar patterns emerging with AI?
3. What obligations, if any, do AI companies have to workers displaced by the systems they build? What obligations do governments have? What obligations do consumers have?
4. The concept of ਕਿਰਤ ਕਰਨਾ (kirat karnā) — honest labor as a spiritual practice — is central to Sikh ethics. How does this teaching frame the question of what work is for, and what it would mean to lose it? Does this perspective offer resources for thinking about the future of work?

Further Reading

Acemoglu and Johnson, Power and Progress: Our Thousand-Year Struggle Over Technology and Prosperity — a historical examination of who captures the gains from technological progress.

Brynjolfsson and McAfee, The Second Machine Age — an influential account of AI's economic implications, optimistic about long-run benefits while acknowledging transition costs.

Daron Acemoglu, Tasks, Automation, and the Rise in US Wage Inequality — a technical but accessible economic analysis of how automation has affected wage distribution.

Key Takeaways

• AI automation is most likely to affect tasks that are well-defined, repetitive, and information-based; tasks requiring physical dexterity, social intelligence, and ethical judgment in novel situations are more resistant.
• Historical technological transitions destroyed specific jobs while creating new ones, but the transitions involved real human costs that were not evenly distributed — and AI may differ in its breadth and pace.
• Policy responses to technological disruption — retraining programs, social safety nets, UBI proposals — require deliberate political choices; adjustment does not happen automatically or equitably.
• The concept of ਕਿਰਤ ਕਰਨਾ (kirat karnā) invites us to ask not only what work we will do, but what work is for — a question that becomes more urgent as AI takes on more cognitive tasks.