AI, LLMs & AGI: What They Really Are

"AI" is not one thing. It is a label we paste onto any software that does something that used to feel like it needed a human mind: recognising a face, suggesting a route, finishing your sentence, beating a chess champion, or writing an essay. Because the label is so broad, two people can both say "AI" and mean completely different machines.

For most of computing history, getting a machine to act "smart" meant writing rules by hand: if the email contains this word, then mark it spam. This works for tidy problems but falls apart in the messy real world, where there are too many exceptions to ever list. The big shift was machine learning: instead of writing the rules, we show the program thousands or millions of examples and let it discover the patterns itself.

The tool that powers today's wave is the neural network — a large grid of simple math units connected together. Each connection has a number (a "weight"). Learning just means nudging those numbers until the network's outputs match the examples. A modern large language model is a neural network with billions of these numbers, trained on text. Nothing magical is added; it is scale, data, and patient tuning.

Keep this in mind for the rest of the course: an LLM is a pattern machine, not a knowing mind. That single idea explains most of its strengths and almost all of its failures.

Training an LLM happens in stages. The first and largest stage is pre-training. The model is shown an enormous amount of text — much of the public internet, books, code, and more — and given one repetitive task: read some text and predict what comes next. It guesses, the guess is compared to the real next piece of text, and its billions of internal numbers are nudged to make a slightly better guess next time. Repeat trillions of times.

The result of pre-training is a model that has absorbed a vast amount of how language, facts, and reasoning tend to look. As the educator Andrej Karpathy describes it in his popular talks, you can picture the finished model as a single huge file of numbers (the weights) plus a small program that runs them — a "compressed" and lossy summary of the text it read, not a copy of it.

But a raw pre-trained model is just a text-continuer; it will happily ramble. So a second stage, fine-tuning, teaches it to behave like a helpful assistant. Humans write or rate example conversations showing good answers, and the model is tuned toward that style. A common technique here is learning from human feedback, where people rank responses and the model learns to prefer the better-rated ones.

Two honest takeaways: training is extremely expensive and slow, which is why models have a knowledge "cutoff" date; and the model's knowledge is frozen at training time unless it is given fresh information at the moment you ask.

An LLM does not read whole words the way you do. Text is first chopped into tokens — small chunks that are often a word, sometimes part of a word, sometimes punctuation. The word "unhappiness" might become three tokens. The model only ever sees and produces these tokens.

When you send a message, the model reads your tokens and produces a single thing: a list of probabilities for what the next token should be. It picks one (usually favouring the likely options, with a touch of randomness for variety), adds it to the text, and then does the whole thing again to pick the next token. Your fluent paragraph is built one token at a time, each step a fresh prediction.

This explains several quirks people notice:

The single most useful mental model is this: the system is always answering the question "given everything so far, what is a plausible next token?" It is astonishingly good at that. It is not, at any point, looking up a fact in a database or deciding whether something is true. That gap is the subject of the next lesson.

A hallucination is when a model states something false while sounding completely sure. It invents a citation, a date, a quote, or a fact. People find this jarring because the tone is so confident. But once you remember the model is predicting plausible-sounding text, hallucination stops being mysterious — it is the system working exactly as built.

The model's goal is fluent, likely text, not true text. If a real answer and a fake-but-plausible answer look equally "normal," the model has no built-in sense that one is grounded in reality and the other is not. It does not know what it knows. There is no internal fact-checker unless one is added on the outside.

Why it happens, in plain terms:

Practical habits help a lot. Ask the model to show its reasoning or sources, then check those sources yourself. Give it the reference material directly rather than trusting its memory. Treat anything specific — names, numbers, citations, legal or medical claims — as a draft to verify, not a final answer. Newer systems reduce hallucination by looking things up while answering, but no current model is free of it.

Almost every AI in use today is narrow AI: very capable at a specific range of tasks and useless outside it. A model that writes beautiful essays cannot drive your car; a champion chess engine cannot summarise your email. Even a broad chat assistant, impressive as it is, is doing one underlying job — producing likely text — applied to many requests.

Artificial General Intelligence (AGI) is the idea of a single system that can learn and reason flexibly across nearly any task a capable human can, including ones it was never specifically trained for, and transfer skills from one domain to another. That does not exist today.

Here is the honest part: experts genuinely disagree, and not just about timing. They disagree about the definition. Some define AGI by economic impact (can it do most valuable human jobs?), others by cognitive breadth, others by whether it truly "understands." Because there is no agreed test, announcements that AGI is "almost here" or "impossible" are often arguing about different things.

As writers like Melanie Mitchell argue, terms such as "understanding" and "intelligence" are slippery, and our intuition about machines often misleads us. A sober stance: today's tools are remarkable narrow tools, the path to general intelligence is unknown, and confident predictions in either direction deserve healthy skepticism.

Let us end with a grounded scorecard, free of both hype and doom. Today's LLMs are genuinely excellent at language-shaped work: drafting, rewriting, summarising, translating, explaining ideas at any level, brainstorming, and helping with code. Used as a thinking partner and first-draft engine, they save real time.

Their limits are just as real. They can be confidently wrong, they have no live access to truth unless given it, their knowledge is frozen at a cutoff, they handle long documents imperfectly, and they have no genuine understanding, intentions, or feelings — they model the patterns of language about those things. They are also sensitive to how you ask: clear context and an example of good output dramatically improve results.

The wise way to use these tools follows directly from how they work: treat output as a capable draft, not an oracle; verify anything that matters; provide the facts you want it to use; and keep a human in charge of judgment. Understanding the machine — pattern prediction, not knowing — is what lets you get the benefit while avoiding the traps. That understanding, more than any single trick, is the real skill.