How does an AI work?
Neurons, layers, parameters: the mechanics behind ChatGPT, Midjourney and the others.
On this page
1. Rules learned, not written
The problem of spotting a cat in a photo
Writing by hand every rule to recognize a cat ("two pointed ears, whiskers…") is nearly impossible: cats come in every shape, color and pose.
Modern AI's workaround
We show the AI millions of photos already labeled "cat" or "not cat". It adjusts its own internal settings by itself until it can tell them apart. This adjustment mechanism is called training.
Why don't we write the rules for recognizing a cat in a photo by hand?
A modern AI already "knows" what a cat is, like a dictionary definition.
What is the mechanism called by which an AI adjusts its own settings from examples?
2. The artificial neuron
A tiny calculator
The artificial neuron is loosely inspired by brain neurons. The first mathematical model dates back to 1943 (Warren McCulloch and Walter Pitts).
What a neuron does, step by step
1. It receives inputs — Numbers: for example the brightness of image pixels, or the numeric representation of a word.
2. It weighs each input — Each input is multiplied by a weight, which shows its importance. A high weight means "this signal matters a lot".
3. It sums everything up — The sum of weighted inputs, plus a small offset called the bias.
4. It applies an activation function — A simple rule decides how strong the transmitted signal is. Without this step, stacking neurons would be pointless.
Put the 4 steps of an artificial neuron in order
What year was the first mathematical model of the artificial neuron published?
A high weight means a signal matters … in the neuron's decision.
What would happen if we removed the activation function from a neural network?
3. The network: layers that extract meaning
Neurons organized in layers
The first layer receives raw data, the last gives the result, and the layers in between ("hidden" layers) progressively transform the information. That's deep learning — "deep" refers to the number of layers.
A hierarchy that emerges on its own
In a network that recognizes images, early layers detect simple traits (edges, colors), later ones detect shapes (eyes, ears), the last ones detect whole objects (a face, a cat). Nobody programmed this hierarchy: it emerges from training.
In an image-recognition network, put the layers in the order information is processed
An engineer explicitly programs, layer by layer, what each layer should detect (edges, then shapes, then objects).
What does "deep" mean in "deep learning"?
4. Parameters: the model's knowledge
Settings, not memory
All the weights and biases of a model are called its parameters. A model holds no database of facts and no memorized sentences: its knowledge is spread across billions of numbers.
A dizzying growth
The number of parameters in GPT models multiplied by roughly 1,500 in two years: GPT-1 ≈ 117 million, GPT-2 ≈ 1.5 billion, GPT-3 ≈ 175 billion parameters.
GPT-3 has roughly how many parameters?
An AI model stores its knowledge as browsable files, similar to a database.
Is a model with more parameters always better?
5. Training and inference
Two very different moments
| Training | Inference (use) | |
|---|---|---|
| What happens? | The model adjusts its parameters from data | The frozen model produces a response to your request |
| When? | Once (or in big phases), before deployment | On every user request |
| Duration | Weeks or months for a large model | A few seconds |
| Cost | Very high: thousands of GPUs, millions of dollars for the largest models | Low per request, but huge in total with millions of users |
"The AI learns from every one of my conversations in real time." In practice, an already-deployed model doesn't change its parameters while talking to you. Some companies may use your conversations to train future models, unless you opt out in the settings.
Sort each statement: Training or Inference?
A deployed AI model updates its parameters in real time while chatting with you.
What costs the most overall for a company running a large model, even though each individual request is cheap?
Boss challenge: Training or Inference?
Sort each statement as fast and accurately as you can. Same exercise type as the last lesson, with more statements and a timer.