A Network Is a Pile of Numbers
Before the experiments can make sense, one idea has to land.
Plain words and small, concrete numbers. - Step up or down anytime with keyboard shortcuts bracket keys or the buttons above.
Every neural network - the game player, the chat model, the image maker - is, at its core, a long list of decimal numbers. Change the numbers and you change the skill. This chapter makes that feel true, with knobs you can turn.
Neuron Hyperplane
w₁x₁ + w₂x₂ + b = 0
Accuracy 50% - move the knobs (or drag up/down on them) until the line splits ○ from ◆
L2 view: A tiny “neuron” multiplies two inputs by two knobs (weights), adds a bias, and picks a side. Same knobs = same behaviour.
TEACHING SCHEMATIC - invented toy data, ILLUSTRATIVE ONLY. The aha: weights ARE behaviour.
A network and its weight space
A neural network is a function f whose behaviour is fully described by P numbers θ - its weights.
- •
- f_θ - the network as a function
- •
- θ (theta) - the P-dimensional vector of all weights and biases
- •
- x - one input
Example: GPT-scale models: P is billions. Exp 1’s game-reader: P = 17,253.
Raw LaTeX (copy/paste)
f_\theta(x),\quad \theta\in\mathbb{R}^{P}The numbers inside a network are called parameters or weights (the two words blur together; parameters includes biases). A modern chat model has hundreds of millions to hundreds of billions of them. The small math model fixed in the paper’s flagship experiment, Qwen3.5-0.8B, has about 800 million.
Same architecture + different numbers = different behaviour. That is the single most important sentence in this guide. The wiring (how many layers, how they connect) is the architecture; the numbers filling that wiring are the weights. Nothing else distinguishes a French poetry model from a chess model - only the numbers.
Now imagine the list of every possible setting of every number - a space of possible machines. Researchers call it weight space. Training is a long walk through this space. Editing is a short jump. And “reading” is trying to tell where in the space a machine sits, from the coordinates alone, without ever running it.
Foundational Refreshers (1)