Part 1·Chapter 1 of 13· 9 min read

A Network Is a Pile of Numbers

Before the experiments can make sense, one idea has to land.

Reading Audience LevelRung 2: L2 · Curious adult

Plain words and small, concrete numbers. - Step up or down anytime with keyboard shortcuts bracket keys or the buttons above.

Every neural network - the game player, the chat model, the image maker - is, at its core, a long list of decimal numbers. Change the numbers and you change the skill. This chapter makes that feel true, with knobs you can turn.

Teaching schematicSame wiring, different numbers, different skill. ILLUSTRATIVE, NOT REAL DATA.
Behaviour: “catch the ball”Behaviour: “run from ghosts”architecture identical · θ different ⇒ f different
Teaching schematic

M1 · The Knob Room

m1-knob-room
○ Class 0◆ Class 1- Decision BoundaryAccuracy: 50%x₁ →x₂ ↑

Neuron Hyperplane

w₁x₁ + w₂x₂ + b = 0

9 / 18 pts
w₁ (weight 1) = +1.00
w₂ (weight 2) = -0.60
w₁ Slider+1.00
w₂ Slider-0.60
Bias b+0.00

Accuracy 50% - move the knobs (or drag up/down on them) until the line splits ○ from ◆

L2 view: A tiny “neuron” multiplies two inputs by two knobs (weights), adds a bias, and picks a side. Same knobs = same behaviour.

TEACHING SCHEMATIC - invented toy data, ILLUSTRATIVE ONLY. The aha: weights ARE behaviour.

Teaching schematic

A network and its weight space

f-network

A neural network is a function f whose behaviour is fully described by P numbers θ - its weights.

•
f_θ - the network as a function
•
θ (theta) - the P-dimensional vector of all weights and biases
•
x - one input

Example: GPT-scale models: P is billions. Exp 1’s game-reader: P = 17,253.

Raw LaTeX (copy/paste)
f_\theta(x),\quad \theta\in\mathbb{R}^{P}

The numbers inside a network are called parameters or weights (the two words blur together; parameters includes biases). A modern chat model has hundreds of millions to hundreds of billions of them. The small math model fixed in the paper’s flagship experiment, Qwen3.5-0.8B, has about 800 million.

Same architecture + different numbers = different behaviour. That is the single most important sentence in this guide. The wiring (how many layers, how they connect) is the architecture; the numbers filling that wiring are the weights. Nothing else distinguishes a French poetry model from a chess model - only the numbers.

Now imagine the list of every possible setting of every number - a space of possible machines. Researchers call it weight space. Training is a long walk through this space. Editing is a short jump. And “reading” is trying to tell where in the space a machine sits, from the coordinates alone, without ever running it.

Foundational Refreshers (1)