Part 2·Chapter 6 of 13· 11 min read

WRITE - Policies Born From Pure Thought

Experiments 5 and 6: game players written with zero data, zero gradients.

Reading Audience LevelRung 2: L2 · Curious adult

Plain words and small, concrete numbers. - Step up or down anytime with keyboard shortcuts bracket keys or the buttons above.

The boldest claim in the paper: an LLM can write a working network from scratch - no data, no loss, no gradients - by reasoning about the task. On three MinAtar games it beat policies trained for 100M steps; on 9×9 Go it beat an equal-size PPO trained on 20M games.

Paper-reportedMinAtar scores, average over 1,000 episodes (Exp 5)
GamePPO · 100M stepsPPO · 500M stepsAstra, one shot
Breakout15.825.3517.4
Seaquest5.688.953.1
Asterix25.855.253.2
Astra's policies were written with zero data, gradients, or rollouts - reasoning only. n=1 run per cell.
Teaching schematic

A hand-compiled rule

f-compile

The paddle rule as an actual network: two hidden units (ball left / ball right of paddle) feeding three logits (left / stay / right).

•
b, p - ball and paddle positions
•
h - hidden activations
•
z - action logits

Example: See lab/01_compile_a_rule.py - 17 numbers that play a passable Breakout.

Raw LaTeX (copy/paste)
h=\mathrm{ReLU}(W_1[b,p]^{\top}+b_1),\quad z=W_2h+b_2
Independent review9×9 Go win rates from the released logs (Exp 6) - 🟩 independent review
Opponent0 passes50 passes
Two small PPO nets54–55%81–84%
Equal-size PPO (20M games)26–43%70–79%
Reported as ranges in the independent review of the released logs; n=1 run per point.
Independent reviewExp 6 Go win rates from the released logs (ranges mid-plotted): reported points only, never interpolated.
win rate vs two small PPO nets · vs equal-size PPO (20M games)vs small PPO0 passes: 54–55%50 passes: 81–84%vs equal-size PPO0 passes: 54–55%50 passes: 81–84%n = 1 run per point · ranges as reported by the review (🟩)

Exp 5 - MinAtar. Three tiny games (Breakout, Seaquest, Asterix). For each, the LLM wrote a 600→32→32→6 tanh network starting from all-zero weights - no data, no loss function, no gradients, no rollouts. Scores (1,000-episode averages) against PPO baselines: Breakout 517.4 vs 15.8 (100M steps) and 25.3 (500M); Seaquest 53.1 vs 5.6 / 88.9; Asterix 53.2 vs 25.8 / 55.2.

So: it crushed Breakout, beat the 100M-step baseline on all three games, and roughly matched the 500M-step PPO on Asterix while losing to it on Seaquest (53.1 vs 88.9). Reading the table honestly is part of the fun - the win is real but uneven, not a uniform stomp.

Exp 6 - 9×9 Go. A 243→512×6→82 network (1,480,274 weights) written from zero in a Python sandbox, then allowed 0–50 forward passes of self-play to refine. At 0 passes it already won the majority of games against two small PPO nets (54–55%); after 50 passes it beat even an equal-size PPO trained on 20M games (70–79%).

Foundational Refreshers (1)