WRITE - Policies Born From Pure Thought
Experiments 5 and 6: game players written with zero data, zero gradients.
Plain words and small, concrete numbers. - Step up or down anytime with keyboard shortcuts bracket keys or the buttons above.
The boldest claim in the paper: an LLM can write a working network from scratch - no data, no loss, no gradients - by reasoning about the task. On three MinAtar games it beat policies trained for 100M steps; on 9×9 Go it beat an equal-size PPO trained on 20M games.
| Game | PPO · 100M steps | PPO · 500M steps | Astra, one shot |
|---|---|---|---|
| Breakout | 15.8 | 25.3 | 517.4 |
| Seaquest | 5.6 | 88.9 | 53.1 |
| Asterix | 25.8 | 55.2 | 53.2 |
A hand-compiled rule
The paddle rule as an actual network: two hidden units (ball left / ball right of paddle) feeding three logits (left / stay / right).
- •
- b, p - ball and paddle positions
- •
- h - hidden activations
- •
- z - action logits
Example: See lab/01_compile_a_rule.py - 17 numbers that play a passable Breakout.
Raw LaTeX (copy/paste)
h=\mathrm{ReLU}(W_1[b,p]^{\top}+b_1),\quad z=W_2h+b_2| Opponent | 0 passes | 50 passes |
|---|---|---|
| Two small PPO nets | 54–55% | 81–84% |
| Equal-size PPO (20M games) | 26–43% | 70–79% |
Exp 5 - MinAtar. Three tiny games (Breakout, Seaquest, Asterix). For each, the LLM wrote a 600→32→32→6 tanh network starting from all-zero weights - no data, no loss function, no gradients, no rollouts. Scores (1,000-episode averages) against PPO baselines: Breakout 517.4 vs 15.8 (100M steps) and 25.3 (500M); Seaquest 53.1 vs 5.6 / 88.9; Asterix 53.2 vs 25.8 / 55.2.
So: it crushed Breakout, beat the 100M-step baseline on all three games, and roughly matched the 500M-step PPO on Asterix while losing to it on Seaquest (53.1 vs 88.9). Reading the table honestly is part of the fun - the win is real but uneven, not a uniform stomp.
Exp 6 - 9×9 Go. A 243→512×6→82 network (1,480,274 weights) written from zero in a Python sandbox, then allowed 0–50 forward passes of self-play to refine. At 0 passes it already won the majority of games against two small PPO nets (54–55%); after 50 passes it beat even an equal-size PPO trained on 20M games (70–79%).
Foundational Refreshers (1)