01 / Recurrent computation
Continuous Thought Machine LMs
16-tick CTM beats its matched 1-tick control by 0.157 nats @ 400M tokens — pre-registered, FLOP-matched
Does temporal synchronization (CTM, arXiv:2505.05522) improve recurrent computation in language models beyond what recurrent depth alone provides (arXiv:2502.05171)? Tested against standard and recurrent-depth Transformers on identical data.
- Pre-registered plan: FLOP- and token-matched comparisons, 2×2 mechanism ablations, paired-bootstrap statistics, go/no-go gates
- Cross-position ticks add 18.6 points on a synthetic word-problem task; tick-count controls show no gain at 100M tokens
- Code audit caught silent bugs — permutation-invariant token attention, unwired config flags, eval tokenization errors
Status: Active[GitHub ↗]