Record: N-gram Backoff + VRL + LeakyReLU² — val_bpb 0.9642 (3-seed mean)#887

Closed

anthony-maio wants to merge 28 commits intoopenai:mainfrom

anthony-maio:submission/match-sota-plus-ttt

Commits on Mar 21, 2026

Record: SOTA recipe (PR #162, 1.1483 bpb) + TTT LoRA eval

anthony-maio
and
claude
committed
Match FarnsworthEngine: 11L + full-weight SGD TTT + tuned hyperparams

anthony-maio
and
claude
committed
Reduce warmdown from 3000 to 1500 steps

anthony-maio
and
claude
committed
Fix TTT eval: use args.eval_stride instead of undefined local variable

anthony-maio
and
claude
committed
Record: 9L MLP3x full SOTA stack, val_bpb=1.1401

anthony-maio
and
claude
committed
Integrate fused ReLU² MLP Triton kernel (1.26x eval speedup)

anthony-maio
and
claude
committed
Add FlashAttention 3 support + fused ReLU² MLP kernel

anthony-maio
and
claude
committed

Commits on Mar 22, 2026

Commits on Mar 23, 2026

Remove broken TTT code from PR #376 to pass review

anthony-maio
and
claude
committed

Commits on Mar 26, 2026

Record: N-gram Backoff + VRL + LeakyReLU² — val_bpb 0.9642 (3-seed)

anthony-maio
and
claude
committed