Record: EMA-GPU + Multi-Order N-gram Backoff + PE Confidence (val_bpb=0.9393)#187

Closed

Idan3011 wants to merge 73 commits intoopenai:mainfrom

Idan3011:submission

Commits on Mar 18, 2026

feat: add pre-enrichment linear projections before transformer blocks
Idan3011
committed
feat: add GELU activation between pre-enrichment projections
Idan3011
committed

Commits on Mar 19, 2026

feat: add encoder depth recurrence (2x encoder pass before decoder)
Idan3011
committed
feat: full 2x U-Net recurrence (encoder+decoder both run twice)
Idan3011
committed
feat: 3x encoder recurrence (3 encoder passes, 1 decoder pass)
Idan3011
committed
revert: back to 2x encoder recurrence (3x hits Triton shared memory limit)
Idan3011
committed
feat: configurable encoder/decoder split via NUM_ENCODER_LAYERS
Idan3011
committed
cleanup: remove NUM_ENCODER_LAYERS override, keep best config
Idan3011
committed
feat: auxiliary encoder loss for direct encoder gradient
Idan3011
committed

Commits on Mar 20, 2026

Commits on Mar 21, 2026

Commits on Mar 23, 2026

feat: TTT (test-time training) eval with SGD, freeze first 2 blocks
Idan3011
committed

Commits on Mar 25, 2026

Commits on Mar 26, 2026