Record: 11L XSA-all + 7-gram cache (mean val_bpb=1.0465)

records/track_10min_16mb/2026-03-25_11L_XSA_7gram/README.md

@@ -0,0 +1,35 @@
+# 11L XSA-all + 7-gram Eval Cache
+
+## Results
+
+| Seed | val_bpb |
+|------|---------|
+| 42   | 1.0467  |
+| 1337 | 1.0470  |
+| 2024 | 1.0457  |
+| **Mean** | **1.0465** |
+| Std  | 0.0007  |
+
+Artifact: 13.99 MB. Train: 600s on 8xH100 SXM. Eval: ~116s.
+
+## Architecture
+
+- 11 layers, 512 dim, 8/4 GQA heads
+- XSA (Exclusive Self-Attention) on all 11 layers
+- LeakyReLU(0.5)^2 MLP, 3x expansion
+- BigramHash (10240, 128d), SmearGate
+- Value Residual, Gated Attention
+- U-Net skip, tied embeddings, LN scaling 1/sqrt(layer+1)
+- GPTQ-lite int6 quantization + zstd-22
+- EMA(0.997), Tight SWA, Late QAT
+- Muon (lr=0.025, mom=0.99, WD=0.04) + AdamW
+
+## 7-gram Eval Cache
+
+- Order 7, fixed alpha=0.40, 4M hash buckets, min_count=2
+- Score-first backward-looking, deterministic
+- No TTT, no gradient updates
+
+## Acknowledgments
+
+Architecture based on community techniques from PRs #609, #549, and others.

records/track_10min_16mb/2026-03-25_11L_XSA_7gram/submission.json

@@ -0,0 +1,9 @@
+{
+  "name": "11L XSA-all + 7-gram cache",
+  "val_loss": 1.0465,
+  "bytes_total": 13991619,
+  "blurb": "11 layers XSA-all, LeakyReLU(0.5)^2, VR, GA, 7-gram eval cache. Mean 1.0465 (std 0.0007) across 3 seeds.",
+  "author": "hypery11",
+  "github_id": "hypery11",
+  "date": "2026-03-25"
+}