Skip to content

Commit 10a5a42

Browse files
sjarmakclaude
andcommitted
chore: update progress.txt with US-014 learnings
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 68882aa commit 10a5a42

File tree

1 file changed

+24
-0
lines changed

1 file changed

+24
-0
lines changed

ralph-mcp-unique/progress.txt

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -325,3 +325,27 @@
325325
- file_set_match + provenance: file coverage + citation verification (text must mention file path and repo)
326326
- pkg/dataobj/explorer/service.go confirmed via direct read_file: 4 endpoints, no middleware wrapping
327327
---
328+
[2026-02-20 20:57:58 UTC] Iteration 4 no story markers found
329+
[2026-02-20 20:57:58 UTC] Iteration 4 complete
330+
[2026-02-20 20:58:00 UTC] Iteration 5 started
331+
332+
## 2026-02-20 - US-014: Starter tasks — Category D: Incident debugging (1 task)
333+
- Authored 1 complete task in benchmarks/ccb_mcp_incident/:
334+
335+
**CCX-incident-031** (D31 — error to code path, multi-org-go fixture):
336+
- Scenario: on-call SRE traces Kubernetes watch-stream error "etcdserver: mvcc: required revision has been compacted" to authoritative etcd source
337+
- Oracle: {etcd-io/etcd, server/storage/mvcc/kvstore.go, ErrCompacted} + {etcd-io/etcd, server/storage/mvcc/kvstore_txn.go, rangeKeys}
338+
- Decoy design: kubernetes/kubernetes has vendored copies at vendor/go.etcd.io/etcd/server/v3/storage/mvcc/kvstore.go + apiserver error-mapping at staging/src/k8s.io/apiserver/pkg/storage/etcd3/errors.go — instruction explicitly warns about these to test precision
339+
- eval.sh: file_set_match (oracle files must match) + keyword_presence ("ErrCompacted", "rangeKeys")
340+
- Validity gate: VALID (gold=1.0, empty=0.0)
341+
- Registered in configs/selected_mcp_unique_tasks.json (6 tasks total now)
342+
343+
- Files changed: benchmarks/ccb_mcp_incident/ccx-incident-031/ (8 new files), configs/selected_mcp_unique_tasks.json, prd.json
344+
- **Learnings for future iterations:**
345+
- Decoy design principle: use vendored copies in local repo as decoys — they look identical to the real source but are not authoritative
346+
- "negative knowledge" in instruction.md (explicitly warning about decoys) creates clearer test of whether agent can distinguish source vs. copy
347+
- kubernetes/kubernetes is large (~800MB) so --depth 1 is essential for Dockerfile clone
348+
- multi-org-go fixture: kubernetes=local, etcd/grafana=MCP-only — natural for incident debugging where error originates in etcd
349+
- Error "mvcc: required revision has been compacted" verified: defined in kvstore.go line 37, returned in kvstore_txn.go line 81 (rangeKeys function)
350+
- oracle_checks.py file_set_match matches on (repo, path) tuples — repo must be "etcd-io/etcd" not "github.com/etcd-io/etcd"
351+
---

0 commit comments

Comments
 (0)