research(orchestration): CommCP conformal prediction for reliable inter-agent message calibration #1890
Description
Summary
Uses conformal prediction to calibrate LLM-generated messages between agents in multi-agent coordination, reducing receiver distraction and improving task success rate by filtering low-confidence inter-agent instructions.
Source: arXiv 2602.06038 — "CommCP: Efficient Multi-Agent Coordination via Conformal Prediction" (February 2026)
Technique
Frames inter-agent information gathering as multi-agent multi-task EQA. Each agent-to-agent message is accompanied by a conformal prediction set — a calibrated uncertainty estimate over possible instruction interpretations. The receiving agent only acts on messages whose prediction sets are narrow (high confidence); uncertain messages trigger clarification requests instead of speculative execution. No model retraining required — calibration uses held-out interaction traces.
Applicability to Zeph
MEDIUM. Relevant to the multi-agent orchestration layer:
- LlmAggregator: sub-agent outputs composed into parent context — conformal calibration could flag low-confidence sub-agent results before aggregation
- LlmPlanner to DagScheduler dispatch: task assignments carry uncertainty; calibration prevents scheduler from executing under-specified tasks
- A2A agent-to-agent calls: external agent responses could be calibrated before being trusted
Lower priority than SYNAPSE (#1887) or SideQuest (#1885) given implementation complexity.
Implementation sketch
- Calibration layer in LlmAggregator: compute conformal set width from logprobs or embedding variance
- Threshold config: [orchestration.commcp] enabled = false, confidence_threshold = 0.8
- On narrow set (confident): proceed as normal
- On wide set (uncertain): emit clarification sub-task back to DagScheduler