feat: abstract SDK-injected prompt text into a configurable PromptTemplates layer

[santoshkumarradha]

Problem

The SDKs inject prompt text into LLM calls in several places, but this text is hardcoded, invisible to users, and not overridable. As we add more features (tool-calling, schema enforcement, memory injection), the amount of SDK-injected text will grow. Without a principled abstraction, this becomes a debugging and customization nightmare.

Current Injection Points

Python SDK (agent_ai.py)

1. Schema instruction (lines 282-289) — hardcoded instruction block:

   schema_instruction = (
       "IMPORTANT: You must exactly adhere to the output schema provided below. "
       "Do not add or omit any fields..."
   )

2. Tool-calling loop (tool_calling.py) — NO system prompt injected for tool usage (the LLM gets tools but zero instructions on how to use them)

3. Tool limit message — hardcoded string sent to LLM:

   {"error": "Tool call limit reached. Please provide a final response."}

4. Tool error framing — hardcoded error format:

   {"error": str(e), "tool": func_name}

5. Tool results — raw json.dumps(result) with no framing or metadata

Go SDK (ai/tool_calling.go)

Same patterns: hardcoded error strings, no system prompt for tool usage, raw tool results.

Proposed Solution

1. PromptTemplates class (Python) / PromptConfig struct (Go)

All SDK-injected text lives in one discoverable, overridable location:

@dataclass
class PromptTemplates:
    """All text the SDK injects into LLM calls. Override any field to customize."""
    
    schema_instruction: str = (
        "IMPORTANT: You must exactly adhere to the output schema provided below..."
    )
    tool_system_prompt: Optional[str] = (
        "You have access to capabilities from an agent network. "
        "Use tools when the user's request requires action."
    )
    tool_limit_message: str = "Tool call limit reached. Please provide a final response."
    tool_error_template: str = '{"error": "{error}", "tool": "{tool_name}"}'
    tool_result_template: Optional[str] = None  # None = raw JSON (current behavior)

# Users can inspect
print(app.ai_config.prompt_templates.schema_instruction)

# Users can override
app.ai_config.prompt_templates.tool_system_prompt = "My custom agent instructions..."

# Users can suppress
app.ai_config.prompt_templates.tool_system_prompt = None  # No system prompt injected

2. Trace-level visibility

The ToolCallTrace (and future trace types) should tag which messages were SDK-injected vs user-provided:

@dataclass
class TracedMessage:
    message: dict           # The actual message
    source: str             # "user" | "sdk.schema_instruction" | "sdk.tool_result" | etc.
    injected_at: float      # timestamp

3. Parity across SDKs

Both Python and Go SDKs should have the same template fields and override mechanism so behavior is consistent regardless of SDK choice.

Scope

• Create PromptTemplates in Python SDK
• Create PromptConfig in Go SDK
• Migrate existing hardcoded schema instruction to use templates
• Add default tool system prompt (with override)
• Add tool result framing template
• Add message source tagging to traces
• Documentation: "Customizing SDK Prompts" guide
• Tests for override behavior

Context

Spawned from PR #228 review (tool-calling support). The tool-calling feature makes this more urgent since it introduces a multi-turn loop where SDK-injected text compounds across turns.

Relates to #225

[idealley]

Please do not forget the TS sdk

[santoshkumarradha]

Please do not forget the TS sdk

Thanks for the info, we have been seeing great traffic for python. But will update it with TS as well !

[santoshkumarradha]

Audit: TS SDK + Go SDK Gap Analysis & Cross-References

After a thorough codebase audit, the TS SDK (and to some extent the Go SDK) has identical hardcoded prompt injection patterns that are missing from this issue's scope. Here's the evidence.

---

Hardcoded Strings Across All 3 SDKs (Side-by-Side)

Injection Point	Python SDK (tool_calling.py)	Go SDK (ai/tool_calling.go)	TS SDK (ai/ToolCalling.ts)
Tool limit message	"Tool call limit reached. Please provide a final response." (L477)	"Tool call limit reached. Please provide a final response." (L167)	"Tool call limit reached. Please provide a final response." (L388)
Tool error format	{"error": str(e), "tool": func_name} (L534)	{"error": err.Error(), "tool": tc.Function.Name} (L193-196)	{ error: record.error, tool: name } (L411)
Tool result framing	Raw json.dumps(result) — no framing (L515)	Raw json.Marshal(result) — no framing (L204)	Raw agent.call() return — no framing (L402)
Tool system prompt	❌ None	❌ None	❌ None
Schema instruction	Hardcoded 6-line block (agent_ai.py L308-316)	❌ No structured output support	❌ Uses Vercel AI SDK native generateObject — no prompt injection needed

Schema Instruction Behavioral Divergence

The Python SDK injects a hardcoded 6-line prompt into the system message for schema adherence:

# agent_ai.py:308-316
schema_instruction = (
    "IMPORTANT: You must exactly adhere to the output schema provided below. "
    "Do not add or omit any fields. Output must be valid JSON matching the schema. "
    ...
)

The TS SDK delegates to Vercel AI SDK's generateObject() with native schema support. The Go SDK has no structured output support at all.

Impact: Users switching between SDKs get different behavior for the same logical operation. This should be documented, not forced into uniformity.

---

Recommended Scope Additions

Add to scope checklist:

• Create PromptTemplates interface in TypeScript SDK
• Migrate TS SDK hardcoded strings (ToolCalling.ts L383, L388, L411) to templates
• Add TracedMessage source tagging to TS SDK ToolCallTrace (parity with Python proposal)
• Document schema enforcement behavioral differences between SDKs (Python = prompt-injected, TS = native, Go = none)
• Go SDK: Add PromptConfig equivalent for ToolCallResponse wrapper (Python has one at tool_calling.py:85-124, Go does not)
• Go SDK: Deduplicate CapabilityToToolDefinition — Reasoner and Skill branches (tool_calling.go L49-73 and L74-99) are identical code

Cross-References

• Depends on: [Python SDK] Create custom exception classes #111 (Python custom exceptions — error templates should use exception classes)
• Related to: [Control Plane] Extract shared ErrorResponse helper #119 (Go ErrorResponse extraction — same "centralize scattered strings" pattern)
• Related to: [TypeScript SDK] Align memory scope naming with Python SDK #93 (TS SDK memory scope naming — broader SDK parity initiative)
• Related to: [Web UI] Replace any types in ExecutionForm.tsx #105 (TSX type safety — the as any in ToolCalling.ts L371/L446 should be fixed when adding PromptTemplates)

Risks to Flag

1. The TS SDK's tool-calling loop uses t as any (L371, L446) to access .description and .inputSchema on Vercel AI SDK tool objects — this needs proper typing when refactoring for PromptTemplates
2. Python's schema_instruction is prompt-injected while TS uses native generateObject — PromptTemplates should document this intentional divergence

---

Filed from codebase audit cross-referencing #225, #119, #111, #93, #105