-
Notifications
You must be signed in to change notification settings - Fork 8
Description
Summary
When an agent step has no applicable controls, evaluation should be a full no-op. If no enabled control applies to the current invocation, the SDK should behave as though no guardrail system is present for that step: no local evaluation work and no server evaluation request.
Motivation
This affects the common case where only a subset of tools or stages are protected. Teams expect unprotected steps to behave like plain function calls with no guardrail overhead. Making unnecessary evaluation calls adds avoidable latency, creates noise in traces/logs, and makes it harder to reason about whether a step is actually protected.
Current behavior
In the SDK local-evaluation path, controls are first partitioned by execution before full applicability is checked.
As a result:
- the SDK may still enter local evaluation whenever any parsed
execution="sdk"control exists for the agent, even if none apply to the current step - the SDK may still call server
POST /api/v1/evaluationwhenever anyexecution="server"control exists for the agent, even if none apply to the current step - the engine or server then repeats applicability filtering and often concludes that no controls were relevant
This means “no control for this tool/step/stage” is not a true no-op today.
Expected behavior
If there are no applicable controls for the current step, evaluation should be skipped entirely.
Concretely:
- if no enabled local control applies to the current tool/step/stage, the SDK should not run local evaluation
- if no enabled server control applies to the current tool/step/stage, the SDK should not call
/api/v1/evaluation - disabled controls and controls scoped to different tools/steps should behave as if they are absent for that invocation
- an invocation with no applicable controls should be treated as a no-op end to end
Reproduction (if bug)
- Create an agent with controls that are scoped away from the current invocation, for example a server or local control with
step_names=["send_email"], or a control that is disabled. - Initialize the SDK so those controls are cached locally.
- Invoke a different
@control()-wrapped tool or step, for examplelookup_order, at a stage where no control applies. - Observe that the SDK still enters local evaluation and/or calls
POST /api/v1/evaluation, even though no control applies to that invocation.
Proposed solution (optional)
Add an SDK-side applicability prefilter before local or server evaluation is triggered. The prefilter should mirror the existing engine applicability checks:
enabledscope.stagesscope.step_typesscope.step_namesscope.step_name_regexexecution
Only controls that pass that applicability check for the current invocation should:
- be included in local SDK evaluation
- cause the SDK to make a server evaluation request
This keeps the no-op path cheap and aligns the SDK’s behavior with how users reason about control scope.
Additional context
- Observed while reviewing feat(server)!: implement recursive control condition trees #115
- Relevant code paths today:
sdks/python/src/agent_control/evaluation.py(check_evaluation_with_local)sdks/python/src/agent_control/control_decorators.py(_evaluate)engine/src/agent_control_engine/core.py(get_applicable_controls)