Skip to content

Skip evaluation entirely when no controls apply to the current step #123

@lan17

Description

@lan17

Summary

When an agent step has no applicable controls, evaluation should be a full no-op. If no enabled control applies to the current invocation, the SDK should behave as though no guardrail system is present for that step: no local evaluation work and no server evaluation request.

Motivation

This affects the common case where only a subset of tools or stages are protected. Teams expect unprotected steps to behave like plain function calls with no guardrail overhead. Making unnecessary evaluation calls adds avoidable latency, creates noise in traces/logs, and makes it harder to reason about whether a step is actually protected.

Current behavior

In the SDK local-evaluation path, controls are first partitioned by execution before full applicability is checked.

As a result:

  • the SDK may still enter local evaluation whenever any parsed execution="sdk" control exists for the agent, even if none apply to the current step
  • the SDK may still call server POST /api/v1/evaluation whenever any execution="server" control exists for the agent, even if none apply to the current step
  • the engine or server then repeats applicability filtering and often concludes that no controls were relevant

This means “no control for this tool/step/stage” is not a true no-op today.

Expected behavior

If there are no applicable controls for the current step, evaluation should be skipped entirely.

Concretely:

  • if no enabled local control applies to the current tool/step/stage, the SDK should not run local evaluation
  • if no enabled server control applies to the current tool/step/stage, the SDK should not call /api/v1/evaluation
  • disabled controls and controls scoped to different tools/steps should behave as if they are absent for that invocation
  • an invocation with no applicable controls should be treated as a no-op end to end

Reproduction (if bug)

  1. Create an agent with controls that are scoped away from the current invocation, for example a server or local control with step_names=["send_email"], or a control that is disabled.
  2. Initialize the SDK so those controls are cached locally.
  3. Invoke a different @control()-wrapped tool or step, for example lookup_order, at a stage where no control applies.
  4. Observe that the SDK still enters local evaluation and/or calls POST /api/v1/evaluation, even though no control applies to that invocation.

Proposed solution (optional)

Add an SDK-side applicability prefilter before local or server evaluation is triggered. The prefilter should mirror the existing engine applicability checks:

  • enabled
  • scope.stages
  • scope.step_types
  • scope.step_names
  • scope.step_name_regex
  • execution

Only controls that pass that applicability check for the current invocation should:

  • be included in local SDK evaluation
  • cause the SDK to make a server evaluation request

This keeps the no-op path cheap and aligns the SDK’s behavior with how users reason about control scope.

Additional context

  • Observed while reviewing feat(server)!: implement recursive control condition trees #115
  • Relevant code paths today:
    • sdks/python/src/agent_control/evaluation.py (check_evaluation_with_local)
    • sdks/python/src/agent_control/control_decorators.py (_evaluate)
    • engine/src/agent_control_engine/core.py (get_applicable_controls)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions