Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
277 changes: 210 additions & 67 deletions examples/crewai/README.md
Original file line number Diff line number Diff line change
@@ -1,122 +1,265 @@
# CrewAI Customer Support with Agent Control + Guardrails
# CrewAI + Agent Control Examples

Combines Agent Control (security/compliance) with CrewAI Guardrails (quality retries) for production customer support.
Combines **Agent Control** (runtime security & compliance) with **CrewAI** (agent orchestration) for production-grade AI agents.

## What It Does
Agent Control (Security): PRE/POST/FINAL blocks unauthorized access and PII.
CrewAI Guardrails (Quality): validates length/structure/tone with up to 3 retries.
**Agent Control** enforces guardrails at tool boundaries -- blocking unauthorized access, PII leakage, SQL injection, and more -- while **CrewAI Guardrails** handle quality retries. Both coexist without code changes to CrewAI.

## Prerequisites
## How It Works

```
User Request
|
v
CrewAI Agent (planning & orchestration)
|
v
@control() decorator (PRE check) <- LAYER 1: Agent Control validates input
|
v
Tool executes (LLM call, DB query, etc.)
|
v
@control() decorator (POST check) <- LAYER 2: Agent Control validates output
|
v
CrewAI Guardrails (quality retries) <- LAYER 3: CrewAI validates quality
|
v
Return to user (or block / steer / warn)
```

**Agent Control** blocks or steers immediately (security). **CrewAI Guardrails** retry with feedback (quality). Controls are defined on the **server** -- update rules without redeploying agents.

Before running this example, ensure you have:
## Prerequisites

- **Python 3.12+**
- **uv** — Fast Python package manager (`curl -LsSf https://astral.sh/uv/install.sh | sh`)
- **Docker** — For running PostgreSQL (required by Agent Control server)
- **uv** -- Fast Python package manager (`curl -LsSf https://astral.sh/uv/install.sh | sh`)
- **Docker** -- For PostgreSQL (`docker compose -f docker-compose.dev.yml up -d`)
- **OpenAI API key** -- `export OPENAI_API_KEY="your-key"` (only needed for full LLM crew runs; simulated scenarios work without it)

## Installation
## Quick Start

### 1. Install Monorepo Dependencies
### 1. Install dependencies

From the monorepo root, install all workspace packages:
From the monorepo root:

```bash
cd /path/to/agent-control
make sync
```

This installs the Agent Control SDK and all workspace packages in editable mode.

### 2. Install CrewAI Example Dependencies
### 2. Start the Agent Control server

Navigate to the CrewAI example and install its specific dependencies:
In a separate terminal:

```bash
cd examples/crewai
uv pip install -e . --upgrade
make server-run
```

Verify it's running:

### 3. Set OpenAI API Key
```bash
curl http://localhost:8000/health
# {"status":"healthy","version":"0.1.0"}
```

Create a `.env` file or export the environment variable:
### 3. Pick an example and run it

Each example has a `setup_controls.py` (one-time, idempotent) and a main script:

```bash
export OPENAI_API_KEY="your-key-here"
cd examples/crewai/secure_research_crew
uv run python setup_controls.py
uv run python -m secure_research_crew.main
```

### 4. Start the Agent Control Server
---

## Examples

### 1. [Content Agent Protection](./content_agent_protection.py) -- PII & Access Control

The foundational example. A single-agent customer support crew with multi-layer protection:

In a separate terminal, start the server from the monorepo root:
- **PRE-execution**: Block unauthorized data access (regex evaluator)
- **POST-execution**: Block PII leakage in tool output (regex evaluator)
- **Final validation**: Catch PII that bypasses tool-level controls
- **CrewAI Guardrails**: Quality retries for length, tone, and structure

| Scenario | Layer | Result |
|----------|-------|--------|
| Unauthorized access | Agent Control PRE | Blocked |
| PII in tool output | Agent Control POST | Blocked |
| Short/low-quality response | CrewAI Guardrails | Retry then pass |
| Agent bypass attempt | Agent Control FINAL | Blocked |

```bash
cd /path/to/agent-control
make server-run
cd examples/crewai
uv run python setup_content_controls.py
uv run python content_agent_protection.py
```

**Verify server is running:**
### 2. [Steering Financial Agent](./steering_financial_agent/) -- Deny, Steer & Warn

Demonstrates all three Agent Control action types in a wire-transfer scenario:

| Action | Behavior | Example |
|--------|----------|---------|
| **DENY** | Hard block, no recovery | Sanctioned country, fraud detected |
| **STEER** | Pause, guide agent to correct, retry | 2FA required, manager approval needed |
| **WARN** | Log for audit, no blocking | New recipient, unusual activity |

```bash
curl http://localhost:8000/health
cd examples/crewai/steering_financial_agent
uv run python setup_controls.py
uv run python -m steering_financial_agent.main
```

### 5. Setup Content Controls (One-Time)
### 3. [Evaluator Showcase](./evaluator_showcase/) -- All 4 Built-in Evaluators

Demonstrates every built-in evaluator in a data-analyst scenario:

From the `examples/crewai` directory, run the setup script:
| Evaluator | Stage | Purpose |
|-----------|-------|---------|
| **SQL** | PRE | Block DROP/DELETE, enforce LIMIT, prevent injection |
| **LIST** | PRE | Restrict access to sensitive tables |
| **REGEX** | POST | Catch SSN, email, credit cards in query results |
| **JSON** | PRE | Validate required fields, enforce constraints, steer for missing data |

```bash
uv run python setup_content_controls.py
cd examples/crewai/evaluator_showcase
uv run python setup_controls.py
uv run python -m evaluator_showcase.main
```

### 4. [Secure Research Crew](./secure_research_crew/) -- Multi-Agent Crew with Per-Role Policies

A production-quality **3-agent sequential crew** (Researcher, Analyst, Writer) where each agent has its own policy with distinct controls. This is the pattern you'd use in real multi-agent systems.

## Running the Example
```
+--------------+ +--------------+ +---------------+
| Researcher | --> | Analyst | --> | Writer |
| | | | | |
| query_database| | validate_data| | write_report |
+--------------+ +--------------+ +---------------+
| | |
data-access-policy analysis-validation content-safety
- SQL safety [deny] - Required fields [deny] - PII blocker [deny]
- Restricted tables - Methodology [steer]
[deny]
```

Make sure you're in the `examples/crewai` directory and run:
**5 scenarios** -- all run without LLM calls (direct tool testing):

| # | Scenario | Agent | Evaluator | Action | Result |
|---|----------|-------|-----------|--------|--------|
| 1 | Happy path | All 3 | All | allow | Report generated with sources |
| 2 | SQL injection | Researcher | SQL | deny | "Multiple SQL statements not allowed" |
| 3 | Restricted table | Researcher | LIST | deny | salary_data access blocked |
| 4 | Missing methodology | Analyst | JSON Schema | steer | Auto-corrected, passes on retry |
| 5 | PII in report | Writer | REGEX | deny | SSN/email/phone blocked |

```bash
uv run python content_agent_protection.py
cd examples/crewai/secure_research_crew
uv run python setup_controls.py
uv run python -m secure_research_crew.main
```

### Expected Behavior
### 5. [Content Publishing Flow](./content_publishing_flow/) -- CrewAI Flow with Routing & Human-in-the-Loop

| Scenario | Layer | Result |
|----------|-------|--------|
| Unauthorized access | Agent Control PRE | Blocked |
| PII in tool output | Agent Control POST | Blocked |
| Short/low-quality response | Guardrails | Retry then pass |
| Agent bypass attempt | Agent Control FINAL | Blocked |
A complete **CrewAI Flow** using `@start`, `@listen`, and `@router` decorators with Agent Control guardrails at every pipeline stage. Content is routed through different paths based on risk level, with embedded crews and steering for human approval.

### Output Legend
PRE checks input before the LLM.
POST checks tool output for PII.
FINAL checks the crew’s final response.
Agent Control blocks immediately (no retries), violations are logged.
Guardrails retry with feedback (quality-only).
```
@start: intake_request (JSON validation)
|
@listen: research (Researcher + Fact-Checker)
|
@listen: draft_content (PII + banned topic checks)
|
@router: quality_gate
|
+-- "low_risk" (blog_post) --> auto_publish (final PII scan)
+-- "high_risk" (press_release) --> compliance_review (legal + editor steering)
+-- "escalation" (internal_memo)--> human_review (STEER: manager approval)
```

## Agent Control + CrewAI Integration
**6 scenarios** covering all three routing paths plus control blocking:

Agent Control works seamlessly **with** CrewAI's agent orchestration:
| # | Scenario | Flow Path | Result |
|---|----------|-----------|--------|
| 1 | Blog post | low_risk -> auto-publish | Published |
| 2 | Press release | high_risk -> compliance review | Steered (exec summary), then published |
| 3 | Internal memo | escalation -> human review | Steered: pending manager approval |
| 4 | Invalid request | intake blocked | JSON evaluator: missing fields |
| 5 | Banned topic | draft blocked | LIST evaluator: "insider trading" detected |
| 6 | PII in draft | draft blocked | REGEX evaluator: email/SSN/phone detected |

1. **CrewAI Agent Layer**: Plans tasks, selects tools, manages conversation flow
2. **Agent Control Layer**: Enforces controls and business rules at tool boundaries
```bash
cd examples/crewai/content_publishing_flow
uv run python setup_controls.py
uv run python -m content_publishing_flow.main
```

---

## Feature Coverage

| Feature | Ex 1 | Ex 2 | Ex 3 | Ex 4 | Ex 5 |
|---------|:----:|:----:|:----:|:----:|:----:|
| `@control()` decorator | Yes | Yes | Yes | Yes | Yes |
| PRE-execution checks | Yes | Yes | Yes | Yes | Yes |
| POST-execution checks | Yes | Yes | Yes | Yes | Yes |
| **deny** action | Yes | Yes | Yes | Yes | Yes |
| **steer** action | | Yes | Yes | Yes | Yes |
| **warn** action | | Yes | | | |
| Regex evaluator | Yes | | Yes | Yes | Yes |
| List evaluator | | Yes | Yes | Yes | Yes |
| JSON evaluator | | Yes | Yes | Yes | Yes |
| SQL evaluator | | | Yes | Yes | |
| Steering context + retry loop | | Yes | Yes | Yes | Yes |
| ControlViolationError handling | Yes | Yes | Yes | Yes | Yes |
| ControlSteerError handling | | Yes | Yes | Yes | Yes |
| **Multi-agent crew** | | | | **Yes** | |
| **Per-agent policies** | | | | **Yes** | |
| **CrewAI Flow (@start/@listen/@router)** | | | | | **Yes** |
| **Conditional routing** | | | | | **Yes** |
| **Human-in-the-loop (steer)** | | | | | **Yes** |
| **Pydantic state management** | | | | | **Yes** |
| CrewAI Guardrails coexistence | Yes | | | | |

## Architecture Deep Dive

### Multi-Agent Crew (Example 4)

The SDK supports one `agent_control.init()` per process. All three CrewAI agents share a single Agent Control identity, but each tool's `step_name` routes it to the right policy:

```
User Request
CrewAI Agent (planning & orchestration)
Decides to call tool
@control() decorator (PRE-execution) ← LAYER 1: Validates input
Tool executes (LLM generation)
@control() decorator (POST-execution) ← LAYER 2: Validates tool output
If blocked, agent may generate own response
Final Output Validation ← LAYER 3: Validates crew output (catches bypass)
Return to user (or block if control violated)
Single agent_control.init()
|
+------------------------------------------------------------+
| CrewAI Sequential Crew |
| |
| +--------------+ +--------------+ +----------------+ |
| | Researcher |-->| Analyst |-->| Writer | |
| | query_database| | validate_data| | write_report | |
| +--------------+ +--------------+ +----------------+ |
+------------------------------------------------------------+
| | |
data-access-policy analysis-validation content-safety
(SQL + LIST deny) (JSON deny + steer) (REGEX deny)
```

### CrewAI Flow (Example 5)

```
@start: intake_request -----> @listen: research -----> @listen: draft_content
(JSON validation) (LIST + REGEX checks) (REGEX + LIST checks)
|
@router: quality_gate
/ | \
/ | \
"low_risk" "high_risk" "escalation"
| | |
auto_publish compliance human_review
(REGEX PII) (JSON + STEER) (STEER)
```
2 changes: 1 addition & 1 deletion examples/crewai/content_agent_protection.py
Original file line number Diff line number Diff line change
Expand Up @@ -467,7 +467,7 @@ def verify_setup():

try:
print("Verifying Agent Control server connection...")
response = httpx.get(f"{server_url}/api/v1/controls", timeout=5.0)
response = httpx.get(f"{server_url}/api/v1/controls?limit=100", timeout=5.0)
response.raise_for_status()

controls_data = response.json()
Expand Down
5 changes: 5 additions & 0 deletions examples/crewai/content_publishing_flow/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# OpenAI API key for CrewAI agents
OPENAI_API_KEY=your-openai-api-key-here

# Agent Control server URL (default: http://localhost:8000)
AGENT_CONTROL_URL=http://localhost:8000
Loading
Loading