Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ repos:
hooks:
- id: nbstripout
args: [--keep-output, '--extra-keys=metadata.kernelspec']
files: ^doc/*\.(ipynb)$
files: ^doc/.*\.ipynb$

- repo: local
hooks:
Expand Down
2 changes: 1 addition & 1 deletion doc/code/converters/7_human_converter.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"metadata": {},
"source": [
"> ⚠️ **Deprecated:** The `HumanInTheLoopConverter` is deprecated and will be removed in PyRIT v0.13.0.\n",
"> Use the React-based GUI (CoPyRIT) instead. See the [GUI documentation](../gui/0_gui.md) for details.\n",
"> Use the React-based GUI (CoPyRIT) instead. See the [GUI documentation](../../gui/0_gui.md) for details.\n",
"\n",
"# 7. Human in the Loop Converter\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion doc/code/converters/7_human_converter.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

# %% [markdown]
# > ⚠️ **Deprecated:** The `HumanInTheLoopConverter` is deprecated and will be removed in PyRIT v0.13.0.
# > Use the React-based GUI (CoPyRIT) instead. See the [GUI documentation](../gui/0_gui.md) for details.
# > Use the React-based GUI (CoPyRIT) instead. See the [GUI documentation](../../gui/0_gui.md) for details.
#
# # 7. Human in the Loop Converter
#
Expand Down
54 changes: 14 additions & 40 deletions doc/code/datasets/5_simulated_conversation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "e88a2a30",
"id": "0",
"metadata": {},
"source": [
"# 5. Simulated Conversations\n",
Expand Down Expand Up @@ -31,16 +31,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"id": "0e4097b9",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:43:58.449045Z",
"iopub.status.busy": "2026-04-08T14:43:58.448892Z",
"iopub.status.idle": "2026-04-08T14:44:23.536340Z",
"shell.execute_reply": "2026-04-08T14:44:23.534625Z"
}
},
"execution_count": null,
"id": "1",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand Down Expand Up @@ -90,7 +83,7 @@
},
{
"cell_type": "markdown",
"id": "1b3f2a33",
"id": "2",
"metadata": {},
"source": [
"## Inspecting the Results\n",
Expand All @@ -104,16 +97,9 @@
},
{
"cell_type": "code",
"execution_count": 2,
"id": "b6a517b3",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:44:23.538379Z",
"iopub.status.busy": "2026-04-08T14:44:23.538095Z",
"iopub.status.idle": "2026-04-08T14:44:23.543978Z",
"shell.execute_reply": "2026-04-08T14:44:23.542643Z"
}
},
"execution_count": null,
"id": "3",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand Down Expand Up @@ -183,7 +169,7 @@
},
{
"cell_type": "markdown",
"id": "8604fef5",
"id": "4",
"metadata": {},
"source": [
"## Replaying on a Different Target\n",
Expand All @@ -198,16 +184,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"id": "153c5476",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:44:23.546118Z",
"iopub.status.busy": "2026-04-08T14:44:23.545878Z",
"iopub.status.idle": "2026-04-08T14:45:52.821098Z",
"shell.execute_reply": "2026-04-08T14:45:52.819962Z"
}
},
"execution_count": null,
"id": "5",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand Down Expand Up @@ -546,7 +525,7 @@
},
{
"cell_type": "markdown",
"id": "625f4681",
"id": "6",
"metadata": {},
"source": [
"> **Note:** If the Crescendo result shows `backtrack_count: 0` even on failure, this is expected.\n",
Expand All @@ -557,7 +536,7 @@
},
{
"cell_type": "markdown",
"id": "58bb69b5",
"id": "7",
"metadata": {},
"source": [
"## Key Parameters\n",
Expand All @@ -580,11 +559,6 @@
}
],
"metadata": {
"kernelspec": {
"display_name": "pyrit-dev",
"language": "python",
"name": "pyrit-dev"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
Expand Down
91 changes: 22 additions & 69 deletions doc/code/executor/benchmark/2_bias_benchmark.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "1f4ad056",
"id": "0",
"metadata": {},
"source": [
"# 2. Bias & Fairness Benchmark\n",
Expand All @@ -28,16 +28,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"id": "40a3d028",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:46:05.706290Z",
"iopub.status.busy": "2026-04-08T14:46:05.706045Z",
"iopub.status.idle": "2026-04-08T14:46:15.733842Z",
"shell.execute_reply": "2026-04-08T14:46:15.732794Z"
}
},
"execution_count": null,
"id": "1",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand Down Expand Up @@ -68,7 +61,7 @@
},
{
"cell_type": "markdown",
"id": "9b3941a2",
"id": "2",
"metadata": {},
"source": [
"## Testing for Definitive YES/NO Answers\n",
Expand All @@ -83,16 +76,9 @@
},
{
"cell_type": "code",
"execution_count": 2,
"id": "8bc07414",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:46:15.736014Z",
"iopub.status.busy": "2026-04-08T14:46:15.735479Z",
"iopub.status.idle": "2026-04-08T14:46:15.818093Z",
"shell.execute_reply": "2026-04-08T14:46:15.816824Z"
}
},
"execution_count": null,
"id": "3",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand All @@ -115,16 +101,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"id": "3a736fc9",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:46:15.819598Z",
"iopub.status.busy": "2026-04-08T14:46:15.819462Z",
"iopub.status.idle": "2026-04-08T14:47:18.712842Z",
"shell.execute_reply": "2026-04-08T14:47:18.710967Z"
}
},
"execution_count": null,
"id": "4",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand Down Expand Up @@ -219,7 +198,7 @@
},
{
"cell_type": "markdown",
"id": "e8755a78",
"id": "5",
"metadata": {},
"source": [
"## Testing for Bias with FairnessBiasBenchmark\n",
Expand All @@ -236,16 +215,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"id": "26632ba1",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:47:18.715021Z",
"iopub.status.busy": "2026-04-08T14:47:18.714728Z",
"iopub.status.idle": "2026-04-08T14:47:18.831445Z",
"shell.execute_reply": "2026-04-08T14:47:18.829881Z"
}
},
"execution_count": null,
"id": "6",
"metadata": {},
"outputs": [],
"source": [
"# Create the pronoun scorer\n",
Expand All @@ -264,16 +236,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"id": "7572d6d9",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:47:18.833546Z",
"iopub.status.busy": "2026-04-08T14:47:18.833274Z",
"iopub.status.idle": "2026-04-08T14:48:27.824543Z",
"shell.execute_reply": "2026-04-08T14:48:27.822875Z"
}
},
"execution_count": null,
"id": "7",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand Down Expand Up @@ -480,7 +445,7 @@
},
{
"cell_type": "markdown",
"id": "a73e8820",
"id": "8",
"metadata": {},
"source": [
"## Analyzing Results\n",
Expand All @@ -493,16 +458,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"id": "509c7c08",
"metadata": {
"execution": {
"iopub.execute_input": "2026-04-08T14:48:27.826680Z",
"iopub.status.busy": "2026-04-08T14:48:27.826451Z",
"iopub.status.idle": "2026-04-08T14:48:27.832534Z",
"shell.execute_reply": "2026-04-08T14:48:27.831089Z"
}
},
"execution_count": null,
"id": "9",
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand All @@ -529,11 +487,6 @@
}
],
"metadata": {
"kernelspec": {
"display_name": "pyrit-dev",
"language": "python",
"name": "pyrit-dev"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
Expand Down
58 changes: 58 additions & 0 deletions doc/code/architecture.md → doc/code/framework.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,61 @@
# Framework

Learn how to use PyRIT's components to build red teaming workflows.

:::::{grid} 1 1 2 3
:gutter: 3

::::{card} 📦 Datasets
:link: ./datasets/0_dataset
Load, create, and manage seed datasets for red teaming campaigns.
::::

::::{card} ⚔️ Attacks & Executors
:link: ./executor/0_executor
Run single-turn and multi-turn attacks — Crescendo, TAP, Skeleton Key, and more.
::::

::::{card} 🔌 Targets
:link: ./targets/0_prompt_targets
Connect to OpenAI, Azure, Anthropic, HuggingFace, HTTP endpoints, and custom targets.
::::

::::{card} 🔄 Converters
:link: ./converters/0_converters
Transform prompts with text, audio, image, and video converters.
::::

::::{card} 📊 Scoring
:link: ./scoring/0_scoring
Evaluate AI responses with true/false, Likert, classification, and custom scorers.
::::

::::{card} 💾 Memory
:link: ./memory/0_memory
Track conversations, scores, and attack results with SQLite or Azure SQL.
::::

::::{card} ⚙️ Setup & Configuration
:link: ./setup/0_setup
Initialize PyRIT, configure defaults, and manage resiliency settings.
::::

::::{card} 📋 Scenarios
:link: ./scenarios/0_scenarios
Run standardized evaluation scenarios at scale across harm categories.
::::

::::{card} 🗂️ Registry
:link: ./registry/0_registry
Register and discover targets, scorers, and converters via class and instance registries.
::::

:::::

---

The sections above link to detailed guides for each component. The architecture below explains how the pieces fit together — it's primarily aimed at contributors.

# Architecture

The main components of PyRIT are prompts, attacks, converters, targets, and scoring. The best way to contribute to PyRIT is by contributing to one of these components.
Expand Down
Loading
Loading