AI-powered codebase audits with rotating focus areas, multi-provider support, and decision memory.
The problem: Codebases drift. Security issues creep in, docs go stale, patterns diverge, dead code accumulates. Linters catch syntax — they miss semantics.
The solution: Noxaudit runs focused AI audits, rotating through different concerns. It remembers what you've already reviewed so only genuinely new findings surface.
Tell noxaudit what to audit — it does the rest. Default is all 7 focus areas:
noxaudit run # all focus areas (default)
noxaudit run --focus security # single area
noxaudit run --focus security,performance # multiple areas (files deduped, ~80% token savings)Each run, Noxaudit:
- Gathers relevant files from your codebase (with optional pre-pass triage)
- Audits via an AI provider (Claude, GPT, Gemini) with focused prompts and decision context
- Validates each finding against source code — classifies confidence, drops false positives
- Deduplicates findings by normalizing titles to canonical forms for cross-run stability
- Scores confidence using cross-run frequency analysis from findings history
- Filters against your decision history so resolved issues don't resurface
- Reports — generates markdown/SARIF, sends notifications, creates GitHub issues
pip install noxaudit
# Create config (edit to match your project)
cp noxaudit.yml.example noxaudit.yml
# Run a security audit
export ANTHROPIC_API_KEY=sk-...
noxaudit run --focus security
# Run multiple focus areas in one call
noxaudit run --focus security,performance
# Run all focus areas at once
noxaudit run --focus all
# Review a finding and dismiss it
noxaudit decide abc123def456 --action dismiss --reason "This is test code"Add to .github/workflows/noxaudit.yml:
name: Noxaudit Audit
on:
schedule:
- cron: '0 6 * * *' # 6am UTC daily
workflow_dispatch:
inputs:
focus:
description: 'Focus area(s) — name, comma-separated, or "all"'
type: string
jobs:
audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v7
- run: uv pip install 'noxaudit[openai]'
- run: noxaudit run --focus ${{ inputs.focus || 'all' }} --format sarif
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
- uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: .noxaudit/reports/Use --format sarif to produce SARIF output compatible with GitHub Code Scanning. The SARIF file is saved alongside the markdown report.
$ noxaudit run --focus security
my-app: 3 new findings
$ noxaudit report
# Security Audit — my-app
## 2025-01-14
### HIGH: Hardcoded API key in test fixture
**File:** tests/fixtures/config.py:12
**Confidence:** high
The string `sk-proj-abc123` is committed to the repo. Even in test fixtures,
real credentials in source control are a liability.
**Suggestion:** Replace with `os.environ.get("TEST_API_KEY", "sk-test-placeholder")`.
### MEDIUM: SQL string interpolation in query builder
**File:** src/db/queries.py:87
**Confidence:** medium
`cursor.execute(f"SELECT * FROM users WHERE id = {user_id}")` is vulnerable
to SQL injection. Use parameterized queries.
**Suggestion:** `cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))`
### MEDIUM: Permissive CORS in production config
**File:** src/config/cors.py:23
**Confidence:** low
`allow_origins=["*"]` in the production config allows any origin.
**Suggestion:** Restrict to known domains before shipping.
---
3 new findings (1 high, 2 medium) | 5 findings suppressed by decisions
Security Audit — my-app
3 new findings: 1 high, 2 medium
Hardcoded API key in test fixture
tests/fixtures/config.py
SQL string interpolation in query builder
src/db/queries.py
Permissive CORS in production config
src/config/cors.py
5 previous findings still resolved
Install the MCP extra and add to your project's .mcp.json:
pip install 'noxaudit[mcp]'{
"mcpServers": {
"noxaudit": {
"command": "noxaudit",
"args": ["mcp-server"]
}
}
}Then ask your AI assistant directly:
You: What security findings are open in this repo?
Claude: I found 3 open security findings:
[HIGH] [security] a1b2c3d4 — Hardcoded API key in test fixture
Location: tests/fixtures/config.py:12
A real API key `sk-proj-abc123` is committed in a test fixture.
Suggestion: Use an environment variable or a clearly fake placeholder.
[MEDIUM] [security] e5f6g7h8 — SQL string interpolation in query builder
Location: src/db/queries.py:87
f-string used in cursor.execute() — vulnerable to SQL injection.
Suggestion: Switch to parameterized queries.
...
After the AI provider returns raw findings, Noxaudit runs three post-processing stages to reduce noise and improve consistency:
Validate — A second LLM pass reads each finding against the actual source code. Findings are classified as high, medium, low, or false_positive confidence. False positives are dropped by default. Configure with validate.enabled and validate.min_confidence.
Deduplicate — Normalizes finding titles to canonical forms via .noxaudit/dedup-vocab.json, so the same issue gets the same title across runs. This stabilizes decision matching and history tracking. On by default (dedup.enabled).
Confidence scoring — Cross-run frequency analysis using .noxaudit/findings-history.jsonl. Findings that recur across multiple runs get higher confidence (60%+ = high, 30%+ = medium, below = low). This upgrades but never downgrades the confidence assigned during validation. Runs automatically.
See Finding Quality in the docs for details.
.noxaudit/
├── decisions.jsonl # Team decisions — commit this
├── latest-findings.json # Latest findings (for MCP server)
├── findings-history.jsonl # Cross-run history (for confidence scoring)
├── dedup-vocab.json # Canonical title mappings (for dedup)
├── cost-ledger.jsonl # Audit cost history (for `noxaudit status`)
└── reports/
└── my-app/
├── 2025-01-13-security.md
├── 2025-01-14-patterns.md
└── 2025-01-15-docs.md
| Command | Description |
|---|---|
noxaudit run |
Run an audit (submit + wait for results) |
noxaudit submit |
Submit a batch audit (returns immediately) |
noxaudit retrieve |
Retrieve results from a submitted batch |
noxaudit decide |
Record a decision about a finding |
noxaudit report |
Show the latest report |
noxaudit estimate |
Preview token count and cost estimate — no API keys needed |
noxaudit status |
Show config, last 30 days of audit costs, projected monthly spend |
noxaudit baseline |
Mass-suppress existing findings for adoption (--focus, --severity, --undo, --list) |
noxaudit mcp-server |
Start the MCP server for editor integration |
See CLI Reference for full usage.
Create a noxaudit.yml in your project root. See noxaudit.yml.example for all options.
| Option | Description | Default |
|---|---|---|
repos[].path |
Path to repository | . |
repos[].provider_rotation |
AI providers to rotate through (see Providers section) | [anthropic] |
model |
AI model to use (see Providers section for provider-specific setup) | claude-sonnet-4-6 |
providers.<name>.model |
Override model for a specific provider (e.g., providers.openai.model) |
(uses global model) |
prepass |
Pre-pass filtering configuration (see Providers section) | disabled |
validate.enabled |
Post-audit LLM validation of findings against source code | false |
validate.drop_false_positives |
Automatically remove findings classified as false positives | true |
validate.min_confidence |
Minimum confidence to keep (low, medium, high, or empty for no filter) |
(none) |
dedup.enabled |
LLM-based deduplication of finding titles | true |
budget.max_per_run_usd |
Hard cap on spend per audit run | 2.0 |
budget.alert_threshold_usd |
Warn when a run approaches this cost | 1.5 |
issues.enabled |
Auto-create GitHub issues for findings above severity threshold | false |
issues.severity_threshold |
Minimum severity for issue creation (low, medium, high) |
medium |
chunk_size |
Split large repos into N-file chunks for batch API | 0 (disabled) |
decisions.expiry_days |
Days before a decision expires | 90 |
notifications |
Where to send summaries | (none) |
Full configuration reference at docs.noxaudit.com/config.
| Area | What It Checks |
|---|---|
| security | Secrets, injection vulnerabilities, permissions, dependency CVEs |
| testing | Missing coverage, edge cases, test quality, flaky tests |
| docs | README accuracy, stale comments, API doc drift |
| patterns | Naming conventions, architecture consistency, duplicated logic |
| performance | Missing caching, expensive patterns, bundle size |
| hygiene | Dead code, orphaned files, stale config |
| dependencies | Outdated packages, security advisories |
When noxaudit finds something you've already addressed, you can record a decision:
# "We fixed this"
noxaudit decide abc123 --action accept --reason "Fixed in PR #42"
# "This is fine, stop flagging it"
noxaudit decide def456 --action dismiss --reason "Test fixture, not real credentials"
# "We know, it's on purpose"
noxaudit decide ghi789 --action intentional --reason "Intentionally permissive CORS for dev"Decisions are stored in .noxaudit/decisions.jsonl and fed to future runs. A finding won't resurface unless:
- The file it's in changes
- The decision expires (default: 90 days)
Commit your decisions file to share across the team.
Reports are saved as markdown in .noxaudit/reports/{repo}/{date}-{focus}.md.
Noxaudit supports three AI providers with 10 models. We benchmarked all of them against real repos to find which ones actually deliver.
Benchmark-informed tiers:
| Tier | Model | Cost/Run | Why |
|---|---|---|---|
| Daily | gpt-5-mini |
~$0.03 | 5/6 consensus issues, minimal noise — best value |
| Deep dive | gpt-5.4 |
~$0.26 | 84 findings, beats Sonnet at 68% the cost |
| Premium | claude-opus-4-6 |
~$0.65 | Most findings overall, maximum depth |
Set your API keys:
export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
export GOOGLE_API_KEY=...Install optional provider dependencies (Anthropic is built-in):
pip install 'noxaudit[openai]' # OpenAI
pip install 'noxaudit[google]' # GeminiRotate between providers and enable pre-pass filtering:
repos:
- name: my-app
path: .
provider_rotation: [anthropic, gemini, openai]
exclude: [vendor, generated, node_modules]
# Use Anthropic by default
model: claude-sonnet-4-6
# Optional: Set provider-specific models (overrides `model` for that provider)
providers:
gemini:
model: gemini-2.5-flash
openai:
model: gpt-5-mini
# Pre-pass: automatically filter large repos before sending to AI
prepass:
enabled: true
threshold_tokens: 600_000 # Auto-enable if codebase exceeds this
auto: true
# Post-audit validation: verify findings against source code
validate:
enabled: true
drop_false_positives: true
# Budget guardrails
budget:
max_per_run_usd: 2.0
alert_threshold_usd: 1.5
# Auto-create GitHub issues for high-severity findings
issues:
enabled: true
severity_threshold: high
labels: [noxaudit, security]Each audit will cycle through provider_rotation: first run uses Anthropic (with default model), second uses Gemini (with gemini-2.5-flash), third uses OpenAI (with gpt-5-mini), then repeat. Use providers.<name>.model to set provider-specific models that override the global model setting. See Key Options for all configuration.
| Provider | Model | Input/M | Output/M | Batch |
|---|---|---|---|---|
| Anthropic | claude-opus-4-6 |
$5.00 | $25.00 | 50% off |
| Anthropic | claude-sonnet-4-6 |
$3.00 | $15.00 | 50% off |
| Anthropic | claude-haiku-4-5 |
$1.00 | $5.00 | 50% off |
gemini-2.5-pro |
$1.25 | $10.00 | 50% off | |
gemini-3-flash-preview |
$0.50 | $3.00 | 50% off | |
gemini-2.5-flash |
$0.30 | $2.50 | 50% off | |
| OpenAI | gpt-5.4 |
$2.50 | $15.00 | 50% off |
| OpenAI | o4-mini |
$1.10 | $4.40 | 50% off |
| OpenAI | gpt-5-mini |
$0.25 | $2.00 | 50% off |
| OpenAI | gpt-5-nano |
$0.05 | $0.40 | 50% off |
Full pricing details, tiered rates, and cache pricing in the Provider Reference.
MIT