diff --git a/partner-built/bonito/.claude-plugin/plugin.json b/partner-built/bonito/.claude-plugin/plugin.json new file mode 100644 index 00000000..62fb034f --- /dev/null +++ b/partner-built/bonito/.claude-plugin/plugin.json @@ -0,0 +1,13 @@ +{ + "name": "bonito", + "version": "1.0.0", + "description": "Deploy and manage AI infrastructure across cloud providers. Connect AWS Bedrock, Azure OpenAI, GCP Vertex AI, OpenAI, Anthropic, and Groq through a unified gateway. Create agents, configure intelligent routing with failover and cost optimization, analyze spending, and debug issues.", + "author": { + "name": "Bonito AI", + "url": "https://getbonito.com" + }, + "repository": "https://github.com/ShabariRepo/bonito", + "homepage": "https://getbonito.com", + "license": "MIT", + "keywords": ["ai-infrastructure", "multi-provider", "routing", "agents", "gateway", "aws-bedrock", "azure-openai", "gcp-vertex", "cost-optimization", "failover"] +} diff --git a/partner-built/bonito/.mcp.json b/partner-built/bonito/.mcp.json new file mode 100644 index 00000000..91980f31 --- /dev/null +++ b/partner-built/bonito/.mcp.json @@ -0,0 +1,10 @@ +{ + "mcpServers": { + "bonito": { + "command": "bonito-mcp", + "env": { + "BONITO_API_KEY": "" + } + } + } +} diff --git a/partner-built/bonito/CONNECTORS.md b/partner-built/bonito/CONNECTORS.md new file mode 100644 index 00000000..f510e3d0 --- /dev/null +++ b/partner-built/bonito/CONNECTORS.md @@ -0,0 +1,54 @@ +# Connectors + +## How tool references work + +Plugin files use `~~category` as a placeholder for whatever tool the user connects in that category. For example, `~~bonito` refers to the Bonito MCP server, which provides access to the Bonito AI Gateway API. + +Plugins are **tool-agnostic** — they describe workflows in terms of categories (gateway, notifications, version control, etc.) rather than specific products. The `.mcp.json` pre-configures the Bonito MCP server, but optional connectors in other categories enhance the experience. + +## Connectors for this plugin + +| Category | Placeholder | Included servers | Other options | +|----------|-------------|-----------------|---------------| +| AI Gateway | `~~bonito` | Bonito MCP | — | +| Version Control | `~~vcs` | — | GitHub, GitLab | +| Notifications | `~~notifications` | — | Slack, Microsoft Teams, Discord | +| Monitoring | `~~monitoring` | — | Datadog, Grafana, PagerDuty | + +## Required connector + +### Bonito MCP (`~~bonito`) + +The core connector. Provides access to the Bonito AI Gateway API for managing providers, agents, routing, and infrastructure. + +**Install:** `pip install bonito-mcp` + +**What it provides:** +- Provider management (create, list, verify, delete) +- Agent configuration (BonBon agents, Bonobot orchestrators) +- Routing policy management (failover, cost-optimized, A/B testing) +- Cost and usage analytics +- Gateway health and diagnostics +- Knowledge base management + +## Optional connectors + +### GitHub (`~~vcs`) + +Connect GitHub to enable infrastructure-as-code workflows, review bonito.yaml changes in PRs, and track deployment history. + +**What it adds:** +- Read bonito.yaml configs from repos +- Review infrastructure changes in pull requests +- Track deployment commits and history +- Manage infrastructure configs alongside application code + +### Slack (`~~notifications`) + +Connect Slack to receive deployment notifications, cost alerts, and provider status updates in your team channels. + +**What it adds:** +- Deployment success/failure notifications +- Cost threshold alerts +- Provider health status updates +- Agent error notifications diff --git a/partner-built/bonito/LICENSE b/partner-built/bonito/LICENSE new file mode 100644 index 00000000..c0843e42 --- /dev/null +++ b/partner-built/bonito/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Bonito AI + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/partner-built/bonito/README.md b/partner-built/bonito/README.md new file mode 100644 index 00000000..c45b6c3a --- /dev/null +++ b/partner-built/bonito/README.md @@ -0,0 +1,108 @@ +# Bonito Plugin + +An AI infrastructure management plugin primarily designed for [Cowork](https://claude.com/product/cowork), Anthropic's agentic desktop application — though it also works in Claude Code. Helps you deploy and manage AI infrastructure across cloud providers, create agents, configure intelligent routing, analyze costs, and debug issues. Works with any team — standalone with the Bonito API, supercharged when you connect GitHub, Slack, and other tools. + +## Installation + +```bash +claude plugin install bonito +``` + +## Skills + +Domain knowledge Claude uses automatically when relevant: + +| Skill | Description | +|---|---| +| `deploy-stack` | Deploy AI infrastructure from a bonito.yaml config — create providers, agents, knowledge bases, and routing in one shot | +| `manage-providers` | Connect, verify, and manage cloud AI providers — AWS Bedrock, Azure OpenAI, GCP Vertex AI, OpenAI, Anthropic, Groq | +| `create-agent` | Create and configure BonBon agents or Bonobot orchestrators with system prompts, models, MCP tools, and RAG | +| `cost-analysis` | Analyze AI spending across providers, identify expensive models, recommend cheaper alternatives, optimize routing | +| `gateway-routing` | Configure routing policies — cost-optimized, failover, A/B testing, model aliases, and cross-region inference | +| `debug-issues` | Troubleshoot gateway errors, provider failures, and agent issues — check logs, verify connections, test endpoints | + +## Example Workflows + +### Deploying Your AI Stack + +Just describe what you want: +``` +Deploy my AI infrastructure from bonito.yaml +``` + +The `deploy-stack` skill reads your config file, creates providers, agents, knowledge bases, and routing policies — all in one shot. It validates each step and reports what was created. + +### Connecting a New Provider + +``` +Connect my AWS Bedrock account as a provider +``` + +The `manage-providers` skill walks you through credential setup, creates the provider, verifies the connection, and lists available models. Works for any supported cloud provider. + +### Building an Agent + +``` +Create a customer support agent using Claude on AWS Bedrock with our FAQ knowledge base +``` + +The `create-agent` skill configures a BonBon agent with your chosen model, system prompt, knowledge base, and MCP tools. It deploys the agent and gives you the endpoint. + +### Optimizing Costs + +``` +What am I spending on AI across all providers? +``` + +The `cost-analysis` skill pulls usage data across all connected providers, breaks down costs by model and agent, identifies expensive patterns, and recommends cheaper alternatives or routing changes. + +## Standalone + Supercharged + +Every skill works without any additional integrations: + +| What You Can Do | Standalone | Supercharged With | +|-----------------|------------|-------------------| +| Deploy infrastructure | bonito.yaml + Bonito API | GitHub (track configs in repos) | +| Manage providers | Bonito API | Monitoring (health dashboards) | +| Create agents | Bonito API | GitHub (version agent configs) | +| Analyze costs | Bonito API | Slack (cost threshold alerts) | +| Configure routing | Bonito API | Monitoring (traffic dashboards) | +| Debug issues | Bonito API logs | Slack (error notifications), Monitoring | + +## MCP Integrations + +> If you see unfamiliar placeholders or need to check which tools are connected, see [CONNECTORS.md](CONNECTORS.md). + +Connect your tools for a richer experience: + +| Category | Examples | What It Enables | +|---|---|---| +| **AI Gateway** | Bonito MCP | Provider management, agent creation, routing, cost analytics | +| **Version Control** | GitHub, GitLab | Infrastructure-as-code, config versioning, PR reviews | +| **Notifications** | Slack, Teams, Discord | Deployment alerts, cost warnings, provider status updates | +| **Monitoring** | Datadog, Grafana | Traffic dashboards, latency tracking, health monitoring | + +See [CONNECTORS.md](CONNECTORS.md) for the full list of supported integrations. + +## Settings + +Create a `settings.local.json` file to personalize: + +- **Cowork**: Save it in any folder you've shared with Cowork (via the folder picker). The plugin finds it automatically. +- **Claude Code**: Save it at `bonito/.claude/settings.local.json`. + +```json +{ + "organization": "Your Company", + "bonito_api_url": "https://api.getbonito.com", + "default_model": "anthropic/claude-sonnet-4-20250514", + "preferences": { + "default_provider": "aws-bedrock", + "cost_alert_threshold": 100, + "routing_strategy": "cost-optimized", + "region": "us-east-1" + } +} +``` + +The plugin will ask you for this information interactively if it's not configured. diff --git a/partner-built/bonito/skills/cost-analysis/SKILL.md b/partner-built/bonito/skills/cost-analysis/SKILL.md new file mode 100644 index 00000000..0a958d5e --- /dev/null +++ b/partner-built/bonito/skills/cost-analysis/SKILL.md @@ -0,0 +1,75 @@ +--- +name: cost-analysis +description: "Analyze AI spending across providers, identify expensive models, recommend cheaper alternatives, and optimize routing for cost savings. Triggers on 'what am I spending', 'cost breakdown', 'optimize my AI costs', 'which models cost the most', or 'reduce my AI costs'." +--- + +# Cost Analysis + +Understand and optimize AI spending across all connected providers. Pull usage data, break down costs, identify expensive patterns, and recommend concrete savings. + +## Step 1: Determine Scope + +- "What am I spending?" -> full overview, current month +- "Costs by provider" -> provider breakdown +- "Most expensive models" -> model ranking +- "Compare months" -> trend analysis +- "Optimize costs" -> recommendations focus +- "Last 7 days" -> custom date range + +## Step 2: Pull Usage Data + +Query ~~bonito for: +1. Total requests, tokens, and costs for the period +2. Breakdown by provider +3. Breakdown by model +4. Breakdown by agent +5. Daily time series for trends + +## Step 3: Analyze Patterns + +- Which models account for the most spending? +- Are agents using expensive models for simple tasks? +- Is usage concentrated or spread across providers? +- Are there traffic spikes or retry storms? +- Is token usage efficient (prompt length vs response length)? + +## Step 4: Generate Recommendations + +**Model substitution:** +- Opus for simple tasks -> suggest Haiku or Sonnet +- GPT-4o for classification -> suggest GPT-4o-mini +- High-volume agent -> suggest Groq for speed + cost + +**Routing optimization:** +- One provider consistently more expensive -> route to cheaper alternative +- Bursty traffic -> suggest response caching +- Quality varies -> A/B test cheaper models + +## Step 5: Present Report + +``` +## AI Cost Analysis + +Period: [Start] to [End] +Total Spend: $[Amount] | Requests: [Count] | Tokens: [Count] + +### By Provider +| Provider | Requests | Cost | % of Total | +|----------|----------|------|------------| +| AWS Bedrock | [Count] | $[Amount] | [X]% | +| OpenAI | [Count] | $[Amount] | [X]% | + +### By Model (Top 5) +| Model | Provider | Requests | Cost | $/1K Req | +|-------|----------|----------|------|----------| +| claude-sonnet | Bedrock | [Count] | $[Amount] | $[Amount] | + +### Recommendations +1. [Action]: Switch [agent] from [expensive model] to [cheaper model] + Estimated savings: $[Amount]/month ([X]%) + +2. [Action]: [Description] + Estimated savings: $[Amount]/month ([X]%) + +Potential monthly savings: $[Amount] ([X]% reduction) +``` diff --git a/partner-built/bonito/skills/create-agent/SKILL.md b/partner-built/bonito/skills/create-agent/SKILL.md new file mode 100644 index 00000000..17ec6005 --- /dev/null +++ b/partner-built/bonito/skills/create-agent/SKILL.md @@ -0,0 +1,82 @@ +--- +name: create-agent +description: "Create and configure BonBon agents or Bonobot orchestrators with system prompts, models, MCP tools, and RAG knowledge bases. Triggers on 'create an agent', 'deploy a chatbot', 'set up a support bot', 'build an orchestrator', 'make a BonBon', or 'create a Bonobot'." +--- + +# Create Agent + +Create AI agents on the Bonito gateway. Handles both BonBon agents (single-model, task-focused) and Bonobot orchestrators (multi-agent, tool-using). + +## Step 1: Determine Agent Type + +- Simple task, single model, "chatbot", "support bot" -> **BonBon** +- Multi-step, coordination, "orchestrator", "multi-agent" -> **Bonobot** + +If unclear, ask: "Do you need a single-purpose agent or a multi-agent orchestrator?" + +## Step 2: Select Model and Provider + +1. List available providers via ~~bonito +2. Recommend model based on use case: + - Customer support -> Claude Sonnet (good balance) + - Complex reasoning -> Claude Opus or GPT-4o + - High volume, simple tasks -> Claude Haiku or Groq Llama + - Code generation -> Claude Sonnet or GPT-4o +3. Confirm with user + +## Step 3: Configure System Prompt + +If user provides one, use it directly. If they describe the role: +1. Generate a draft (role, tone, boundaries, output format) +2. Present for approval +3. Iterate until satisfied + +## Step 4: Attach Knowledge Base (Optional) + +1. Check for existing KBs via ~~bonito +2. If new: create KB, upload documents, wait for indexing +3. Configure retrieval settings (top-K, similarity threshold) + +## Step 5: Configure MCP Tools (Optional) + +Select and attach relevant MCP tool servers for the agent's purpose. + +## Step 6: Create, Deploy, and Test + +1. Assemble config and call ~~bonito to create +2. Send a basic greeting -> verify response +3. Send a domain query -> verify relevance +4. If tools/KB attached -> test those too + +## Output Format + +``` +## Agent Created: [Name] + +| Field | Value | +|-------|-------| +| Type | [BonBon / Bonobot] | +| Model | [Model] via [Provider] | +| Agent ID | [ID] | +| Endpoint | /v1/agents/[id]/chat | +| Knowledge Base | [Name or None] | +| MCP Tools | [Count or None] | +| Status | Deployed | + +### Test Results +| Test | Status | +|------|--------| +| Basic greeting | OK | +| Domain query | OK | + +### Quick Start +curl -X POST https://api.getbonito.com/v1/agents/[id]/chat \ + -H "Authorization: Bearer $BONITO_API_KEY" \ + -d '{"message": "Hello!"}' +``` + +## Agent Types + +**BonBon:** Single-model agent for focused tasks (support bots, Q&A, code review, content generation). One model, optional system prompt, optional KB, optional MCP tools. + +**Bonobot:** Multi-agent orchestrator that delegates to specialized sub-agents. For complex workflows spanning multiple domains or requiring different models for different tasks. diff --git a/partner-built/bonito/skills/debug-issues/SKILL.md b/partner-built/bonito/skills/debug-issues/SKILL.md new file mode 100644 index 00000000..b8f9a6c3 --- /dev/null +++ b/partner-built/bonito/skills/debug-issues/SKILL.md @@ -0,0 +1,78 @@ +--- +name: debug-issues +description: "Troubleshoot gateway errors, provider failures, agent issues, and routing problems. Check logs, verify connections, test endpoints, and diagnose root causes. Triggers on 'why is my agent failing', 'debug gateway', 'check provider status', 'something is broken', 'getting errors', or 'troubleshoot'." +--- + +# Debug Issues + +Diagnose and fix problems with the Bonito gateway, providers, agents, and routing. Systematically check each layer, identify the root cause, and guide to a fix. + +## Step 1: Understand the Problem + +- Specific error message -> start with that component +- General "it's broken" -> run full diagnostic sweep +- Specific agent -> start with agent health +- Specific provider -> start with provider check +- Performance issue -> start with latency analysis + +## Step 2: Check Each Layer + +**Gateway:** Is the API reachable? Is auth valid? What's the latency? + +**Providers:** For each connected provider: +- Is the API reachable? Are credentials valid? +- What's the error rate (last hour)? +- Are specific models affected, or all? + +**Agents:** For affected agents: +- Is the config valid? Does the referenced model exist? +- Send a test message. Check response or error. +- If KB attached, test retrieval. If tools attached, test availability. + +**Routing:** For affected policies: +- Are all referenced providers/models valid? +- Does failover trigger correctly? +- Any circular dependencies? + +## Step 3: Pull Logs + +Query ~~bonito for recent errors: +- Filter by time range and component +- Identify patterns: sudden spike (outage), gradual increase (rate limiting), intermittent (network) +- Count errors by type + +## Step 4: Present Diagnosis and Fix + +``` +## Diagnostic Report + +Issue: [User's reported problem] +Root Cause: [Brief description] + +### System Health +| Component | Status | Details | +|-----------|--------|---------| +| Gateway | Healthy | 45ms | +| Provider: Bedrock | Down | 503 Service Unavailable | +| Agent: support-bot | Error | Upstream provider failure | + +### What Happened +[Clear explanation of the issue chain] + +### Fix +1. [Immediate action to resolve] +2. [Verification step] + +### Prevention +1. [Step to prevent recurrence] +``` + +## Common Issues + +| Symptom | Likely Cause | Fix | +|---------|-------------|-----| +| 401/403 errors | Expired API key | Regenerate key, update config | +| Timeouts (504) | Provider overloaded | Retry or failover to alt provider | +| Model not found (404) | Typo or model deprecated | Verify model name, use aliases | +| Rate limiting (429) | Too many requests | Spread across providers, upgrade tier | +| Agent not responding | Bad config or provider down | Check config, test provider health | diff --git a/partner-built/bonito/skills/deploy-stack/SKILL.md b/partner-built/bonito/skills/deploy-stack/SKILL.md new file mode 100644 index 00000000..8b94eb8a --- /dev/null +++ b/partner-built/bonito/skills/deploy-stack/SKILL.md @@ -0,0 +1,87 @@ +--- +name: deploy-stack +description: "Deploy AI infrastructure from a bonito.yaml configuration file. Creates providers, agents, knowledge bases, and routing policies in one shot. Triggers on 'deploy my AI stack', 'set up Bonito', 'deploy bonito.yaml', 'create my infrastructure', or 'deploy from config'." +--- + +# Deploy Stack + +Deploy entire AI infrastructure from a single `bonito.yaml` config. Validates the schema, resolves dependencies, and creates all resources in the correct order. + +## Step 1: Parse and Validate + +Read bonito.yaml and extract: +- **Providers** (cloud AI services to connect) +- **Knowledge bases** (documents and embedding configs) +- **Agents** (BonBon agents and Bonobot orchestrators) +- **Routing** (policies, aliases, fallback chains) + +Validate: required fields present, provider types supported, model references resolve, no circular dependencies. + +## Step 2: Resolve Dependency Order + +1. Providers first (everything depends on them) +2. Knowledge bases second (agents may reference them) +3. Agents third (depend on providers + KBs) +4. Routing policies last (reference providers and models) + +## Step 3: Create Resources + +For each resource in order: +1. Call ~~bonito to create it +2. Verify it's healthy (test connection, check response) +3. Record the ID for downstream references + +If credentials are missing, prompt the user. Never store or log raw API keys. + +## Step 4: Verify and Report + +``` +## Deployment Report + +| Resource | Type | Status | +|----------|------|--------| +| [Name] | AWS Bedrock provider | Connected (12 models) | +| [Name] | Knowledge base | Indexed (42 chunks) | +| [Name] | BonBon agent | Deployed (85ms) | +| [Name] | Failover routing | Active | + +Total: [X] providers, [X] KBs, [X] agents, [X] routing policies +``` + +## Config Reference + +```yaml +# bonito.yaml +organization: my-company + +providers: + - name: aws-primary + type: aws-bedrock + region: us-east-1 + - name: openai-fallback + type: openai + +knowledge_bases: + - name: support-faq + documents: ./docs/faq/ + embedding_model: text-embedding-3-small + +agents: + - name: support-bot + type: bonbon + model: anthropic/claude-sonnet + provider: aws-primary + knowledge_base: support-faq + system_prompt: "You are a helpful customer support agent." + +routing: + - name: production + strategy: failover + targets: + - provider: aws-primary + model: anthropic/claude-sonnet + priority: 1 + - provider: openai-fallback + model: gpt-4o + priority: 2 +``` diff --git a/partner-built/bonito/skills/gateway-routing/SKILL.md b/partner-built/bonito/skills/gateway-routing/SKILL.md new file mode 100644 index 00000000..f42cb5a8 --- /dev/null +++ b/partner-built/bonito/skills/gateway-routing/SKILL.md @@ -0,0 +1,72 @@ +--- +name: gateway-routing +description: "Configure routing policies for the Bonito AI gateway. Failover chains, cost-optimized routing, A/B testing, model aliases, and cross-region inference. Triggers on 'set up failover', 'configure routing', 'add a fallback model', 'cross-region inference', 'create a model alias', or 'A/B test models'." +--- + +# Gateway Routing + +Configure intelligent request routing across AI providers. Handles failover, cost optimization, A/B testing, model aliases, and cross-region inference. + +## Step 1: Determine Strategy + +- "Set up failover" -> failover chain +- "Route to cheapest" -> cost-optimized +- "A/B test Claude vs GPT" -> ab-test +- "Load balance" -> round-robin / weighted +- "Create alias 'fast'" -> model alias +- "Cross-region" -> geo-routing + +## Step 2: Gather Requirements + +**Failover:** Primary provider/model, fallback(s), failure detection criteria (timeout, error codes). + +**Cost-optimized:** Quality threshold (minimum model tier), budget constraints, acceptable substitutes. + +**A/B test:** Models A and B with providers, traffic split percentage, test duration. + +**Model alias:** Alias name, target model and provider. + +## Step 3: Validate and Create + +1. List connected providers via ~~bonito +2. Verify requested models are available and healthy +3. Create the routing policy via ~~bonito +4. Set strategy, targets, priorities/weights, failover thresholds + +## Step 4: Verify + +1. Send test request -> confirm it routes to primary +2. For failover -> simulate failure -> confirm switch +3. For A/B -> send multiple requests -> confirm split ratio +4. Measure failover latency + +## Output Format + +``` +## Routing Policy: [Name] + +Strategy: [Failover / Cost-Optimized / A/B Test] +Status: Active + +| Priority | Provider | Model | Status | +|----------|----------|-------|--------| +| Primary | AWS Bedrock | claude-sonnet | Healthy | +| Fallback | OpenAI | gpt-4o | Healthy | +| Last resort | Groq | llama-3-70b | Healthy | + +Verification: +- Primary route: OK (85ms) +- Failover trigger: OK (<500ms switch) +``` + +## Strategies + +**Failover:** Route to primary. If it fails, try the next. Use when uptime matters most. + +**Cost-optimized:** Route to cheapest model meeting quality requirements. Tiers: budget (Groq Llama) -> standard (Claude Sonnet) -> premium (Claude Opus). + +**A/B testing:** Split traffic between models for comparison. Set duration and metrics (latency, quality, cost). + +**Round-robin / Weighted:** Distribute traffic across providers. Optional session affinity. + +**Geo-routing:** Route to nearest or lowest-latency region. Requires multi-region provider setup. diff --git a/partner-built/bonito/skills/manage-providers/SKILL.md b/partner-built/bonito/skills/manage-providers/SKILL.md new file mode 100644 index 00000000..46585869 --- /dev/null +++ b/partner-built/bonito/skills/manage-providers/SKILL.md @@ -0,0 +1,72 @@ +--- +name: manage-providers +description: "Connect, verify, and manage cloud AI providers through the Bonito gateway. Supports AWS Bedrock, Azure OpenAI, GCP Vertex AI, OpenAI, Anthropic, and Groq. Triggers on 'connect AWS Bedrock', 'add a provider', 'check my providers', 'list my providers', 'set up Azure OpenAI', or 'manage my AI providers'." +--- + +# Manage Providers + +Connect and manage cloud AI providers from one place. Handles adding providers, verifying credentials, listing available models, checking health, and removing unused connections. + +## Step 1: Identify the Action + +- "Connect AWS Bedrock" / "Add OpenAI" -> create provider +- "List my providers" / "Check my providers" -> list all +- "Is Bedrock healthy?" / "Check provider status" -> health check +- "Remove Groq" -> delete provider +- "Rotate my Anthropic key" -> update credentials + +## Step 2: Gather Credentials (for create) + +Collect required credentials based on provider type: + +| Provider | Required | +|----------|----------| +| AWS Bedrock | Access Key ID, Secret Access Key, Region | +| Azure OpenAI | API Key, Endpoint URL, API Version | +| GCP Vertex AI | Project ID, Region, Service Account JSON | +| OpenAI | API Key | +| Anthropic | API Key | +| Groq | API Key | + +Never log or display raw credentials. Prompt the user to provide via environment variables when possible. + +## Step 3: Create and Verify + +1. Call ~~bonito to create the provider with type and credentials +2. Test API authentication (valid credentials?) +3. Test API reachability (can we reach the endpoint?) +4. List available models on the provider +5. Measure baseline latency + +## Step 4: Present Results + +**For create:** + +``` +## Provider Connected: [Name] + +| Field | Value | +|-------|-------| +| Type | [AWS Bedrock / Azure OpenAI / etc.] | +| Region | [Region] | +| Status | Connected | +| Models | [Count] available | +| Latency | [X]ms | +``` + +**For list:** + +``` +| Provider | Type | Region | Status | Models | Latency | +|----------|------|--------|--------|--------|---------| +| [Name] | AWS Bedrock | us-east-1 | Healthy | 12 | 85ms | +| [Name] | OpenAI | — | Healthy | 8 | 120ms | +``` + +## Provider Notes + +- **AWS Bedrock:** Requires IAM permissions for `bedrock:InvokeModel` and `bedrock:ListFoundationModels`. Some models need explicit access grants. Cross-region inference available. +- **Azure OpenAI:** Models are deployed as named deployments. You need deployment names, not just model names. +- **GCP Vertex AI:** Service account needs `aiplatform.endpoints.predict`. Supports Gemini and Claude via Model Garden. +- **OpenAI/Anthropic:** Direct API access. Simplest setup. +- **Groq:** Optimized for fast inference with open-source models (Llama, Mixtral).