Feature Request: Add Amazon Bedrock as LLM Provider

[obue]

Summary

Add support for Amazon Bedrock as an LLM provider in OpenFang, with authentication via Bedrock's long-term API keys.

Motivation

Amazon Bedrock provides access to foundation models from leading AI companies (Anthropic Claude, Amazon Titan, Meta Llama, Cohere, AI21 Labs, Stability AI, and more) through a unified API. Many users prefer Bedrock for:

• Unified access to multiple model providers through a single service
• Enterprise features like VPC deployment, encryption, and compliance certifications
• Regional availability including EU regions (eu-central-1, eu-west-1, etc.) for data residency requirements
• Simplified billing through AWS consolidated billing

Currently, OpenFang supports 26 LLM providers but lacks Bedrock support, requiring users to set up proxy solutions like LiteLLM as a workaround.

Proposed Solution

Authentication

Support Bedrock's long-term API keys (introduced July 2024) using Bearer token authentication:

rust
// Example authentication header
Authorization: Bearer

API endpoint format:

https://bedrock-runtime.{region}.amazonaws.com/model/{model-id}/converse

Configuration Example

toml
[llm.bedrock]
enabled = true
api_key = "${AWS_BEARER_TOKEN_BEDROCK}"
region = "eu-central-1"
default_model = "eu.anthropic.claude-sonnet-4-6"

Supported Models

• Anthropic Claude (Opus, Sonnet, Haiku)
• Amazon Titan
• Meta Llama
• Cohere Command
• AI21 Jurassic
• Mistral AI
• And more as AWS adds them

API Compatibility

Bedrock supports multiple API formats:

• /converse - Unified conversation API (recommended)
• /invoke - Model-specific invocation
• Streaming via /invoke-with-response-stream

Alternatives Considered

1. Use AWS IAM credentials - More complex setup, requires AWS SDK integration
2. Use LiteLLM proxy - Current workaround, adds latency and complexity
3. Use OpenRouter - Doesn't support private VPC deployments or EU-specific regions

Additional Context

• AWS Bedrock API Keys documentation: https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-how.html
• Long-term keys are IAM service-specific credentials scoped only to Bedrock
• Keys can be set with expiration dates or never expire
• Bedrock is available in 10+ AWS regions globally

Implementation Notes

The provider implementation would need to:

1. Handle Bearer token authentication (simpler than AWS SigV4)
2. Support region-specific endpoints
3. Map Bedrock's response format to OpenFang's internal format
4. Handle streaming responses for real-time output
5. Support model-specific parameters (temperature, max_tokens, etc.)

Benefits

• Native Bedrock support without proxy overhead
• Better performance (direct API calls)
• Simplified configuration for Bedrock users
• Access to EU-hosted models for GDPR compliance
• Enterprise-grade security and compliance features

[obue]

Updated Analysis: Amazon Bedrock Bearer Token Support

Current State (as of March 2, 2026)

✅ What Already Exists

OpenFang already has Bedrock support with 8 models in the catalog:

Anthropic Models:

• bedrock/anthropic.claude-opus-4-6 - Claude Opus 4.6 (Frontier)
• bedrock/anthropic.claude-sonnet-4-6 - Claude Sonnet 4.6 (Smart)
• bedrock/anthropic.claude-opus-4-20250514 - Claude Opus 4 (Frontier)
• bedrock/anthropic.claude-sonnet-4-20250514 - Claude Sonnet 4 (Smart)
• bedrock/anthropic.claude-haiku-4-5-20251001 - Claude Haiku 4.5 (Fast)

Amazon Models:

• bedrock/amazon.nova-pro-v1:0 - Amazon Nova Pro (Smart)
• bedrock/amazon.nova-lite-v1:0 - Amazon Nova Lite (Fast)

Meta Models:

• bedrock/meta.llama3-3-70b-instruct-v1:0 - Llama 3.3 70B (Balanced)

Location: crates/openfang-runtime/src/model_catalog.rs lines 2710-2823

❌ What's Missing

The current implementation uses AWS IAM credentials (AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY) which require AWS SigV4 request signing.

It does NOT support Bedrock's long-term API keys with Bearer token authentication (introduced July 2024).

The Problem

Current Authentication Method

// crates/openfang-runtime/src/model_catalog.rs:608
ProviderInfo {
    id: "bedrock".into(),
    display_name: "AWS Bedrock".into(),
    api_key_env: "AWS_ACCESS_KEY_ID".into(),  // ← Expects IAM credentials
    base_url: BEDROCK_BASE_URL.into(),        // ← Fixed to us-east-1
    key_required: true,
    auth_status: AuthStatus::Missing,
    model_count: 0,
}

Issues with Current Implementation

1. Wrong authentication method: Expects IAM credentials, not Bearer tokens
2. Fixed region: Base URL hardcoded to us-east-1 (no EU region support)
3. Complex setup: Requires AWS SDK integration for SigV4 signing
4. No documentation: Bedrock not mentioned in docs/providers.md

Proposed Changes

1. Update Provider Configuration

File: crates/openfang-runtime/src/model_catalog.rs (line ~608)

ProviderInfo {
    id: "bedrock".into(),
    display_name: "AWS Bedrock".into(),
    api_key_env: "AWS_BEDROCK_API_KEY".into(),  // ← Change to Bearer token
    base_url: BEDROCK_BASE_URL.into(),
    key_required: true,
    auth_status: AuthStatus::Missing,
    model_count: 0,
}

2. Add Region Support

File: crates/openfang-types/src/model_catalog.rs (line ~44)

// Current:
pub const BEDROCK_BASE_URL: &str = "https://bedrock-runtime.us-east-1.amazonaws.com";

// Proposed: Make region configurable via env var or config
// Default to us-east-1, allow override via AWS_BEDROCK_REGION

3. Update Environment Variables

File: .env.example

Add:

# AWS Bedrock (Bearer token authentication)
# AWS_BEDROCK_API_KEY=your-bearer-token-here
# AWS_BEDROCK_REGION=us-east-1  # or eu-central-1, eu-west-1, etc.

4. Add Documentation

File: docs/providers.md

Add Bedrock section with:

• How to create long-term API keys in AWS Console
• Available regions (especially EU regions for GDPR compliance)
• Model list with pricing
• Configuration examples

5. Update Configuration Example

File: openfang.toml.example

Add example:

# AWS Bedrock with Bearer token
# [default_model]
# provider = "bedrock"
# model = "bedrock/anthropic.claude-sonnet-4-6"
# api_key_env = "AWS_BEDROCK_API_KEY"
# base_url = "https://bedrock-runtime.eu-central-1.amazonaws.com"

Why This Matters

Benefits of Bearer Token Authentication

1. Simpler setup: No AWS SDK, no SigV4 signing complexity
2. Better security: Scoped credentials (Bedrock-only, no other AWS access)
3. EU compliance: Easy region selection for GDPR requirements
4. Expiration control: Keys can be set to expire or never expire
5. Direct API calls: No proxy needed, better performance

Use Cases

• EU customers: Need eu-central-1 or eu-west-1 for data residency
• Enterprise: VPC deployment, encryption, compliance certifications
• Unified billing: AWS consolidated billing across services
• Multi-model access: Single provider for Anthropic, Amazon, Meta models

Implementation Complexity

Effort: LOW ⭐

The OpenAI driver already handles Bearer tokens correctly:

// crates/openfang-runtime/src/drivers/openai.rs:302
if !self.api_key.as_str().is_empty() {
    req_builder = req_builder
        .header("authorization", format!("Bearer {}", self.api_key.as_str()));
}

Required changes:

1. Update api_key_env string (1 line)
2. Add region configuration logic (~20 lines)
3. Update documentation (~50 lines)
4. Update example configs (~10 lines)

Total: ~80 lines of changes

Testing Checklist

• Bearer token authentication works with Bedrock API
• Region override via AWS_BEDROCK_REGION env var
• Region override via base_url in config
• All 8 Bedrock models accessible
• Streaming responses work
• Tool calling works (Claude models)
• Vision support works (Claude, Nova models)
• Cost tracking accurate
• EU regions work (eu-central-1, eu-west-1)
• Error messages clear when key missing/invalid

Alternative: Support Both Methods

Could support both authentication methods:

1. Bearer token (new): AWS_BEDROCK_API_KEY → Simple Bearer auth
2. IAM credentials (existing): AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY → SigV4 signing

Check for Bearer token first, fall back to IAM if not present.

Pros: Maximum flexibility, no breaking changes
Cons: More complex, requires AWS SDK for SigV4

Recommended Approach

Start simple: Implement Bearer token support only.

1. Change api_key_env to AWS_BEDROCK_API_KEY
2. Add region configuration
3. Document the setup process
4. Add to providers guide

If users need IAM credential support later, add it as a separate feature.

References

• AWS Bedrock API Keys docs: https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-how.html
• Bedrock Converse API: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html
• Available regions: us-east-1, us-west-2, eu-central-1, eu-west-1, eu-west-3, ap-southeast-1, ap-southeast-2, ap-northeast-1, ca-central-1, sa-east-1

Code Locations

Key files to modify:

• crates/openfang-runtime/src/model_catalog.rs (line 608) - Provider config
• crates/openfang-types/src/model_catalog.rs (line 44) - Base URL constant
• docs/providers.md - Add Bedrock documentation section
• .env.example - Add Bedrock env vars
• openfang.toml.example - Add Bedrock config example

The OpenAI driver (crates/openfang-runtime/src/drivers/openai.rs) requires no changes - it already supports Bearer tokens correctly.