Customer locally hosted LLM gateway provider support

[pradip-allwyn]

Feature Request

There are a bunch of locally hosted LLM providers that are available, e.g., litellm, Helicone , Envoy AI Gateway, and Bifrost.

They host both LLM and MCP gateways. It would be good to have a customer and a generic way to add the gateway provider.

Motivation

to host the entire infra internally, for LLMs, e.g., vLLM, gateway, langflow. etc.

Your Contribution

No response

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!