Skip to content

fix: inject fallback model name into CompletionRequest#202

Open
voidborne-d wants to merge 1 commit intoRightNow-AI:mainfrom
voidborne-d:fix/fallback-model-override
Open

fix: inject fallback model name into CompletionRequest#202
voidborne-d wants to merge 1 commit intoRightNow-AI:mainfrom
voidborne-d:fix/fallback-model-override

Conversation

@voidborne-d
Copy link

Problem

When FallbackDriver retries a request with a different provider, request.model still carries the primary model name. This causes the fallback provider to receive an invalid model ID (e.g., sending claude-3-opus to an OpenAI endpoint).

Fixes #200

Root Cause

FallbackDriver cloned the request as-is for every driver in the chain. The model field was never updated to match the fallback entry's configured model.

Solution

  • Added FallbackEntry struct that pairs each driver with an optional model override
  • New FallbackDriver::with_models() constructor accepts Vec<(Arc<dyn LlmDriver>, String)>
  • Both complete() and stream() now inject the correct model before forwarding
  • Updated both kernel boot path and resolve_driver() to use with_models()
  • Existing FallbackDriver::new() is preserved for backward compatibility
  • Added test test_with_models_overrides_model_on_fallback verifying the override

Changes

  • crates/openfang-runtime/src/drivers/fallback.rs — new FallbackEntry, with_models(), model injection in complete()/stream()
  • crates/openfang-kernel/src/kernel.rs — both fallback construction sites now pass model names

When FallbackDriver retries with a different provider, it now correctly
overrides request.model with the fallback entry's model name instead of
forwarding the primary model name unchanged.

Fixes RightNow-AI#200
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: fallback_models switches provider but not model

1 participant