-
Notifications
You must be signed in to change notification settings - Fork 1.3k
When no providers are available, reduce all cooldowns by 1 minute. #3298
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Current
|
| if (this.isRateLimitError(error) || this.isOverloadError(error)) { | |
| // Set cooldown for the current provider | |
| await this.usage.setCooldown(this.activeProfileId, 10 * 60 * 1000) | |
| // Switch to a different provider | |
| await this.adjustActiveHandler("Retryable Error") | |
| // Retry the request with the new provider | |
| yield* this.createMessage(systemPrompt, messages, metadata) | |
| return | |
| } | |
| // For non-rate limit errors, set cooldown and rethrow | |
| await this.usage.setCooldown(this.activeProfileId, 10 * 60 * 1000) | |
| throw error |
|
This is a solid idea. |
Context
Fallback recovery: When no providers are available, reduce all cooldowns by 1 minute.
This allows the next adjustActiveHandler() call to potentially find an available provider without waiting for the full cooldown duration.
Implementation
When all providers are unavailable.
Reduce each provider's cooldown by 1 minute to progressively unblock them.
This creates a "soft recovery" mechanism where providers gradually become available rather than waiting for full cooldown expiry.