Skip to content

Conversation

@IlumCI
Copy link
Contributor

@IlumCI IlumCI commented Oct 13, 2025

Description: Fix MultiAgentRouter model compatibility issues with structured outputs and JSON parsing

Issue: #1137

Dependencies: none

Tag maintainer: @kyegomez

Twitter handle: https://x.com/IlumTheProtogen

Changes made:

  1. Added conditional structured outputs - only sets response_format for models that support json_schema (gpt-4.1, gpt-4o, o3-, o4-)
  2. Added JSON parsing fallback for models without structured outputs - routes to first available agent when JSON parsing fails
  3. Updated model names in test file to use correct LiteLLM conventions (anthropic/claude-, gemini/gemini-)

Why it didn't work last time:
The router was unconditionally setting response_format=MultipleHandOffsResponse for all models, causing BadRequestError on models like gpt-3.5-turbo that don't support json_schema structured outputs. Additionally, model names were using incorrect formats (claude-3-5-sonnet instead of anthropic/claude-3-5-sonnet-20241022) causing provider resolution failures.

Testing:

  • gpt-4.1, gpt-4o, o3-mini: Success with structured outputs
  • gpt-3.5-turbo: Success with JSON parsing fallback
  • Anthropic/Google models: Fail only due to missing API keys (expected behavior)

Results:

===== Model Run Summary =====
gpt-4.1: ✅ Success
gpt-4o: ✅ Success
gpt-3.5-turbo: ✅ Success
gpt-4.1: ✅ Success
o3-mini: ✅ Success
anthropic/claude-3-5-sonnet-20241022: ❌ Error: litellm.AuthenticationError: AnthropicException - {"type":"error","error":{"type":"authentication_error","message":"x-api-key header is required"},"request_id":"req_011CU5WbPhYDzgh6k3TCBmym"}```


<!-- readthedocs-preview swarms start -->
----
📚 Documentation preview 📚: https://swarms--1142.org.readthedocs.build/en/1142/

<!-- readthedocs-preview swarms end -->

}

if _supports_structured_outputs(self.model):
lite_llm_kwargs["response_format"] = MultipleHandOffsResponse

Check failure

Code scanning / Pyre

Incompatible parameter type Error

Incompatible parameter type [6]: In call dict.\_\_setitem\_\_, for 2nd positional argument, expected Union[float, str] but got Type[MultipleHandOffsResponse].
# Handle JSON parsing with fallback for models without structured outputs
try:
boss_response_str = orjson.loads(boss_response_str)
except (orjson.JSONDecodeError, TypeError):

Check failure

Code scanning / Pyre

Invalid except clause Error

Invalid except clause [66]: Exception handler type annotation unknown must extend BaseException.
"handoffs": [{
"reasoning": "Fallback routing due to JSON parsing error",
"agent_name": first_agent,
"task": task

Check failure

Code scanning / Pyre

Unbound name Error

Unbound name [10]: Name orjson is used but not defined in the current scope.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant