[LangChain v1] LLMToolSelectorMiddleware selects a tool that is not in the provided tool list. seems like the middleware is not receiving the correct tool list. #33651
Description
Checked other resources
- This is a bug, not a usage question.
- I added a clear and descriptive title that summarizes this issue.
- I used the GitHub search to find a similar question and didn't find it.
- I am sure that this is a bug in LangChain rather than my code.
- The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
- This is not related to the langchain-community package.
- I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example).
- I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.
Example Code
Tool Definition
from langchain_tavily import TavilySearch
from langchain_community.utilities import GoogleSerperAPIWrapper
from langchain.tools import tool
# --- Google Serper --------------------------------------------------------
search = GoogleSerperAPIWrapper()
@tool('serper_search', description='Use only when you must retrieve up-to-date factual information (e.g., news, company financials). Do NOT use for definitions, conceptual questions, or theoretical explanations. (especially good for Korean language information search)')
def serper_search_tool(query: str) -> str:
"""
Search the web using the Google Serper API.
Use this when you need up-to-date factual or real-world information.
"""
return search.results(query)['organic'][:5]
# --- Tavily Search Tool ---------------------------------------------------
tavily_search_tool = TavilySearch()Creating Agent
from langchain.agents import create_agent
from langchain.agents.middleware import TodoListMiddleware, LLMToolSelectorMiddleware
competitor_analyst = create_agent(
model='google_genai:gemini-2.5-flash',
tools=[serper_search_tool, tavily_search_tool],
middleware=[TodoListMiddleware(), LLMToolSelectorMiddleware(model='google_genai:gemini-2.5-flash')]
)Invocation
from langchain.messages import HumanMessage
user_prompt = f'''You are a strategy planning assistant at NVIDIA.
Your main task is to analyze **competitors' current strategies** and **potential threats**
based on the given 10-K summary and additional web research.
Please provide URLs for all referenced materials.
IMPORTANT: Your response must be strictly based on the given report or verified web sources.
Do not rely on prior knowledge or assumptions.
Below is the summary report to analyze for NVIDIA threats:
<10-K Analysis Summary>
{summary_10k}
</10-K Analysis Summary>
You must start by **writing a task plan** (using write_todos)
and include at least one **web search task** in that plan.
'''
result = competitor_analyst.invoke(
{'messages': [HumanMessage(content=user_prompt)]},
config={'recursion_limit': 50}
)Invocation Params
{
"model": "models/gemini-2.5-flash",
"temperature": 0.7,
"top_k": null,
"n": 1,
"safety_settings": null,
"response_modalities": null,
"thinking_budget": null,
"include_thoughts": null,
"_type": "chat-google-generative-ai",
"stop": null,
"tools": [
{
"type": "function",
"function": {
"name": "ToolSelectionResponse",
"description": "Use to select relevant tools.",
"parameters": {
"properties": {
"tools": {
"description": "Tools to use. Place the most relevant tools first.",
"items": {
"anyOf": [
{ "const": "write_todos" },
{ "const": "serper_search" },
{ "const": "tavily_search" }
]
},
"title": "Tools",
"type": "array"
}
},
"required": ["tools"],
"type": "object",
"parameters": {}
}
}
}
],
"tool_choice": "ToolSelectionResponse"
}
Error Message and Stack Trace (if applicable)
Error Message
[/usr/local/lib/python3.12/dist-packages/langchain/agents/middleware/tool_selection.py](https://localhost:8080/#) in _process_selection_response(self, response, available_tools, valid_tool_names, request)
229 if invalid_tool_selections:
230 msg = f"Model selected invalid tools: {invalid_tool_selections}"
--> 231 raise ValueError(msg)
232
233 # Filter tools based on selection and append always-included tools
ValueError: Model selected invalid tools: ['Web_Search_Tool']
Unexpected Behavior
The middleware sometimes attempts to call tools not included in the tools argument (e.g., tools from other sessions or internal defaults).
This occurs intermittently and is reproducible when calling the same agent multiple times in a single runtime.
It seems that the LLMToolSelectorMiddleware either:
- Receives a stale or incorrect tool list from a previous agent instance, or
- Is referencing an internal cached list instead of the provided runtime
tools.
Expected Behavior
The LLMToolSelectorMiddleware should only reference the tools explicitly provided to create_agent.
It should not attempt to call or rank tools outside the declared tool list.
Description
When using LLMToolSelectorMiddleware with a Gemini model, it occasionally selects a tool that is not included in the provided tools list during agent creation. It seems that the middleware may not be receiving the correct tool list or is referencing an outdated internal tool registry.
Below is the full minimal reproducible example.
System Info
Runtime
{
"sdk": "langsmith-py",
"sdk_version": "0.4.37",
"library": "langchain-core",
"platform": "Linux-6.6.105+-x86_64-with-glibc2.35",
"runtime": "python",
"py_implementation": "CPython",
"runtime_version": "3.12.12",
"langchain_version": "1.0.1",
"langchain_core_version": "1.0.0",
"library_version": "1.0.0"
}