How to best support hundreds or thousands of abilities

[JasonTheAdams]

The challenge of tons of tools

AI is funny in that it works like magic sometimes and other times will completely fall apart counterintuitively. One example of this is tools — whether direct calls functions on a model level or tools from an MCP level — wherein the model's ability to select the right tool for the job is finicky. Ironically, the more tools the AI has to work with the less likely it is to know which tool to use. Less is more in this case.

We're introducing the Abilities API in 6.9, which will be the foundation for how folks surface functionality to AI, which will be utilities by the MCP Adapter, at least. It's defensively best to assume that people will register a lot of abilities. I could imagine sites with dozens of plugins totaling hundreds (or thousands) of abilities. If these abilities are simply surfaced 1-to-1 as tools within the MCP server, it will not go over well.

Tentative solution: Layered Tool Pattern

One example of an approach to this is the Square MCP Server, which has three tools:

• get_service_info — which is used for discovering the services available
• get_type_info — which provides parameter requirements for a given service's method
• make_api_request — which executes a service's method

This is described as a Layered Tool Pattern. Basically, the surface area of the API for the AI starts very small and expands via discovery. This makes the underlying tools more like a choose-your-own-adventure than a massive diner menu.

Ability Layers

Given the sheer number of abilities, we likely first need a taxonomy for breaking them down further, likely a category. The category would have a name and description. This list would be provided as part of the system prompt, and would hopefully be at most dozens of terms.

The AI would then be given three tools:

• get_abilities_by_category — which receives a category, and returns a name/description list of abilities
• get_ability_info — which receives a category and ability, and returns the parameters for the ability
• use_ability — which takes a category, ability, parameters objects, and returns the output of the ability

[JasonTheAdams]

The above is intended to frame the issue and propose a solution. Let's discuss the nuances of the issue, make sure it's accurate, and then the merits of this or another idea. 😄

[swissspidy]

At first glance I think this pattern would work really well for exposing REST API routes via MCP (like Square does).

For abilities perhaps too, but it needs careful consideration for getting the UX right. If someone is getting started with MCP in WordPress and they need 3 tool calls just to do a basic task, that's not great. Also need to find the balance between this approach and other kinds of categorization, for example potential features being discussed in MCP itself, see modelcontextprotocol/modelcontextprotocol#1300

I could imagine sites with dozens of plugins totaling hundreds (or thousands) of abilities.

Good reminder that we'll have to have really good documentation to set expectations. Thousands of abilities would cause issues with a lot of things, not just MCP.

[JasonTheAdams]

Thanks, @swissspidy!

and they need 3 tool calls just to do a basic task, that's not great

I'm not sure what you mean by this. Yes, the AI would make three tool calls, but with a couple of notes:

1. The user shouldn't really notice; it's just part of the AI's "thinking" moment
2. The context will build, so the AI won't need to discover things it's already learned about, reducing subsequent calls

Also need to find the balance between this approach and other kinds of categorization, for example potential features being discussed in MCP itself, see modelcontextprotocol/modelcontextprotocol#1300

Thanks for linking to this! The trick is the point of issue is really quick. The OP in that Issue you linked says it well: "When more than about 10 tools are presented to an LLM it can become confused, and quality of tool selection suffers." This really means that we can't have a 1-to-1 Ability to Tool, as the built-in WP abilities alone would exceed this.

I think the abstraction of adding a required category when registering an ability will help with this. If, initially, we use the Layered Tool Pattern and then later update to a version of the MCP spec that supports taxonomies, having the category will help and wouldn't require further action on the part of the implementors. Seeing this proposal and discussion encourages me that we're thinking in the right direction!

[gziolo]

Some related discussion is also happening in:

• WordPress Core Abilities #40

@Jameswlepage started the conversation about which abilities should be added to WordPress core. That also requires defining the level of granularity we want to promote from the start, and setting good examples of how to scale the registry.

[justlevine]

Thousands of abilities would cause issues with a lot of things, not just MCP.

&

This really means that we can't have a 1-to-1 Ability to Tool, as the built-in WP abilities alone would exceed this.

I think the abstraction of adding a required category when registering an ability will help with this.

Really not a fan of using MCP in its current stage to infer the needs of a generic, holistic abstraction. Especially when it's due to (relatively) short term considerations like LLMs struggling to determine what is the correct tool to use.

For now, I recommend we deal with MCPs limitations as an implementation detail inside MCP Adapter, where we can discuss/iterate on layered toolcalls, categorization (via nested namespaces, metadata -including a potential $meta['category'], db table with embeddings, or any number of other approaches) , etc as the spec/usage patterns/models themselves continue to evolve.

And independently we iterate on discoverability patterns for the Abilities registry that make sense for an abstraction in the context of Command Palette, REST (in all its permutations), plugin ecosystem inference, in addition to the other back/future-compat considerations we should take into account as we work towards our 6-week merge target.

[JasonTheAdams]

Really not a fan of using MCP in its current stage to infer the needs of a generic, holistic abstraction. Especially when it's due to (relatively) short term considerations like LLMs struggling to determine what is the correct tool to use.

I'll push back on this a bit. 😄

I don't think this is something specific to MCP. When working directly with a model's API "tools" take the form of function declarations and function calls. If we made a chat box, for example, the abilities would translate to function declarations. As far as I know, the models run into the same issue (and I would imagine around the same threshold) as the number of declarations increases.

Right now we're watching Providers such as OpenAI continue to work on model and function selection (as seen in GPT-5), but still with mixed success. This clues to me that it's not a simple solution — otherwise someone would have knocked this out already. So I'm hesitant to wait with bated breath for this to be resolved, especially to the point where a solution like the one being described here is problematic — which I'm not sure it ever will be.

For now, I recommend we deal with MCPs limitations as an implementation detail inside MCP Adapter, where we can discuss/iterate on layered toolcalls, categorization (via nested namespaces, metadata -including a potential $meta['category'], db table with embeddings, or any number of other approaches) , etc as the spec/usage patterns/models themselves continue to evolve.

I'm cool with tackling this in the specific context of MCP and various implementations. My goal in surfacing this is twofold:

1. To make sure we are aware of this issue, as it could be very easy to miss in small-scale local testing
2. To make sure the Abilities API has the necessary abstractions and requirements to make this effectively possible

Regarding the Abilities API, my concern is that if we make this too tucked away and optional then users will simply ignore it, undermining making our beautiful efforts to handle this.

Appreciate the great thoughts on this, folks! 😊

[swissspidy]

Relatedly, this was just shared on Slack: https://github.com/github/github-mcp-server/blob/main/docs/remote-server.md