Skip to content

Enable non-existent tool fallback with MCP server #400

@danecor

Description

@danecor

Priority Level

Medium (Nice to have)

Is your feature request related to a problem? Please describe.

Right now, when using an MCP server, Data Designer immediately errors out and drops the row if a model attempts to call a non-existent tool. E.g.:

[03/12/26 02:43:16] INFO     Processing request of type            server.py:720
                             CallToolRequest                                    
[02:43:17] [WARNING] ⚠️ Generation for record at index 197 failed. Will omit this record from the dataset.
Tool 'CalcNumSaturatedHeterocycles' not found on any configured provider.

I've found that all tested models hallucinate tools sometimes, resulting in many failed records. For SDG, there are many cases where this is sub-optimal because it doesn't show the model the consequences of hallucinated tool calls.

Describe the solution you'd like

I would like to have an optional catch and fallback to alert the model that the tool doesn't exist. Ideally, we would provide a default error message which could be overridden by the user.

Describe alternatives you've considered

Right now, I inspect execution.log directly to identify when the model called hallucinated tools, since there is no statistics reporting on it and the row is dropped.

I don't have a workaround for recovering from hallucinated tool calls.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions