Skip to content

[Bug]: testRunnerEnv tool parameter schema doesn't work with OpenAI #103

@eastlondoner

Description

@eastlondoner

Bug Description

The testRunnerEnv schema doesn't work with OpenAI because their json schema implementation doesn't support additionalProperties.

The test_* tools have a testRunnerEnv parameter with this schema (take from mcp inspector)

testRunnerEnv:{
  type: "object"
  additionalProperties:{
    type: "string"
  }
}

This is (basically) correct json schema for a { string: string } dictionary BUT additionalProperties is not supported by OpenAI's function calling so tools like Cursor simply omit this when OpenAI is being used. In Cursor this manifests as testRunnerEnv field being "hidden" from the AI and it's always empty IF you use it with an OpenAI model but it works fine with Claude models.

You can workaround the platform limitations by changing the testRunnerEnv to an array of key: string, value: string objects.

Cursor (and other client implementations) can also work around this by doing that transformation (and reverse transformation) but in the meantime you may want to do the workaround yourself, or document this issue.

In this example my cursor rules have instructions to always set

"testRunnerEnv": {
    "runsForEachTargetApplicationUIConfiguration": "NO"
  },
Image

Claude vs Chat GPT continued

Image

Debug Output

NA

Editor/Client

Cursor 1.5.5

MCP Server Version

1.12.3

LLM

GPT-5 / Sonnet-4

MCP Configuration

Steps to Reproduce

Ask the agent to call test_sim with testRunnerEnv set to { "DEV_MODE": "YES"}, switch between gpt-5 and sonnet-4

Expected Behavior

testRunnerEnv should be set regardless of model used

Actual Behavior

when gpt 5 is used testRunnerEnv is not set

Error Messages

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions