Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 7, 2025

MSTest 4.x introduced a breaking change where accessing TestContext.Properties with non-existent keys throws KeyNotFoundException instead of returning null as in 3.x. This affects both direct dictionary access and property accessors.

Root Cause

The issue had two parts:

  1. GetProperty method: Used conditional compilation with platform-specific behavior
    • WINDOWS_UWP/WIN_UI: Used TryGetValue (correct)
    • Other platforms: Used dictionary indexer Properties[name] (throws on missing keys)
  2. Direct dictionary access: Properties returned raw Dictionary<string, object?> which throws on missing keys

Changes

  • src/TestFramework/TestFramework.Extensions/TestContext.cs

    • Removed platform-specific behavior, unified to use TryGetValue across all platforms
    • Fixes property accessors like TestRunDirectory, DeploymentDirectory, ResultsDirectory
  • src/Adapter/MSTestAdapter.PlatformServices/Services/TestContextImplementation.cs

    • Added NullReturningDictionary wrapper class that implements IDictionary<string, object?>
    • Overrides indexer getter to return null for non-existent keys instead of throwing
    • All dictionary operations (Add, TryGetValue, indexer) delegate to underlying dictionary
    • Removed redundant _properties field - all operations go through the wrapper
    • Constructor uses local variable for initialization before wrapping
    • IsReadOnly property properly delegates to underlying dictionary
    • Added comprehensive documentation explaining backwards compatibility requirements
    • Updated Properties property to return the wrapper
    • Updated TryGetPropertyValue and AddProperty methods to use wrapper
  • Added comprehensive tests

    • Unit test: Verifies both direct dictionary access (Properties["NonExistent"]) and property accessors return null when unset
    • Acceptance test: Tests both direct dictionary access (Properties["NonExistent"]) and property accessors in end-to-end scenario

Example

// MSTest 3.x and now fixed 4.x
var value = testContext.Properties["NonExistent"];  // Returns null
var dir = testContext.TestRunDirectory;              // Returns null when not set

// MSTest 4.x before fix
var value = testContext.Properties["NonExistent"];  // Throws KeyNotFoundException
var dir = testContext.TestRunDirectory;              // Throws KeyNotFoundException
Original prompt

This section details on the original issue you should resolve

<issue_title>MSTest 4.x breaking change causes TestContext.Properties to throw KeyNotFoundException instead of returning null</issue_title>
<issue_description>## Describe the bug

During our migration from MSTest 3.x to 4.x, I've noticed a breaking change where calls to TextContext.Properties["NonExistent"] throw an exception instead of returning null. We have a lot of code that depends on this returning null.

Steps To Reproduce

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
    <EnableMSTestRunner>true</EnableMSTestRunner>
    <OutputType>Exe</OutputType>
    
    <!-- Set MSTestVersion property to switch versions: 3 or 4 (default) -->
    <MSTestVersion Condition="'$(MSTestVersion)' == ''">4</MSTestVersion>
  </PropertyGroup>

  <!-- MSTest 3.x -->
  <ItemGroup Condition="'$(MSTestVersion)' == '3'">
    <PackageReference Include="MSTest" Version="3.6.3" />
  </ItemGroup>

  <!-- MSTest 4.x (default) -->
  <ItemGroup Condition="'$(MSTestVersion)' == '4'">
    <PackageReference Include="MSTest" Version="4.0.2" />
  </ItemGroup>

</Project>
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace MSTestPropertyRepro;

[TestClass]
public class PropertyAccessTests
{
    public TestContext? TestContext { get; set; }

    [TestMethod]
    public void TestAccessingNonExistentProperty()
    {
        // Try to access a property that doesn't exist
        try
        {
            var nonExistent = TestContext?.Properties["NonExistentProperty"];
            Console.WriteLine($"NonExistentProperty returned: {nonExistent ?? "null"}");
            
            if (nonExistent == null)
            {
                Console.WriteLine("✓ MSTest 3.x behavior: Returns null for missing properties");
            }
            else
            {
                Console.WriteLine($"✗ Unexpected: Got value '{nonExistent}'");
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine($"✗ MSTest 4.x behavior: Throws exception for missing properties");
            Console.WriteLine($"Exception: {ex.GetType().Name} - {ex.Message}");
            throw;
        }
    }
}

MSTest 3.x

MSTest v3.6.3 (UTC 11/11/2024) [win-x64 - .NET 10.0.0]

Test run summary: Passed! - bin\Debug\net10.0\MSTestPropertyRepro.dll (net10.0|x64)
  total: 1
  failed: 0
  succeeded: 1
  skipped: 0

✅ Returns null when accessing non-existent properties - Test PASSES

MSTest 4.x

MSTest v4.0.0-preview.25358.7 (UTC 7/8/2025) [win-x64 - .NET 10.0.0]

Test run summary: Failed! - bin\Debug\net10.0\MSTestPropertyRepro.dll (net10.0|x64)
  total: 1
  failed: 1
  succeeded: 0
  skipped: 0
  
Exception: System.Collections.Generic.KeyNotFoundException: 
The given key 'NonExistentProperty' was not present in the dictionary.

❌ Throws KeyNotFoundException when accessing non-existent properties - Test FAILS

Breaking Change

// MSTest 3.x
var value = TestContext.Properties["NonExistent"]; // Returns null

// MSTest 4.x
var value = TestContext.Properties["NonExistent"]; // Throws KeyNotFoundException
```</issue_description>

<agent_instructions>Fix the implementation to return `null` instead of throwing if the `key` doesn't exist</agent_instructions>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits December 7, 2025 15:59
…-existent keys

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
…access

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix KeyNotFoundException in TestContext.Properties due to MSTest 4.x Fix TestContext property accessors throwing KeyNotFoundException instead of returning null Dec 7, 2025
Copilot AI requested a review from Evangelink December 7, 2025 16:04
Copilot AI and others added 4 commits December 7, 2025 16:21
…nstead of throwing

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI changed the title Fix TestContext property accessors throwing KeyNotFoundException instead of returning null Fix TestContext.Properties to return null instead of throwing KeyNotFoundException Dec 7, 2025
Copilot AI requested a review from Evangelink December 7, 2025 16:27
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI and others added 2 commits December 7, 2025 16:44
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
@Evangelink Evangelink marked this pull request as ready for review December 9, 2025 20:51
@Evangelink Evangelink enabled auto-merge (squash) December 9, 2025 20:51
@Evangelink Evangelink marked this pull request as draft December 10, 2025 09:00
auto-merge was automatically disabled December 10, 2025 09:00

Pull request was converted to draft

@Evangelink
Copy link
Member

Moving back to draft so we can discuss with @Youssef1313 as it seems the issue reported internally is different.

@nohwnd
Copy link
Member

nohwnd commented Dec 15, 2025

This is documented, and better behavior than before. And good time to do breaking change between major versions.

IDictionary also documents that key not found will throw exception, so hiding a type behind that interface that works differently is a change that will surprise new users.

@Evangelink Evangelink closed this Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MSTest 4.x breaking change causes TestContext.Properties to throw KeyNotFoundException instead of returning null

3 participants