[MTP]: Timeout doesn't abort hanging tests

[cbersch]

Describe the bug

With the VSTest-based test execution it was quite easy to add a real™ timeout which could kill hanging tests.
The current --timeout doesn't work in that case.
Using

• .NET 10 RC2
• MSTest 4.0.1
• Windows 11

Steps To Reproduce

Consider the test

[TestClass]
public sealed class TimeoutTests
{
    [TestMethod]
    public void Lock()
    {
        ManualResetEventSlim mre = new(false);
        mre.Wait();
    }
}

with project

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>

  </PropertyGroup>

  <PropertyGroup Condition="'$(UseMtp)' == 'true'">
    <EnableMSTestRunner>true</EnableMSTestRunner>
    <OutputType>Exe</OutputType>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="MSTest.TestFramework" Version="4.0.1" />
    <PackageReference Include="MSTest.TestAdapter" Version="4.0.1" />
  </ItemGroup>

  <ItemGroup Condition="'$(UseMtp)' != 'true'">
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.0" />
  </ItemGroup>
</Project>

and global.json

{
  "test": {
    "runner": "Microsoft.Testing.Platform"
  }
}

MTP

dotnet test --project .\Timeout.csproj --timeout 10s -p:UseMtp=true

runs forever (also Ctrl+C has no effect).

VSTest

Delete the global.json and execute

dotnet test .\Timeout.csproj --blame-hang-timeout=10s

This

• "nicely" aborts the test run
• Reports a useful message.

The active test run was aborted. Reason: Test host process crashed

The active Test Run was aborted because the host process exited unexpectedly. Please inspect the call stack above, if available, to get more information about where the exception originated from.
The test running when the crash occurred:
TimeoutTests.Lock

This test may, or may not be the source of the crash.
  Timeout test net10.0 failed with 1 error(s) and 2 warning(s) (12,0s)
    C:\Program Files\dotnet\sdk\10.0.100-rc.2.25502.107\Microsoft.TestPlatform.targets(48,5): warning Data collector 'Blame' message: The specified inactivity time of 10 seconds has elapsed. Collecting hang dumps from testhost and its child processes.
    C:\Program Files\dotnet\sdk\10.0.100-rc.2.25502.107\Microsoft.TestPlatform.targets(48,5): warning Data collector 'Blame' message: Dumping 62032 - testhost.
    C:\temp\mtp_timeout\bin\Debug\net10.0\Timeout.dll : error TESTRUNABORT: Test Run Aborted.

• Creates a sequence file which contains the executed tests, including the one which was killed.

[Youssef1313]

This is by-design. --timeout only requests cancellation.
Can you try HangDump extension for MTP instead of --timeout?

See https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-extensions-diagnostics#hang-dump for information.

But in short, add Microsoft.Testing.Extensions.HangDump to your test projects, matching the same version of the Microsoft.Testing.Platform version you use, and run with dotnet test --project .\Timeout.csproj --hangdump --hangdump-timeout 10s -p:UseMtp=true

[cbersch]

This is by-design. --timeout only requests cancellation. Can you try HangDump extension for MTP instead of --timeout?

See https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-extensions-diagnostics#hang-dump for information.

Had missed that, thanks.

But in short, add Microsoft.Testing.Extensions.HangDump to your test projects, matching the same version of the Microsoft.Testing.Platform version you use, and run with dotnet test --project .\Timeout.csproj --hangdump --hangdump-timeout 10s -p:UseMtp=true

That doesn't work, tested with v 2.0.1:

[cbersch]

As a side note: Old option --blame-hang-dump-type had type full, mini and none (https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test?tabs=dotnet-test-with-vstest). I'm missing none, which I'm using, for the HangDump extension.

[Evangelink]

I don't recall all details, but I think none was only creating a text file and not a proper dump. This file is not so helpful with helping diagnose so we decided not to port it to MTP.

[cbersch]

I don't recall all details, but I think none was only creating a text file and not a proper dump.

That's exactly what I need. The blame-hang-timeout timeout is to guard against too long execution of a single test, but I don't want the actual dump which can be tens of GB.

[Evangelink]

@cbersch do you care about having trx etc? There is no good way to force kill an app and still leave room for extensions to properly complete tasks.

[cbersch]

@cbersch do you care about having trx etc?

No, I don't care about TRX or other extensions in that case.

But having a sequence file which reports the already executed tests and the test which hung, helps a lot. Would that be possible as part of the HangDump extension itself?