Debugging HTTP functions with Specflow

[JohnKrlin]

I am able to successfully launch the function but when debugging in Visual Studio the debugger does not attach to the running function. Being able to debug the Specflow test is one of the major benefits of BDD testing.

I've spend several days trying to get this to work with Corvus and several other paths however am coming up short.

How can I debug the HTTP function from the specflow test via calls over HTTP?

[idg10]

The fundamental issue here is not Corvus-specific (which I would guess is why you've come up short despite investigating other paths too). The basic problems are this:

1. functions necessarily run in a separate process from the test runner because that's how the local Azure Functions emulator works
2. when the test runner process launches child processes (such as the Azure Functions emulator), a debugger attached to the parent (test runner) process will not automatically attach to the child process

Visual Studio does not have any built-in option to enable automatic attachment to child processes,. Microsoft has made an extension that does this available in the Marketplace—the Visual Studio 2022 version is available from https://marketplace.visualstudio.com/items?itemName=vsdbgplat.MicrosoftChildProcessDebuggingPowerTool2022

However, this relies on features that are only available to native debuggers, so if you want to use it with .NET projects, you need to debug the top-level project in mixed (both managed and unmanaged) mode.

I've yet to find a better way, unfortunately. And it is for this reason that we have started writing multi-mode tests, in which we execute the same .feature files in multiple ways. In particular, we have them execute code both in-process and via the emulator.

For example, look at https://github.com/marain-dotnet/Marain.Claims/blob/16ffa126dc23f48da787e4ca0147f4e23c665781/Solutions/Marain.Claims.OpenApi.Specs/Features/ClaimPermissionsEvaluation.feature - this defines various specifications for one of the services we write. The specs are written in terms of what a consumer of the service requires. (We use this service itself across multiple applications, which is why there is nothing application-domain-specific in these specs. Domain-specific tests would belong in the individual applications repos.)

Obviously it's important that the real service is able to meet all these specs so we want to run them against the hosted service, and we do. But we also want to be able to debug these tests with ease, and as you're finding, that's not something that VS makes particularly practical.

So we execute these specs multiple times in different ways. If you look at https://github.com/marain-dotnet/Marain.Claims/blob/16ffa126dc23f48da787e4ca0147f4e23c665781/Solutions/Marain.Claims.OpenApi.Specs/Features/ClaimPermissions.feature.multi.cs you'll see we add a partial class to make the specflow-generated class derive from a base called MultiTestHostBase. And the effect of this is to cause multiple instances of each test to appear. It will produce an instance of each test for everything in this list: https://github.com/marain-dotnet/Marain.Claims/blob/16ffa126dc23f48da787e4ca0147f4e23c665781/Solutions/Marain.Claims.OpenApi.Specs/MultiHost/MultiTestHostBase.cs#L46-L48

The effect of this is that for each feature file, the test explorer shows three entries:

Only the UseFunctionHost ones actually fire up the functions host. The other two run in-process meaning there are no complications with debugging. The reason for having two in-process modes is that it's sometimes useful to be able to debug in-process but still invoke the relevant code via HTTP so that we can test that all the handling at that level works as expected, but in situations where that's not where the problem lies, it's easier just to invoke the code under test directly. Also, by avoiding HTTP entirely, those tests better support parallelization, so those of us using NCrunch configure the runner to execute those tests continually and in parallel, but not the ones that will attempt to listen on a fixed HTTP port.

Currently we're putting this multi-mode testing code directly into projects that require it, but at some point we're really hoping to be able to move it out into Corvus.Testing to make it reusable. However, SpecFlow currently doesn't make this easy. They seem to have taken a position that nobody should be doing this sort of thing (specifically, running the same specs multiple times in different ways) so we have to get a little devious to make it work, and it's not yet clear whether there's a reliable way to package this up for reuse.