Exposing the built-in jest config

[danny12321]

Command

config

Description

Expose the jest config used by angular so third party apps can use it. For example: StrykerJS needs this config to run mutation testing on angular project.

Describe the solution you'd like

It would be nice to have something simular as npx ng generate config karma but than for jest.

Describe alternatives you've considered

I tried reverse engineer the config but it was quite complex and may be prone to change in the future which increases the potential of errors with new releases.

[dgp1130]

it was quite complex and may be prone to change in the future which increases the potential of errors with new releases.

This is exactly why the Jest config is not exposed right now. We've talked about this a good amount on the team to understand the trade-offs being made here. The challenge is that Angular CLI has to make a lot of assumptions about Jest's setup and configuration to provide a reliable experience and for us as developers to effectively reason about what your Jest execution actually looks like and how we can improve it. This is easy to do when we control that configuration and can specify it exactly. When we start allowing user-specific configurations, it becomes very easy for users to break that configuration and run into confusing edge cases.

One of the primary goals of the Angular CLI is to solve these kinds of tooling and configuration problems so users don't have to deal with them as much as possible. We typically try to enable common web development use cases directly and provide a holistic experience as much as we can, rather than implementing a minimal core and asking users to fill in the blanks. To that end, one area we're experimenting with here is trying to see how far we can go without needing to expose the Jest configuration. Is it possible for Angular CLI to support most common testing use cases through other means without having to expose the Jest config?

As one example, I expect some users might think "I want to modify the Jest configuration so I can measure code coverage". That's certainly one approach, but code coverage is a pretty common task I would much rather just support code coverage in Angular CLI directly and say "Just use that". Let Angular CLI handle the Jest configuration for you and solve the problem more directly. We don't yet have support for code coverage in the current Jest implementation, but it is something we're hoping to add following this mindset.

Jest support is still experimental, and I think the answer to the above question remains to be seen. What I think would be helpful here is understanding everyone's motivations for modifying the Jest configuration. Mutation testing is one such use case. I'm not terribly familiar with it and will need to do more research to make sure this kind of approach would be the right way of integrating it. But I would love to hear from the rest of the community about what problems you all are trying to solve. If we can collect enough use cases here, I think it will greatly help to answer the above question and understand whether or not exposing the Jest config is truly the best path forward for the Angular ecosystem.

[nicojs]

@dgp1130, thanks for your response.

Believe me when I say: we understand. I think using something like npx ng generate config jest is a suboptimal approach. I would rather have something like import jestConfig from '@angular-devkit/build-angular/src/builders/jest/config.mjs'. That way, we have the best of both worlds: a great user experience and a way for plugin creators to get at the jest config.

An alternative:

import { createJestConfig } from '@angular-devkit/build-angular';

const projectName = 'app';
const jestConfig = await createJestConfig(projectName);

[nicojs]

@dgp1130 would you agree with this approach? I would be happy to implement this in a PR 👍

[johncrim]

@dgp1130 - I agree with the intent of automating/hiding the jest config. I've certainly found it to be brittle when working with Angular, and particularly when upgrading to ESM. I think there's a lot of value in wrapping/hiding it in most cases. The question is whether completely hiding it without providing an escape hatch is sufficient in 100% of cases.

Let me try to add a little value by inventorying what we've had to configure in jest.config.js and in karma.conf.js. (I'm including karma just b/c Angular does support the config file, and it's worth considering why it's needed):

karma.conf.js settings:

• plugins (reporters, coverage plugins, browser launchers)
• Reporter settings
• Launcher settings
• log level
• single run
• restart on file change

jest.config.js settings:

• Things presumably handled by the builder:
  • roots
  • testMatch glob pattern
  • transform
• testPathIgnorePatterns
• moduleDirectories
• moduleFileExtensions
• moduleNameMapper - Fixup all the packages that don't have correct node ESM packaging
• setupFilesAfterEnv

Things that we've needed to pass to jest on the command-line:

• environment variables
• switches like coverage and reporter parameters
• Run test on single file or path pattern
• Debug watch

Hope that helps.

[dgp1130]

@nicojs, that's definitely an approach to consider, but I think it still has the same challenges of exposing the underlying implementation details (albeit in a more constrained and less accessible capacity). For example, one aspect we're experimenting with right now is pre-building tests with esbuild prior to passing them through to Jest. This means Jest is operating on pure JavaScript files and performs no transformations whatsoever. If a user then tries to add a custom transform expecting TypeScript files, it's not going to find anything. Future changes to aspects of the build process become observable to custom plugins and break them in a hard to anticipate way. Basically any change is a potential breaking change and it becomes increasingly difficult to do anything with the system outside of major versions.

There is potentially some API design here which could minimize that risk such as allowing customization of specific pieces of the Jest config as managed by Angular. A strawman example for this might look like:

import { runJest } from '@angular-devkit/build-angular';

runJest(['**/*.spec.ts'], {
  // Override specific, supported pieces of the Jest config.
  moduleNameMapper: { /* ... */ },
});

That's definitely not perfect either, but it's an approach to consider which might serve as a useful middle ground for supplying some constraints to the system without being completely closed off to extensions.

At this stage though I think the most helpful thing would just be to understand use cases for modifying the Jest config and if/how it would make sense for Angular CLI to meet those needs (whether that includes exposing the Jest config or addressing the problem through a different means).

[nicojs]

Hi @dgp1130 🙋‍♂️, thanks for the help 🙏

Wouldn't it be possible to run esbuild using the jest transformer API?

If running esbuild from a jest transformer API is not possible, then a drop-in replacement for jest would be nice to work with. However, I would like to point you to a new programmatic API I've been working on. Preferably, it would use that API:
jestjs/jest#14062

So something like this:

import {createJest} from '@angular-devkit/build-angular';

const jest = await createJest();
jest.globalConfig = {
  ...jest.globalConfig,
  // Override specific, supported pieces of the Jest config.
  collectCoverage: false,
};
const {results} = await jest.run();
console.log(`run success, ${result.numPassedTests} passed tests.`);

[dgp1130]

Wouldn't it be possible to run esbuild using the jest transformer API?

Possibly, though Angular's build process is a lot more complicated than just running esbuild. Angular CLI includes a complete build system for orchestrating work like this, and I think we would rather decouple the building and testing parts of the infrastructure as much as possible so Jest doesn't need to have knowledge of Angular's build processes. That does lead to some performance questions as I mentioned and we're still trying to figure out if that is quite the right approach, but it's something we're at least hoping to experiment with.

As for the programmatic API, technically we can't easily use that right now anyways just because we are using native ESM and need to spawn a Node process with --experimental-vm-modules. Angular CLI won't have that flag (unless users remember to pass it when running ng test) so we need to spawn a second Node process just to set the flag. As a result, running Jest as a separate Node process with --expeirmental-vm-modules is the best place to do that.

We could avoid that and fork the Angular CLI process instead, but the Jest programmatic API would need to be useful enough to make that desirable. Overriding the config is something we could already do even when running Jest as a separate process, so I'm not sure how useful it would be in this particular case. Structured test result data would be nice, but we'd probably rather rely on Jest's built-in reporter instead of trying to build our own on top of a generic test result data structure. Are there specific features planned for this which might be useful for Angular?

[nicojs]

As for the programmatic API, technically we can't easily use that right now anyways just because we are using native ESM and need to spawn a Node process with --experimental-vm-modules.

This would be fine for StrykerJS, since Stryker spawns a separate process for a test runner anyway. So we can already pass --experimental-vm-modules that way.

As a result, running Jest as a separate Node process with --experimental-vm-modules is the best place to do that.

Yes, that is what we're planning to do.

We could avoid that and fork the Angular CLI process instead, but the Jest programmatic API would need to be useful enough to make that desirable.

If I understand this correctly, you're opting to do Angular work with just the normal jest CLI, but with a build step before running the tests, is that correct? This would put Angular back in the camp that says: "let's not recreate our build pipeline in our test pipeline". I would love that 😻! That would mean that Stryker would work with a simple --buildCommand (something like ng build --test). We could run jest as we would normally.

[dgp1130]

If I understand this correctly, you're opting to do Angular work with just the normal jest CLI, but with a build step before running the tests, is that correct? This would put Angular back in the camp that says: "let's not recreate our build pipeline in our test pipeline". I would love that 😻!

That's the idea. It's still TBD if this will actually work out in practice given performance and maintainability concerns, but it's what we're trying.

That would mean that Stryker would work with a simple --buildCommand (something like ng build --test). We could run jest as we would normally.

I would love to support something like this, though unfortunately it's not quite that simple. I think it's worth considering adding this kind of functionality, but that's a separate question from the Jest configuration. I created #26132 to track that specific feature request.