Skip to content

test_runner: change-aware test selection #66006

Description

@araujogui

What is the problem this feature will solve?

node --test runs every file matching the discovery glob, every time. Locally that makes the edit-test loop slow enough that people stop running it; in CI every pull request pays for the whole suite regardless of what it touched.

Jest (--onlyChanged) and Vitest (--changed) have had this for years.

What is the feature you are proposing to solve the problem?

Run only the test files whose module graph reaches a given set of files.

$ node --test --related=src/util.ts               # tests that depend on a file
$ git diff --name-only | node --test --related=-  # tests affected by a change

Graph-aware, not path-based: a test that imports a module that imports the changed file is selected. Conservative by default, so anything the graph cannot see has to run, and a change to package.json or a lockfile disables filtering entirely.

Paths rather than a --changed=<rev> flag, because that would mean core shelling out to git. There is no VCS dependency anywhere in lib/ today and I don't think this justifies introducing one. --changed can be layered on later if the team wants it.

Things to figure out

  • CJS. Static imports can be extracted from an ES module without executing it. There is no equivalent for require(), since the bundled lexer reports exports rather than requires. CJS files would be opaque: any test reaching one always runs. Correct, but a CJS-heavy project gets little out of this.
  • Invisible dependencies. A test that reads a fixture with fs, or uses dynamic import(), cannot be selected statically. Those have to resolve to "run it".

Under-selection is the failure mode that matters. Silently skipping a test the change broke is worse than not having the feature.

What alternatives have you considered?

Userland: a wrapper computing the list and passing it to run({ files }). Every such tool has to reimplement module resolution, and gets TypeScript, subpath imports and node_modules boundaries subtly wrong. The resolver is already in core.

--test-rerun-failures covers rerunning what failed, not running what a change could break.

cc @nodejs/test_runner

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions