pin-override: re-resolve in range before writing any resolution

.github/actions/pin-override/action.yml

@@ -1,8 +1,10 @@
 name: Dependabot pin override
 description: >-
-  Write Yarn `resolutions` for the transitive advisories Dependabot cannot fix, when a
-  parent pins the vulnerable package to an exact version. Stays inside the compatibility
-  line, verifies every entry by re-running the audit, and names whatever it refuses.
+  Fix the advisories Dependabot cannot. First re-resolves every alerted package inside the
+  ranges the tree already declares (`yarn up -R`, lockfile only); then writes Yarn
+  `resolutions` for what is left when a parent pins the vulnerable package exactly. Stays
+  inside the compatibility line, verifies every step by re-running the audit, and names
+  whatever it refuses.
 
 inputs:
   path:
@@ -23,6 +25,11 @@ outputs:
   summary:
     description: One-line summary of what was applied, reverted and skipped.
     value: ${{ steps.override.outputs.summary }}
+  bumped:
+    description: >-
+      Markdown list of the packages re-resolved inside their already-declared ranges
+      (lockfile only, no package.json change) and the advisories that cleared.
+    value: ${{ steps.override.outputs.bumped }}
   applied:
     description: Markdown list of the resolutions written.
     value: ${{ steps.override.outputs.applied }}

.github/actions/pin-override/pin_override.py

@@ -1,5 +1,28 @@
 #!/usr/bin/env python3
-"""Unstick the transitive advisories Dependabot cannot fix, by writing Yarn resolutions.
+"""Unstick the advisories Dependabot cannot fix: re-resolve in range first, then pin.
+
+THE FIRST PASS, AND WHY IT COMES FIRST
+--------------------------------------
+Most of what Dependabot leaves open does not need a resolution at all. It needs the
+lockfile to be re-resolved inside the ranges the tree ALREADY declares, which is what
+`yarn up -R <pkg>` does and what Dependabot cannot: Dependabot targets one exact version
+for a name across the whole tree and gives up when any copy cannot reach it, so a
+package installed at 1.x, 2.x and 5.x under three minimatch parents is unfixable to it
+forever, though each copy is one patch from clear inside its own range. Measured across
+the fleet on 2026-09-11, this pass alone took 220 audit findings to 102, touched no
+package.json, and reopened nothing — it also clears the fix a hand-closed Dependabot PR
+has taught Dependabot to ignore (fundraiser-api's axios, 29 advisories, one month), because
+it never asks Dependabot.
+
+`-R` is load-bearing. `yarn up axios` rewrites the manifest range to the new version; with
+`-R` it keeps `^1.3.6` and only moves the lock, for direct and transitive alike. That is
+why this pass may include direct dependencies where the resolution pass below may not: it
+shadows nothing and pins nothing, it does what a fresh `yarn install` without a lockfile
+would have done. Only what clears an advisory is kept; a package that moved without
+clearing anything is put back, so the PR carries no unexplained churn.
+
+THE SECOND PASS: RESOLUTIONS
+----------------------------
 
 THE PROBLEM
 -----------
@@ -36,6 +59,9 @@
 
 Usage:
     pin_override.py --root frontend [--dry-run] [--json]
+
+`--dry-run` skips the first pass entirely: it cannot be previewed without running it, and
+a dry run must not write the lockfile.
 """
 from __future__ import annotations
 
@@ -340,6 +366,95 @@ def plan(advisories: list[dict], root: Path,
     return sorted(proposals.values(), key=lambda p: p["pkg"]), blocked
 
 
+# ------------------------------------------------------------------ the bump pass
+
+def _worst_of(advisories: list[dict]) -> str:
+    sev = "unknown"
+    for a in advisories:
+        sev = _worst(sev, a.get("severity") or "unknown")
+    return sev
+
+
+def bump_in_range(root: Path, advisories: list[dict]) -> tuple[list[dict], list[dict], str]:
+    """Re-resolve every alerted package inside its declared ranges; keep what helped.
+
+    Returns (bumped, advisories still open, note). `bumped` is one entry per package that
+    moved AND cleared at least one advisory; `note` is non-empty only when the pass was
+    abandoned, and says why. The lockfile and package.json are exactly as they were
+    whenever nothing is kept.
+
+    Two runs of `yarn up -R`, not one. The first moves everything alerted and shows which
+    packages actually cleared something. If any moved without clearing anything, the
+    snapshot is restored and the second run moves only the helpful ones — a PR that says
+    "fixes GHSA-x" must not also carry a bump nobody asked for. Both runs are
+    `--mode=update-lockfile`, so nothing is linked and no package script executes.
+
+    A changed package.json abandons the pass. `-R` does not rewrite manifests (measured on
+    a direct axios in fundraiser-api: lock moved 1.9.0 -> 1.20.0, manifest byte-identical),
+    so if it ever did, the assumption this pass rests on is wrong and it must do nothing.
+    """
+    pkgs = sorted({a["pkg"] for a in advisories if a.get("pkg")})
+    if not pkgs:
+        return [], advisories, ""
+    lock_path, manifest_path = root / "yarn.lock", root / "package.json"
+    lock_before, manifest_before = lock_path.read_bytes(), manifest_path.read_bytes()
+    tree_before = lock_versions(lock_path)
+    open_before = {(a["pkg"], a["ghsa"]) for a in advisories}
+
+    def restore() -> None:
+        lock_path.write_bytes(lock_before)
+        manifest_path.write_bytes(manifest_before)
+
+    def attempt(names: list[str]) -> tuple[list[dict] | None, str]:
+        proc = run(["yarn", "up", "-R", *names, "--mode=update-lockfile"], root)
+        if proc.returncode != 0:
+            restore()
+            return None, "yarn up -R failed; lockfile restored\n" + (proc.stderr or proc.stdout)[-2000:]
+        if manifest_path.read_bytes() != manifest_before:
+            restore()
+            return None, "yarn up -R rewrote package.json, which it must never do here; restored"
+        return audit(root), ""
+
+    after, why = attempt(pkgs)
+    if after is None:
+        return [], advisories, why
+    still = {(a["pkg"], a["ghsa"]) for a in after}
+    helpful = sorted({pkg for pkg, ghsa in open_before - still})
+    if not helpful:
+        restore()
+        return [], advisories, ""
+    if helpful != pkgs:
+        restore()
+        after, why = attempt(helpful)
+        if after is None:
+            return [], advisories, why
+        still = {(a["pkg"], a["ghsa"]) for a in after}
+
+    tree_after = lock_versions(lock_path)
+    bumped = []
+    for pkg in helpful:
+        cleared = sorted(g for p_, g in open_before - still if p_ == pkg)
+        if not cleared:
+            continue
+        bumped.append({
+            "pkg": pkg,
+            "from": sorted(tree_before.get(pkg, set()), key=lambda v: parse(v) or ()),
+            "to": sorted(tree_after.get(pkg, set()), key=lambda v: parse(v) or ()),
+            "ghsas": set(cleared),
+            "severity": _worst_of([a for a in advisories if a["pkg"] == pkg]),
+        })
+    return bumped, after, ""
+
+
+def render_bumped(bumped: list[dict]) -> str:
+    lines = []
+    for b in bumped:
+        moved = f"{', '.join(b['from'])} → {', '.join(b['to'])}"
+        lines.append(f"- `{b['pkg']}` {moved} ({b['severity']}, "
+                     f"{', '.join(sorted(b['ghsas']))})")
+    return "\n".join(lines)
+
+
 # ------------------------------------------------------------------ writing
 
 def apply_resolutions(root: Path, entries: dict[str, str]) -> dict[str, str]:
@@ -377,10 +492,16 @@ def install(root: Path) -> tuple[bool, str]:
 
 # ------------------------------------------------------------------ output
 
-def summarise(applied: list[dict], unresolved: list[dict], blocked: list[dict]) -> str:
+def summarise(applied: list[dict], unresolved: list[dict], blocked: list[dict],
+              bumped: list[dict] | None = None) -> str:
     bits = [f"{p['key']} -> ^{p['target']} ({p['severity']}, {len(p['ghsas'])} "
             f"advisor{'y' if len(p['ghsas']) == 1 else 'ies'})" for p in applied]
     out = "; ".join(bits) if bits else "nothing to change"
+    if bumped:
+        n = sum(len(b["ghsas"]) for b in bumped)
+        out = (f"{len(bumped)} package{'s' if len(bumped) != 1 else ''} re-resolved in "
+               f"range ({n} advisor{'y' if n == 1 else 'ies'})"
+               + ("" if not bits else " | " + out))
     if unresolved:
         out += f" | {len(unresolved)} reverted (did not clear)"
     if blocked:
@@ -421,8 +542,21 @@ def main() -> int:
         print(f"{root}: no package.json", file=sys.stderr)
         return 2
 
+    advisories = audit(root)
+    # First pass: move what the declared ranges already allow. Skipped on a dry run, which
+    # must not write the lockfile and cannot preview this without doing so.
+    bumped, bump_note = [], ""
+    if not args.dry_run:
+        bumped, advisories, bump_note = bump_in_range(root, advisories)
+        if bump_note:
+            print(bump_note, file=sys.stderr)
+    for b in bumped:
+        print(f"  {b['pkg']}: {', '.join(b['from'])} -> {', '.join(b['to'])}  "
+              f"[{b['severity']}] {', '.join(sorted(b['ghsas']))}")
+    # The tree is re-read after the bump: a resolution's shape depends on which lines are
+    # present, and the first pass may have removed or merged some.
     tree = lock_versions(root / "yarn.lock")
-    proposals, blocked = plan(audit(root), root, tree)
+    proposals, blocked = plan(advisories, root, tree)
 
     if args.json:
         print(json.dumps({"proposals": proposals, "blocked": blocked},
@@ -436,10 +570,11 @@ def main() -> int:
 
     emit("blocked", render_blocked(blocked))
     emit("blocked_count", str(len({(b["pkg"], b["why"]) for b in blocked})))
+    emit("bumped", render_bumped(bumped))
 
     if not proposals or args.dry_run:
-        emit("changed", "false")
-        emit("summary", summarise([], [], blocked))
+        emit("changed", "true" if bumped else "false")
+        emit("summary", summarise([], [], blocked, bumped))
         if not proposals:
             print("nothing a resolution can fix")
         return 0
@@ -451,8 +586,11 @@ def main() -> int:
         install(root)
         print("install failed with the proposed resolutions; reverted\n" + log,
               file=sys.stderr)
-        emit("changed", "false")
-        emit("summary", "install failed with the proposed resolutions; reverted")
+        # The first pass's lockfile is still in place and still verified; only the
+        # resolutions are gone. Say so rather than reporting the whole run as nothing.
+        emit("changed", "true" if bumped else "false")
+        emit("summary", summarise([], [], blocked, bumped)
+             + " | install failed with the proposed resolutions; reverted")
         return 1
 
     # Verify against reality, not against the version arithmetic: an advisory we claimed to
@@ -471,13 +609,15 @@ def main() -> int:
             install(root)
             print("install failed after dropping the unresolved entries; reverted\n" + log,
                   file=sys.stderr)
-            emit("changed", "false")
+            emit("changed", "true" if bumped else "false")
+            emit("summary", summarise([], [], blocked, bumped)
+                 + " | install failed after dropping the unresolved entries; reverted")
             return 1
         for p in unresolved:
             print(f"  reverted {p['key']}: advisories still open after the bump")
 
-    emit("changed", "true" if applied else "false")
-    emit("summary", summarise(applied, unresolved, blocked))
+    emit("changed", "true" if (applied or bumped) else "false")
+    emit("summary", summarise(applied, unresolved, blocked, bumped))
     emit("applied", "\n".join(f"- `{p['key']}` → `^{p['target']}` ({p['severity']}, "
                               f"{', '.join(sorted(p['ghsas']))})" for p in applied))
     print(summarise(applied, unresolved, blocked))

.github/actions/pin-override/test_pin_override.py

@@ -347,3 +347,136 @@ def test_summary_reads_as_a_sentence():
     assert "cheerio/undici -> ^7.29.1 (high, 2 advisories)" in got
     assert "1 need a human" in got
     assert p.summarise([], [], []) == "nothing to change"
+
+
+# ------------------------------------------------------------------ the bump pass
+
+LOCK_AFTER = LOCK.replace("version: 1.1.16", "version: 1.1.18").replace(
+    "version: 5.0.7", "version: 5.0.9").replace("version: 6.15.3", "version: 6.16.0")
+
+
+class _Yarn:
+    """Fake `run`: a `yarn up` rewrites the lockfile to LOCK_AFTER (or to `lock_after`
+    if given) and records every command; anything else exits 0 with no output."""
+
+    def __init__(self, root, rc=0, rewrite_manifest=False, lock_after=LOCK_AFTER):
+        self.root, self.rc, self.rewrite_manifest = root, rc, rewrite_manifest
+        self.lock_after, self.calls = lock_after, []
+
+    def __call__(self, cmd, cwd, check=False):
+        self.calls.append(cmd)
+
+        class Proc:
+            returncode = self.rc
+            stdout = ""
+            stderr = "boom" if self.rc else ""
+        if cmd[:2] == ["yarn", "up"] and self.rc == 0:
+            (self.root / "yarn.lock").write_text(self.lock_after)
+            if self.rewrite_manifest:
+                (self.root / "package.json").write_text('{"name": "changed"}')
+        return Proc()
+
+
+def _audits(monkeypatch, *results):
+    """`audit` returns each result in turn, then the last one forever."""
+    queue = list(results)
+
+    def fake(root):
+        return queue.pop(0) if len(queue) > 1 else queue[0]
+    monkeypatch.setattr(p, "audit", fake)
+
+
+def test_bump_keeps_what_cleared_and_touches_only_the_lockfile(tmp_path, monkeypatch):
+    root = _pkg(tmp_path, dependencies={"qs": "^6.11.0"})
+    manifest = (root / "package.json").read_text()
+    yarn = _Yarn(root)
+    monkeypatch.setattr(p, "run", yarn)
+    before = [_adv("brace-expansion", "GHSA-A", "<1.1.17", ["1.1.16"], ["minimatch@npm:3.1.5"]),
+              _adv("brace-expansion", "GHSA-B", ">=4.0.0 <5.0.8", ["5.0.7"], ["x@npm:1"]),
+              _adv("qs", "GHSA-C", "<6.16.0", ["6.15.3"], ["y@npm:1"], severity="low")]
+    _audits(monkeypatch, [])                      # everything cleared
+    bumped, left, note = p.bump_in_range(root, before)
+    assert note == "" and left == []
+    assert [b["pkg"] for b in bumped] == ["brace-expansion", "qs"]
+    be = bumped[0]
+    assert be["from"] == ["1.1.16", "5.0.7"] and be["to"] == ["1.1.18", "5.0.9"], be
+    assert be["ghsas"] == {"GHSA-A", "GHSA-B"} and be["severity"] == "high"
+    # A DIRECT dependency is bumped too: -R keeps the manifest range and moves the lock.
+    assert bumped[1]["pkg"] == "qs"
+    assert (root / "package.json").read_text() == manifest
+    assert len(yarn.calls) == 1
+    assert yarn.calls[0][:3] == ["yarn", "up", "-R"] and "--mode=update-lockfile" in yarn.calls[0]
+
+
+def test_a_package_that_moved_without_clearing_is_put_back(tmp_path, monkeypatch):
+    root = _pkg(tmp_path)
+    yarn = _Yarn(root)
+    monkeypatch.setattr(p, "run", yarn)
+    before = [_adv("brace-expansion", "GHSA-A", "<1.1.17", ["1.1.16"], ["m@npm:3"]),
+              _adv("qs", "GHSA-C", "<6.17.0", ["6.15.3"], ["y@npm:1"])]
+    still_qs = [_adv("qs", "GHSA-C", "<6.17.0", ["6.16.0"], ["y@npm:1"])]
+    _audits(monkeypatch, still_qs, still_qs)      # qs moved to 6.16.0, still vulnerable
+    bumped, left, note = p.bump_in_range(root, before)
+    assert [b["pkg"] for b in bumped] == ["brace-expansion"]
+    assert left == still_qs
+    # Two runs: the discovery run with both names, then the keep run with only the helpful.
+    assert len(yarn.calls) == 2
+    assert "qs" in yarn.calls[0] and "qs" not in yarn.calls[1]
+    assert "brace-expansion" in yarn.calls[1]
+
+
+def test_nothing_cleared_means_nothing_kept(tmp_path, monkeypatch):
+    root = _pkg(tmp_path)
+    monkeypatch.setattr(p, "run", _Yarn(root))
+    before = [_adv("qs", "GHSA-C", "<6.17.0", ["6.15.3"], ["y@npm:1"])]
+    _audits(monkeypatch, before)
+    bumped, left, note = p.bump_in_range(root, before)
+    assert bumped == [] and left == before and note == ""
+    assert (root / "yarn.lock").read_text() == LOCK            # restored byte for byte
+
+
+def test_a_rewritten_manifest_abandons_the_pass(tmp_path, monkeypatch):
+    root = _pkg(tmp_path, dependencies={"qs": "^6.11.0"})
+    manifest = (root / "package.json").read_text()
+    monkeypatch.setattr(p, "run", _Yarn(root, rewrite_manifest=True))
+    before = [_adv("qs", "GHSA-C", "<6.16.0", ["6.15.3"], ["y@npm:1"])]
+    _audits(monkeypatch, [])
+    bumped, left, note = p.bump_in_range(root, before)
+    assert bumped == [] and left == before
+    assert "package.json" in note
+    assert (root / "package.json").read_text() == manifest
+    assert (root / "yarn.lock").read_text() == LOCK
+
+
+def test_a_failed_yarn_up_restores_and_says_why(tmp_path, monkeypatch):
+    root = _pkg(tmp_path)
+    monkeypatch.setattr(p, "run", _Yarn(root, rc=1))
+    before = [_adv("qs", "GHSA-C", "<6.16.0", ["6.15.3"], ["y@npm:1"])]
+    _audits(monkeypatch, before)
+    bumped, left, note = p.bump_in_range(root, before)
+    assert bumped == [] and left == before
+    assert "failed" in note and "boom" in note
+    assert (root / "yarn.lock").read_text() == LOCK
+
+
+def test_no_advisories_means_no_yarn_run_at_all(tmp_path, monkeypatch):
+    root = _pkg(tmp_path)
+    yarn = _Yarn(root)
+    monkeypatch.setattr(p, "run", yarn)
+    assert p.bump_in_range(root, []) == ([], [], "")
+    assert yarn.calls == []
+
+
+def test_bumped_report_names_versions_and_advisories():
+    out = p.render_bumped([{"pkg": "axios", "from": ["1.9.0"], "to": ["1.20.0"],
+                            "severity": "high", "ghsas": {"GHSA-B", "GHSA-A"}}])
+    assert out == "- `axios` 1.9.0 → 1.20.0 (high, GHSA-A, GHSA-B)"
+
+
+def test_summary_counts_the_bump_pass_first():
+    bumped = [{"pkg": "axios", "ghsas": {"A", "B"}}, {"pkg": "qs", "ghsas": {"C"}}]
+    got = p.summarise([], [], [], bumped)
+    assert got.startswith("2 packages re-resolved in range (3 advisories)"), got
+    applied = [{"key": "tar", "target": "7.5.22", "severity": "high", "ghsas": {"G1"}}]
+    got = p.summarise(applied, [], [{"pkg": "vite"}], bumped)
+    assert "re-resolved" in got and "tar -> ^7.5.22" in got and "1 need a human" in got

README.md

@@ -268,6 +268,8 @@ jobs:
 Version tags `v1`–`v13` predate the current scheme and are frozen point releases. The
 floating-major convention (see [`RELEASING.md`](RELEASING.md)) starts at **`v14`**.
 
+- **v14.6** — `pin-override` action: a first pass re-resolves every alerted package inside the ranges the tree already declares (`yarn up -R <pkg>`, lockfile only, manifest untouched) before any resolution is written. This is what clears a package installed at several versions, which Dependabot cannot fix, and a fix a hand-closed Dependabot PR taught Dependabot to ignore. Measured across the fleet on 2026-09-11: 220 audit findings to 102. New `bumped` output; `changed` is true when either pass changed something.
+- **v14.5** — `pin-override` composite action: writes Yarn `resolutions` for the transitive advisories Dependabot cannot fix when a parent pins the vulnerable package exactly.
 - **v14.2** — `dependabot-auto-merge.yml`: arming is now retried (5 attempts) with a direct-merge fallback, so a transient GraphQL error no longer strands a PR and an already-green PR still merges. Consolidates logic that existed only in the inline copies in wemove.eu, youmove, wemove-charity.eu and pubstatic, which this lets them drop.
 - **v14.1** — `dependabot-auto-merge.yml`: accepts optional `app-id` / `app-private-key` secrets so the merge is armed with a GitHub App token and fires a real push event (a `GITHUB_TOKEN`-armed merge does not, leaving push-triggered deploys silently unrun). Falls back to `GITHUB_TOKEN` when the secrets are omitted, and warns. The update-type gate is now a whole-token, fail-closed match — previously an empty `update-type` from `fetch-metadata` satisfied `contains()` and could auto-merge a major.
 - **v14** — First release under the semver + floating-major scheme. Ships the `docker-smoke` composite action (build + run + production-Host-header probe, with a `build-only` mode).