fix: hold bitgo back via committed private + --include-private

[zahin-mohammad]

Summary

• The two-phase publish from fix: split publish into siblings-then-bitgo passes #9424 held the bitgo umbrella out of the siblings pass by editing modules/bitgo/package.json at runtime (set-umbrella-publishable.ts toggling private). That edit is never committed, and lerna publish from-package runs an unconditional clean-tree pre-flight (detectFromPackage → verifyWorkingTreeClean → throwIfUncommitted, lerna/dist/commands/publish/index.js:8238-8248), so pass 1 aborts with EUNCOMMIT before publishing anything. Master currently cannot cut a release — prod or beta. Beta run 31038353257 confirms it (M modules/bitgo/package.json).
• Fix: commit bitgo as private and hold it back the lerna-native way. from-package filters private packages before packing, so pass 1 publishes the siblings with a clean tree; pass 2 re-includes bitgo by name with --include-private, which strips private in-memory after the clean-tree check (preparePrivatePackages, index.js:8343) and packs with BITGO_GENERATE_SHRINKWRAP=true. The published tarball carries no private field.
• Deletes set-umbrella-publishable.ts and both hold-back/restore steps in both workflows.

Why --include-private now, when WCI-1270 rejected it

WCI-1270 rejected --include-private because a permanently-private umbrella would silently drop out of three checks that enumerate non-private packages. The EUNCOMMIT failure is the new fact that flips the decision: the runtime flip is the only alternative and it fights lerna's clean-tree requirement. Committing private is the sole approach that doesn't, and the three checks are one-line patches (this PR) rather than blockers.

npm contract, and the repo tooling that also reads private

Per npm docs, private: true does exactly one thing to npm — makes npm publish refuse (accidental-publication guard); it does not affect pack, install, or workspaces. express, beta-tools, and web-demo are already private and build/link normally. lerna --include-private is purpose-built to lift the publish block transiently at pack time. Bonus: a stray npm publish in modules/bitgo outside the two-phase flow is now refused by npm.

But this repo's own tooling also keys on .private, beyond npm's contract, so making bitgo private has internal consequences that must be handled explicitly (see below).

Sites that read .private, patched to keep covering the umbrella

Keyed on the umbrella's directory/name (prepare-release.ts rescopes the name, not the dir, so publish-side sites derive the name; internal sites match bitgo):

• .github/actions/verify-npm-packages/index.js — package-level existence (HEAD on package root; bitgo exists, so this passes today).
• scripts/verify-release.ts — beta + recovery verification.
• .github/workflows/npmjs-release.yml — recovery "verify missing versions".
• scripts/update-dockerfile.ts — functional, not just a check: it walks @bitgo/express's deps and emits COPY/yarn link lines only for non-private modules. express depends on bitgo, and the committed Dockerfile copies+links modules/bitgo. Left unpatched, marking bitgo private drops it from the express image. Patched to keep walking bitgo; verified yarn update-dockerfile produces no Dockerfile diff.

(One cosmetic .private read remains at npmjs-release.yml recovery-context output — it only labels packages in a log line, no behavior change.)

Sharp edge

--include-private takes the on-disk package name, which is channel-specific (bitgo on stable, @bitgo-beta/bitgo after prepare-release.ts). Both workflows derive it with jq -r '.name' modules/bitgo/package.json; a hardcoded name would fail silently (umbrella never publishes).

Test plan

• Beta dispatch from this branch (recovery-mode: false): pass 1 publishes the siblings with bitgo excluded (no EUNCOMMIT); pass 2 attempts only the umbrella and generates the shrinkwrap.
• curl .../@bitgo-beta/bitgo/<v> | jq '._hasShrinkwrap' → true; npm install @bitgo-beta/bitgo@<v> + require() works in a clean dir.
• No runtime commit to rel/latest; modules/bitgo/package.json is never left flipped.
• Prod recovery release (dry-run: false, recovery-mode: true) publishes the missing 52.4.2 with a correct shrinkwrap; npm install bitgo@52.4.2 works in a clean dir.

Note: built on the phase-2 base; rebase on latest master before merge (local fetch was auth-blocked at authoring time).

Ticket: WCN-1901

[linear-code]

WCN-1901

[zahin-mohammad]

✅ Alpha canary from this branch — end-to-end pass

Dispatched Publish @bitgo-beta on this branch: run 31040810632 succeeded.

Workflow (the EUNCOMMIT is gone):

• Lerna Publish (siblings) ✅ — no EUNCOMMIT; clean tree because bitgo is committed private (no runtime flip).
• Lerna Publish (bitgo) ✅ — log shows @bitgo-beta/bitgo => 14.7.1-alpha.500 (private!), confirming --include-private re-included it and lerna stripped private at pack time.
• Verify Publish ✅.

Registry: @bitgo-beta/bitgo@14.7.1-alpha.500 has _hasShrinkwrap: true; the published npm-shrinkwrap.json carries 143 resolved @bitgo-beta/* entries (each with version + resolved + integrity).

Clean install (the WCI-1200 path): npm install @bitgo-beta/bitgo@14.7.1-alpha.500 in an empty dir installs 102 siblings (nested under the umbrella, as expected for a dependency's shrinkwrap) and require('@bitgo-beta/bitgo') succeeds.

Observation, not a blocker: pass 2 published bitgo plus 2 siblings (sdk-coin-rbtc, sdk-rpc-wrapper) that pass 1 hadn't landed — the idempotent from-package safety net swept them up. Worth watching whether pass 1 consistently publishes all siblings in one go.

Scope note: this exercises publish.yml + shrinkwrap generation. It does not exercise the prod npmjs-release.yml two-pass (lerna version split, GPG signing) — only a real prod release covers that half.