feat(envs): remove core envs from the manifest and their sources from the workspace - #10465
feat(envs): remove core envs from the manifest and their sources from the workspace#10465davidfirst wants to merge 308 commits into
Conversation
PR Summary by QodoLoad former core envs as regular registry envs with legacy version pinning
AI Description
Diagram
High-Level Assessment
Files changed (15)
|
Code Review by Qodo
1. Unfixable NonLoadedEnv remediation
|
|
Code review by qodo was updated up to the latest commit c7dd1a7 |
|
Code review by qodo was updated up to the latest commit e6418b9 |
|
Code review by qodo was updated up to the latest commit c9eca3d |
|
Code review by qodo was updated up to the latest commit 3f5c24e |
|
Code review by qodo was updated up to the latest commit 0607c7d |
|
Code review by qodo was updated up to the latest commit b23b273 |
|
Code review by qodo was updated up to the latest commit 94eddce |
|
Code review by qodo was updated up to the latest commit ac4b7d0 |
|
Code review by qodo was updated up to the latest commit ab21e34 |
|
Code review by qodo was updated up to the latest commit 66fd06b |
|
Code review by qodo was updated up to the latest commit 684cdf6 |
|
Code review by qodo was updated up to the latest commit 3df0fcd |
|
Code review by qodo was updated up to the latest commit eedfdcd |
Diagnostic addition to narrow down the perf regression found in this check: run it once with the repo's own binary (gating, as before) and once with the bvm-linked nightly release already fetched by setup_harmony (informational only, never fails the job). Reuses the existing install_bvm/bvm_upgrade commands and the same .bvm cache key setup_harmony populates this pipeline run, so it should mostly reuse the cached bundle rather than re-download. Remove once the regression in the repo binary's aspect-loading path is root-caused.
CircleCI skips steps after a failed one unless that specific step is
marked when:always - it doesn't propagate through a whole reusable
command's inner steps. install_bvm/bvm_upgrade sat after the repo-bit
check step, so when that step failed/timed out (as expected), bvm
setup never ran and the diagnostic comparison silently no-opped
("bbit: command not found", swallowed by the || true). Move the bvm
setup earlier so it runs unconditionally, independent of the repo-bit
check's outcome.
We already know the repo-bit check is slow/fails; running the known- good bvm baseline first gives that signal without waiting on the repo-bit run, and it stays non-blocking (|| true) so it can't skip the repo-bit gate either way.
Repo bit has a confirmed, reproducible perf regression on this check (~5min timeout vs ~1-2min on a bvm release, across 4 environments) - see the new CI-HANG-INVESTIGATION.md for the full root-cause writeup (narrowed to workspace.ts's self-as-aspect recursion, unlocked by isCoreAspect() now returning false for the envs this branch removed from the core manifest). Disabling the repo-bit step until it's fixed so it can't block merges on a broken check; the non-blocking bvm-bit comparison stays as a sanity signal. Re-enable once fixed.
Runtime-instrumented the workspace.ts self-as-aspect branch in a disposable /tmp clone - it's never entered during the hang, disproving the earlier hypothesis. Re-reading the captured debug.log's INFO-level loadAspects lines (not the much noisier DEBUG-level file-write lines) shows the real shape: one single trace root recurses for the entire 5-minute window, alternating consumer-fs-load/extension-merge calls into workspace.loadAspects without ever terminating - not fan-out across many components. Found a near-exact match already diagnosed and fixed (unmerged) on origin/refactor/component-loading-v2-take-3-stage2: commits f9ae003 and its follow-up 1213c36 describe and fix the identical mechanism in WorkspaceAspectsLoader.loadAspects (concurrent calls for different root aspects independently re-isolating a shared env dependency, because isAspectLoaded only flips true after a load completes). Their fix serializes loadAspects through a single queue; measured similarly (13:54 -> 10s on a 311-component workspace). Porting that approach is now the primary recommended next step.
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml
@teambit/node, react, aspect, env: 1.0.1105 -> 1.0.1107 @teambit/mdx, readme: 1.0.1106 -> 1.0.1108
…tall, document root cause
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml # scopes/react/react/react.templates.ts
…om-manifest # Conflicts: # .circleci/config.yml # pnpm-lock.yaml
…om-manifest # Conflicts: # .bitmap # .circleci/config.yml # pnpm-lock.yaml # scopes/workspace/workspace/build-graph-from-fs.ts
…into remove-core-envs-from-manifest # Conflicts: # pnpm-lock.yaml
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml
…after merging master
…om-manifest # Conflicts: # .bitmap # e2e/harmony/dependency-resolver.e2e.ts # pnpm-lock.yaml # scripts/generate-e2e-timings.js
…n in the grouped pipeline
Removes the env aspects (
teambit.react/react,teambit.harmony/node,teambit.harmony/aspect,teambit.envs/env,teambit.mdx/mdx,teambit.mdx/readme) from the core manifest to slim Bit. They now act like any other env, installed from the registry.New default env:
teambit.harmony/empty-env(core). A totally empty env - no compiler, no tester, no preview, no dependency policy. Components with no env configured use it and work fully offline out of the box (add → compile no-op → tag/snap → export). Since it has no behavior, it has nothing to drift when bit itself changes - the one env that is safe to keep core (and versionless in models) forever. To get a dev experience, users configure a real env (bit createflows already do).teambit.harmony/aspectandteambit.envs/envare removed like the rest, with zero behavior change. Their implementation is untouched (react-based, preview and all) - users get the exact released behavior afterbit install(the pinned-version machinery auto-installs them). New envs are created from the bitdev env packages (bit create react-envetc.), so these built-in envs are legacy surface. Thebit-aspecttemplate and the harmony starters moved to the core generator aspect, sobit create bit-aspectandbit newkeep working out of the box (the created aspect needsbit installbefore it loads, like any env).Versionless by design. Config entries for the removed env ids are persisted by name, without a version - exactly as they were when core (registered as core-extension names). Keeping them versionless is deliberate on two counts. First, it keeps the env from becoming a dependency edge of its own components; otherwise an env such as react, whose dependency closure includes components that use it as their env, creates circular TS project references and breaks lane/tag builds. Second, it preserves forward compatibility: a re-tag under the new bit keeps the env id versionless, so a teammate who has not upgraded yet (whose bit still ships these as core) can import the re-tagged component and resolve the env - instead of receiving a versioned id their bit has no component for. The alternative (showing the component as modified and pinning the env on the next tag) would silently break not-yet-upgraded consumers.
Backward compatibility. Old components have the removed envs saved without a version.
legacy-core-envs.tsmaps them to pinned versions, applied only at the resolution/loading/install level - stored objects are never mutated. Versionless legacy ids match the env slot ignoring version,bit installauto-adds their packages, and single-instance semantics are enforced (a loaded version is reused rather than loading another copy). Not-installed legacy envs fail fast with aNonLoadedEnvissue suggestingbit install- no scope-capsule isolation in workspace context (which used to take minutes). Old components load without being reported as modified, and re-tagging keeps the env versionless - covered end-to-end bye2e/harmony/legacy-core-env-back-compat.e2e.ts, which imports a component exported by a pre-removal bit (env saved versionless) and asserts it is not modified and stays versionless after a re-tag.Relocated core wiring: the
bit aspectCLI command moved toteambit.workspace/workspace;validateBeforePersistHookmoved toteambit.dependencies/dependency-resolver; the dead@teambit/legacylink is now skipped instead of crashing.Also fixes latent issues this path exposed: versionless seeders filtering out all manifests in
loadExtensionsByManifests, circular env chains causing infinite component-load recursion, versioned core-aspect ids escaping core filters anddoRequiremutating shared core manifests, stack overflows from recursive graph traversal, and a spuriousMissingDistsissue for compiler-less envs.Verified locally: fresh workspace (JS and TS components) - clean status in ~1s, tag/snap/export offline,
bit envs/bit testgraceful; this repo's workspace - status/insights/list-core clean; the seven repo components that relied on the default env are now explicitly set to the node env.bit create <template> --env <removed-env>loads the env's templates on demand from the global scope (pinned version); this path also loads the full manifest graph, and binds manifest deps of legacy envs to their pinned versions (models built when these envs were core don't list them as dependencies). The e2esetCustomEnvhelper installs the env package the fixture imports (e.g.@teambit/node).Also removes the former-core env sources from this repo's workspace (
scopes/harmony/node,scopes/react/react,scopes/harmony/aspect,scopes/envs/env,scopes/mdx/mdx,scopes/docs/readme) - bit now dogfoods them as installed packages like any consumer, and the source-vs-installed duality is gone. Making this pass end-to-end surfaced several general fixes that ride along:.docs.mdximports are detected even when the mdx aspect isn't loaded (latent gap once mdx is no longer core - without it, docs deps silently drop from dependency computation and preview bundling fails).Module._extensionsrequire hooks are restored after each build task. An in-process tester leaves@babel/register's pirates hook installed; the hook claims all.jsfiles (including node_modules, regardless of babelignoreconfig) and breaksrequire()of ESM-only packages in every later task in the process (pirates drops theformatarg node >=22.12 uses to routerequire(esm)).import()instead of a top-level require, immune to the same stale-hook hazard.@bit-no-check; timings manifest covers the split spec files so shard balancing accounts for the heavier env-install suites.