pool: one tree-wide code-images/ pool per results-dir; per-org pools stay readable

[FileSystemGuy]

Fourth and last PR of the results-dir hygiene effort (#851 logging, #852 resolver, #853 runs). Scope was narrowed on 2026-09-18 to the pool relocation only; leaf version stamping and the per-submission manifest follow as a separate PR once their schema is settled.

What changes

Code images now live in <results-dir>/code-images/code-<hash8>/ — one content-addressed pool per tree, shared by every organization and both divisions — instead of one pool per organization at <results-dir>/<orgname>/. Identical images from two submitters coincide in a merged reviewer tree, runs gc has one place to sweep, and the pool no longer sits next to the org's own results subtree.

Per-organization pools remain readable permanently. The frozen v3.0 submissions tree carries 19 of them and must validate exactly as before (see proof below). Nothing ever rewrites a run leaf: pointers are hash-only, so relocation never touches .mlps-code-image.

Writer

• capture_or_verify_code_image captures into / reuses from code-images/ and writes the .mlps-image-pool sentinel there. It no longer creates <results-dir>/<orgname>/.
• New legacy_migration.migrate_org_pool relocates a v3.0 per-org pool into code-images/: os.rename per image (copy + rename on EXDEV), duplicates dropped only after the kept copy re-hashes clean, org sentinel removed, empty org dir removed. Idempotent, crash-resumable, two status lines (D-74).
• _check_and_migrate_legacy_layout now runs, in order: legacy code/ migration (materializing into code-images/), org-pool relocation, the [BUG]“.mlps-image-pool” sentinel never auto-created, causing validate to ERROR #716 sentinel self-heal. Fast path when the global sentinel exists and no <orgname>/ dir does: two exists probes, nothing else.

Readers

• code_image.resolve_pool_image(results_dir, hash, orgname): code-images/ first, then the leaf's own <org>/ pool, never another org's pool (same isolation as before).
• CHECK-01 accepts either sentinel as the org's pool root; the not-found message now says which two places were looked in.
• CHECK-02/03 iterate the global root plus every per-org root. CHECK-03 counts references from every org against code-images/, and — as before — only the org's own leaves against <org>/.
• CHECK-04 flags an unsentinelled code-images/ (D-91) and warns on an empty one (D-90). A per-org pool with its sentinel is a supported layout and draws no diagnostic.
• 2.1.2 treats code-images as a reserved top-level name.
• reportgen Code/Logs hrefs point at code-images/code-<hash8>/ when the image is there, else <org>/code-<hash8>/ as today.
• mlpstorage runs gc discovers the global pool by its sentinel (no code change needed).

Docs

Rules.md §2.1.2, §2.1.5.a/b (the stale "code" subdirectory text is gone), §2.1.6 (new location + v3.0 compatibility paragraph), §2.1.27 diagram. ManPage RESULTS DIRECTORY tree and FILES. No CLI surface changed; the --help_all parity suite is untouched.

Validator-diff proof (v3.0 tree)

main (58a55af, worktree) vs this branch on ~/MLPerfStorage/submissions_storage_v3.0 @ b50de9cf0:

	main	branch
rule lines	644	644
ERROR / WARNING / INFO	362 / 85 / 197	362 / 85 / 197
per-rule counts	identical	identical
Some … checks failed for lines	identical	identical
summary.csv	byte-identical	byte-identical

The only text change is the pre-existing HPE CHECK-01 error (code-64104f04/, still unuploaded) gaining "(looked in code-images/ and HPE/)".

An earlier draft pooled CHECK-03 references across orgs, which silently cleared two Nebius orphan errors; per-org semantics were restored for per-org pools so the frozen tree's findings are unchanged.

Smoke test (real CLI, scratch tree)

init → closed training unet3d datasize captured into code-images/code-d1dd9bfc with sentinel and pointer; second invocation reused it; moving the pool to SmokeOrg/ by hand and running again relocated it back ("Relocating SmokeOrg's code-image pool into code-images/ (1 images)…"); validate reports no CHECK-0x or 2.1.2 findings.

Tests

RED first (cbf3eed), then GREEN (6adb19f). New tests/unit/test_global_code_image_pool.py (29 tests). The per-org isolation tests are rewritten as cross-org sharing tests; migration/capture tests re-pointed at the new location; the D-74 source test now allows two status lines per migration routine.

All four CI suites locally: tests 3641 passed, mlpstorage_py/tests 947, kv_cache 238, vdb 228.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅