ci: make the DOOP dataset manifest depend on the data, not the path - #1307
Merged
Conversation
This was referenced Sep 2, 2026
run-doop-perf-gate.sh hashed `sha256sum "$data_dir"/*.facts`. sha256sum prints "<hash> <path>", and nothing stripped the directory, so the manifest embedded whatever $data_dir expanded to: relative, absolute and trailing-slash spellings of one directory produced three different hashes. The mismatch surfaces as "DOOP dataset manifest ... != pinned", which reads as dataset substitution -- the same false accusation #1294 removed from a different cause, in the same message. The manifest is now computed from inside the directory, so no path is ever embedded rather than embedded and then stripped: ( CDPATH= cd -- "$1" && sha256sum -- *.facts | LC_ALL=C sort -k2 ... ) Stripping the prefix afterwards was the obvious fix and is not enough: GNU sha256sum escapes a path containing a backslash by prefixing the whole line, which a prefix strip does not remove, so two spellings of a backslash-containing directory still disagreed. CDPATH= is load-bearing rather than hygiene. `cd` consults CDPATH for a relative argument -- which $data_dir is by default -- and on a hit both prints the resolved path, landing it inside the command substitution, and reaches a different directory. With CDPATH pointing at anything that also contains a `doop`, the function returned that directory's hash and exited 0: a manifest whose purpose is proving the benchmark ran on the pinned data, certifying data it never read. `--` covers option-shaped names; it does not cover a bare `-`, which bash converts to $OLDPWD after option parsing, and the comment says so. The expression is a function so it can be exercised. It could not be tested before without running the whole benchmark, which is why the collation gate from #1294 could only string-match this site. Scope: this changes the computed value. Nothing in this repository pins WIRELOG_DOOP_DATASET_MANIFEST_SHA256 -- it is read from the environment and the gate skips when unset -- so there is nothing here to recompute, but an operator holding a value recorded under the old formula will see a mismatch on the first run. The failure message now says so and names where the correct value already lives, because a fix that re-creates the accusation it removes is worse than the bug. CHANGELOG carries the same. manifest_for_doop in run-downstream-matrix.sh is deliberately untouched. Its output is pinned in downstream-matrix-oracles.tsv, and moving it inside a change premised on manifests not moving would be self-contradictory. The two agree on a canonical path and diverge on trailing slashes, dotfiles and symlinks; the self-test asserts the agreement and pins the divergences, so unifying them has to be deliberate. That unification is the shared helper #1297 asks for. Three defects in the self-test are worth recording, because they share a cause. The backslash assertion put the backslash in the leaf, so both spellings carried it, sha256sum escaped both identically, and reverting to the broken implementation passed the assertion named for that bug. The option-shaped fixture was first placed where both sides of an equality broke together, so two empty strings compared equal. And the locale assertion interpolated a path into single quotes, so a TMPDIR containing a quote killed both subshells and the empty results compared equal. Each fixture was built from the shape of the description rather than the mechanism of the defect, and each is now built from the mechanism and killed by mutation. Also fixed here: the SIGPIPE twin in check-manifest-collation.sh, where `locale -a | grep -Fxq` reports the locale absent because grep exits first, and that gate's skip is exit 0 -- so on a locales-all host it reported OK having asserted nothing. Refs #1297.
justinjoy
force-pushed
the
fix/1297-doop-manifest-path
branch
from
September 3, 2026 01:02
d6a850f to
2faf99b
Compare
justinjoy
enabled auto-merge (rebase)
September 3, 2026 01:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1297. Stacked on #1298 (
fix/1294-manifest-collation) — both change the same line, so basing onmainwould guarantee a conflict. Merge #1298 first.The defect
run-doop-perf-gate.shhashedsha256sum "$data_dir"/*.facts.sha256sumprints<hash> <path>, and nothing stripped the directory — so one directory gave three hashes:Reported as
DOOP dataset manifest … != pinned …, which reads as dataset substitution — the same false accusation #1294 removed from a different cause, in the same message.The fix, and why the obvious one wasn't enough
Stripping the prefix afterwards was my first fix. GNU
sha256sumescapes a path containing a backslash by prefixing the whole line, which a prefix strip can't remove — so two spellings of a backslash-containing directory still disagreed.CDPATH=is load-bearing.cdconsultsCDPATHfor a relative argument — which$data_diris by default — and on a hit it prints the resolved path into the command substitution and reaches a different directory. WithCDPATHpointing at anything containing adoop, the function returned that directory's hash and exited 0: a manifest whose purpose is proving the benchmark ran on pinned data, certifying data it never read.--covers option-shaped names; it does not cover a bare-, which bash converts to$OLDPWDafter option parsing. The comment says so rather than claiming otherwise.Compatibility
This changes the computed value. Nothing in-repo pins
WIRELOG_DOOP_DATASET_MANIFEST_SHA256— it's read from the environment and the gate skips when unset — so there's nothing here to recompute. But an operator holding a value recorded under the old formula sees a mismatch on the first run. The failure message now says the pin is stale rather than the data wrong, and names where the correct value already lives (downstream-matrix-oracles.tsv'sfiles:field). CHANGELOG carries the same.manifest_for_doopdeliberately untouchedIts output is pinned in
downstream-matrix-oracles.tsv; moving it inside a change premised on manifests not moving would be self-contradictory. The two agree on a canonical path and diverge on trailing slashes, dotfiles and symlinks — the self-test asserts the agreement and pins the divergences, so unifying them has to be deliberate. That unification is the shared helper the issue asks for.Three defects in my own test, all one cause
sha256sumescaped both identically — reverting to the broken implementation passed the assertion named for that bug-x.factsin a two-spelling fixture--broke both sides; two empty strings compared equal'$data'single-quote interpolationTMPDIRkilled both subshells; two empty strings compared equalEach was built from the shape of the description rather than the mechanism of the defect. Each is now built from the mechanism and killed by mutation.
Validation
en_USis codepoint order)meson test -C build: 306 Ok / 0 Fail / 12 SkippedCDPATH=(2 named assertions), dropsha256sum --, revert to the prefix-strip form, dropLC_ALL=C, name-dropping.manifest_for_dircorrectly survives;cd --survives and is documented as out of scope.TMPDIRshapes (space, quote,#,*,[,\,$, dot, relative),CDPATH=.,CDPATH=/tmp,CDPATH=.:/tmp:/var,OLDPWD=/etcAlso fixes the SIGPIPE twin in
check-manifest-collation.sh:locale -a | grep -Fxqreported the locale absent becausegrepexits first, and that gate's skip isexit 0— so on alocales-allhost it reported OK having asserted nothing.Four review rounds; final verdicts Reviewer and Critic both approving.