Retire strict_provenance docs; drop provenance terminology#199
Retire strict_provenance docs; drop provenance terminology#199dimitri-yatsenko wants to merge 2 commits into
Conversation
Match datajoint-python's removal of the strict_provenance runtime guardrail, and align the docs with the decision to keep provenance terminology out of DataJoint's own documentation. DataJoint implements the relational workflow model — data integrity, consistency, lineage, and reproducibility; provenance is a platform-level compatibility concern documented separately. - Rename reference/specs/provenance.md -> trace.md; rewrite it as the spec for the upstream read surface (Diagram.trace + self.upstream), removing the strict_provenance section entirely and reframing the "read/write boundary" in lineage/reproducibility terms. - about/whats-new-23.md: reframe the "provenance trinity" as the trace + self.upstream pair; drop the strict_provenance subsection and adoption step; retarget links to trace.md and the make() contract. - mkdocs.yaml: nav "Provenance Trace & Strict Provenance" -> "Upstream Trace". - Sweep "provenance" from the conceptual and tutorial pages (index, about/index, relational-workflow-model, normalization, comparison-to-workflow-languages, schema-as-workflow-specification, data-pipelines, faq, and three tutorial notebooks), replacing it with "lineage"/"reproducibility"/"traceability" as fits each context.
Add mkdocs-redirects so reference/specs/provenance/ (renamed to reference/specs/trace/ when the provenance framing was retired) keeps resolving instead of 404-ing.
The strict_provenance runtime guardrail is being retired (datajoint-python #1489), and its spec page is removed in #199 (deletes provenance.md, adds trace.md). Drop this PR's strict_provenance content so it stays coherent: - remove the 'Provenance-safe make() (strict provenance)' how-to section (run-computations.md) - revert the strict_provenance enforcement-limit edits (provenance.md, deleted by #199) - reword the autopopulate.md upstream-only note to drop the runtime-flag claim Retain the cascade U3/limitations, Diagram.trace() docs, and self.upstream section.
MilagrosMarin
left a comment
There was a problem hiding this comment.
Thanks @dimitri-yatsenko — nicely-scoped counterpart to datajoint-python#1489. Verified end-to-end.
The retirement side is clean: provenance.md (441 lines, the strict_provenance spec) becomes trace.md (301 lines, focused on the Diagram.trace + self.upstream pair). grep for strict_provenance, assert_read_allowed, assert_write_allowed, contextvars.ContextVar, and "Enforcement model" in the new file all return empty — the flag content is fully purged, not just relabeled. The what's-new-23.md reframe from "provenance trinity" to "trace + self.upstream pair" drops the "Enable strict_provenance=True in staging" migration step cleanly. mkdocs.yaml nav renames "Provenance Trace & Strict Provenance" → "Upstream Trace" and adds the redirects plugin (with pip_requirements.txt picking up mkdocs-redirects) so the old /specs/provenance/ URL doesn't 404. Cross-link to autopopulate.md#43-the-make-reproducibility-contract resolves against the anchor introduced in the stacked #198.
The terminology sweep across the conceptual pages reads well. "Provenance" mostly becomes "lineage" in contexts about tracing derivation ("Queries trace provenance" → "Queries trace lineage", "Rows carrying provenance to their inputs" → "traceable to their inputs"), and "provenance and lineage" (previously redundant) collapses to "lineage and reproducibility". Section labels in the Codd/Chen/RWM comparison table follow: "Provenance" → "Data lineage". Consistent choice per context.
One residual worth calling out: grep -rni "provenance" src/ returns one hit — src/images/dj-platform.svg:155 still lists "Reproducibility • Provenance • Integrity • Collaboration" in the platform-architecture text label. This is defensible under the PR's own framing (the platform's provenance-compatibility features live separately in datajoint-provenance), and the SVG is describing platform-side capabilities rather than framework docs. But the PR body's claim "no 'provenance' terminology remains in tracked source (src/)" is a slight overclaim — the SVG is tracked source. Either update the SVG to match the doc-side terminology reset, or tighten the PR-body claim to note the platform-diagram exception. Small.
Sequencing note (already flagged in the PR body): stacked on #198. Merge #198 first; GitHub retargets this one to main automatically. Verified this PR's diff isolates cleanly against upstream/docs/197-make-contract — 16 files, 341 insertions, 496 deletions.
Approving.
Documentation counterpart to datajoint/datajoint-python#1489 (retire the
strict_provenanceruntime guardrail), and alignment with the decision to keep provenance terminology out of DataJoint's own documentation.DataJoint implements the relational workflow model — data integrity, consistency, lineage, and reproducibility. "Provenance" is associated with a different class of systems; DataJoint's provenance compatibility features live in the platform and are documented separately (
datajoint-provenance). So the public docs use the model's native vocabulary: lineage, reproducibility, traceability.Changes
Retire
strict_provenance:reference/specs/provenance.md→reference/specs/trace.mdand rewrite it as the spec for the upstream read surface (Diagram.trace+self.upstream) — thestrict_provenancesection (§3), config docs, examples, and references are removed; the "read/write boundary" is reframed in lineage/reproducibility terms.about/whats-new-23.md: reframe the "provenance trinity" as thetrace+self.upstreampair, drop thestrict_provenancesubsection and adoption step, retarget links.mkdocs.yaml: nav "Provenance Trace & Strict Provenance" → "Upstream Trace".Drop "provenance" terminology across the conceptual and tutorial pages (
index,about/index,relational-workflow-model,normalization,comparison-to-workflow-languages,schema-as-workflow-specification,data-pipelines,faq, and three tutorial notebooks) — replaced with "lineage" / "reproducibility" / "traceability" per context.Verification
grepconfirms no "provenance" terminology remains in tracked source (src/), and no links point to the oldspecs/provenance.mdpath.llms-full.txtis a generated, untracked build artifact — regenerated on build.)Published URL
reference/specs/provenance/becomesreference/specs/trace/. This PR addsmkdocs-redirects(and themkdocs-redirectsbuild dependency) so the old URL redirects to the new page instead of 404-ing.