fix(economic): a slot observation keeps its taxonomy, and the shipped backend proves the registers - #768
Merged
Conversation
… backend proves the registers #767 gave the four-valued cell observation a sound core but let four of the five read paths collapse it back to two answers, and the register properties it proved were only ever executed against SQLite while the fleet deploys Postgres. This closes both, plus the durability posture that was asserted by comment and set nowhere. The resolver contract no longer has a channel a caller can collapse. `settlement_slot_observation` returns `CellObservation` — not `Option`, not `Result` — so a transport failure IS `Unavailable` and there is no `None` standing for four different things. `verify_transition_provenance` matches all four arms and gives each its own verdict: `Conflict` quarantines, `Unavailable` is retryable, `EmptyAtQuorum` is an evidence refusal saying this settle never won the parent it names, and only `Claimed` proceeds. None of the three refusals is an invalid-credit verdict. `SettlementSlotWin` and `read_winning_settlement_slot` are deleted rather than deprecated: they existed only to carry the collapsed answer. A member's 404 now counts as emptiness only when it SAYS the cell is absent (`x-dsm-slot-outcome: absent`). Any other 404 — a route miss reaching the fallback, which still carries the identity echo — is `Unavailable`. An absence must be asserted, not inferred from a status code. Durability is set, not inherited. All three of the node's one-shot register claims — settlement slot, faucet ticket, economic root — now open their write transaction through one `begin_durable_write` helper that issues `SET LOCAL synchronous_commit = on`, so the posture cannot hold for one register and silently lapse for another; only the settlement slot had it before. `require_durable_commit_posture` refuses to start a node whose server reports `fsync` or `full_page_writes` off, and its refusal DECISION is now a pure function so both directions are testable without a differently configured server. The write-once properties are stated once for all three registers in `db::write_once_properties` and run against whichever backend is compiled. On Postgres they refuse to run without `DSM_TEST_DATABASE_URL` rather than skipping, so a job that loses its service container fails loudly instead of reporting a green board that executed nothing. A new `Storage Node (Postgres)` CI job runs the node's DEFAULT features against a real server and then counts the register tests that executed, because a green board is not evidence that these tests ran. Mutation controls, all performed against a live Postgres 14 and restored: - slot write-once -> `DO UPDATE`: all three settlement-slot properties red, 16 of 16 racers accepted. - `begin_durable_write` stops setting the posture: the transaction reports `off` on a session weakened to `off`; the named test red. - the posture gate stops refusing: the weak-posture test red. Verification (Rust 1.98.0, the version rust-toolchain.toml declares): - root `make lint` exit 0 - workspace board 3979 passed, 0 failed - node board (local-dev,strict / SQLite) 282 passed, 0 failed - node board (default features / Postgres 14) 252 passed, 0 failed - ci/production_safety_checks.sh, scripts/ci_scan.sh, ci/no_clock_and_no_json.sh, scripts/check_forbidden_symbols.sh all exit 0
… just the register-test one registry_gate builds a real app through `build_app_for_tests`, which reads DSM_DATABASE_URL and whose Postgres default is a placeholder that was never meant to resolve. With only DSM_TEST_DATABASE_URL set, the job reached a listening server that demanded a password and failed with 'password missing'. This did not reproduce locally because this machine already had a `dsm_storage` database with trust auth — the default URL happened to work here and cannot work on a fresh runner. Re-verified against a freshly created, empty database with both variables set, which is the shape the job actually has: 241 lib + 11 integration passed, 0 failed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item A of the locked remediation order. The vault-scoped lineage verifier (E) does not start until the fresh adversarial gate (D) passes, and the admin-auth hotfix (C) is a separate PR.
What #767 got wrong, and what this fixes
1. The regression I introduced in #767 — four of five read paths collapsed the four-way distinction.
The observation core was sound; every consumer of it was not.
winning_settlement_slot_claimreturnedOption<SettlementSlotWin>, soConflict,UnavailableandEmptyAtQuorumall arrived asNoneand the arm reported all three as an invalid credit. That reports a network fault as a forgery and a genuine divergence in a write-once cell as a network fault.The contract now has no channel a caller can collapse:
A transport failure is
Unavailable, so there is no error channel to.ok()away and noNonestanding for four things.verify_transition_provenancematches all four arms:Claimed(bytes)Conflict { distinct }PeerLineageFailure::QuarantinedUnavailable { attributed, required }PeerLineageFailure::Incomplete(retryable)EmptyAtQuorumNone of the three refusals is an invalid-credit verdict.
SettlementSlotWinandread_winning_settlement_slotare deleted, not deprecated — they existed only to carry the collapsed answer.2. An absence must be asserted. A member's 404 counts as
Absentonly when it says so (x-dsm-slot-outcome: absent). Any other 404 — a route miss reaching the fallback, which still carries the identity echo — isUnavailable.3. Durability was asserted by comment and set on one register of three. #767 added
SET LOCAL synchronous_commit = ontoclaim_settlement_slotonly;claim_faucet_ticketandclaim_economic_rootstill inherited whatever the image defaulted to. All three now open their write transaction through onebegin_durable_write, so the posture cannot hold for one register and silently lapse for another.require_durable_commit_posturestill refuses to start a node reportingfsync/full_page_writesoff, and its refusal decision is now a pure function over the readings, so both directions are testable without a differently configured server.4. CI never executed the Postgres backend. The node board runs
--no-default-features --features local-dev,strict— SQLite. The three write-once economic registers had their non-equivocation asserted only against a backend nobody deploys.db::write_once_propertiesand run against whichever backend is compiled.DSM_TEST_DATABASE_URLrather than skipping, so a job that loses its service container fails loudly instead of reporting a green board that executed nothing.Storage Node (Postgres)runs the node's default features (strict,postgres) against apostgres:16service, then counts the register tests that actually executed — a green board is not evidence that these tests ran.Mutation controls
Every one performed against a live Postgres 14, each restored with a verified tree stamp (
git diff | shasumidentical before and after).ON CONFLICT DO UPDATEsettlement_slotproperties; 16 of 16 racers accepted, re-ack became a second acceptance, a re-opened store no longer refused Bbegin_durable_writestops issuingSET LOCALa_claim_transaction_sets_its_own_durability_rather_than_inheriting_it— transaction reportedoffon a session weakened tooffa_weaker_posture_is_refused_and_the_refusal_names_the_settingDSM_TEST_DATABASE_URLunset on the pg buildThe durability test is deliberately hostile: it first weakens the session to
synchronous_commit = off(which the pool'sRecyclingMethod::Fastwould carry to every later borrow of that connection) and requires the transaction to still reporton. Readingonoff a server that was alreadyonwould prove nothing.Verification
Rust 1.98.0 — the version
rust-toolchain.tomldeclares. (Homebrew'scargo1.96.0 wins on PATH here;ci/check_toolchain_consistency.shcaught it, and every board below was re-run under the pinned toolchain.)make lint— exit 0cargo test --locked --workspace --exclude dsm_storage_node --release -- --nocapture --test-threads=1— 3979 passed, 0 failed--no-default-features --features local-dev,strict— 282 passed, 0 failedci/production_safety_checks.sh(incl. TLA+),scripts/ci_scan.sh,ci/no_clock_and_no_json.sh,scripts/check_forbidden_symbols.sh— all exit 0What this does NOT do — stated so the board is not read as more than it is
DSM_RUN_DB_TESTS=1. Those legacy suites (device_api,b0x,bytecommit_chain) skip themselves by default and do not pass against Postgres today — enabling them produces 7 failures (device registration errors, then 401 instead of 204/400). That is a pre-existing defect from tests that have never run in CI; resurrecting them is separate work and would make this board red for reasons unrelated to the registers.tests/economic_register_conformance.rsstayslocal-dev-only. It builds three independent members, one:memory:database each. Against one Postgres URL all three would share a database and a claim held at n1 would appear held at n2/n3, destroying the fan-out semantics it exists to test. Running it on Postgres needs three databases or per-member schemas — a separate change.Storage Node (Postgres)to the required checks is yours.72648db8remains parked.