Skip to content

fix(economic): an observer derives one of four answers about a slot cell, and an error is never emptiness - #767

Merged
cryptskii merged 1 commit into
mainfrom
fix/settlement-slot-observation-soundness
Sep 5, 2026
Merged

fix(economic): an observer derives one of four answers about a slot cell, and an error is never emptiness#767
cryptskii merged 1 commit into
mainfrom
fix/settlement-slot-observation-soundness

Conversation

@cryptskii

Copy link
Copy Markdown
Collaborator

Summary

Hardening the reader that turns member responses into an authoritative observation, before the settlement-slot register is promoted from market coordination to economic authority.

Given the vault's birth-committed storage set S, an observer derives exactly one of:

    Claimed(C)        a quorum returned byte-identical contents
    EmptyAtQuorum     a quorum EXPLICITLY reported no row — true of the moment it was read
    Conflict          contradictory non-empty claims, no quorum winner
    Unavailable       the read establishes nothing

What the gate review found

The write-once primitive is sound: one writer, insert-if-absent with same-transaction read-back on both backends, no update or delete path, durable before the acknowledgement, nothing replicates the claim table, no rollback path in code, and disjoint quorums impossible at an honest quorum. The defects were all in the reader.

# defect severity
1 any non-200 counted as "no value", and the node-id echo is an outermost layer present on 5xx — so a quorum of members with a failing database manufactured EmptyAtQuorum, the terminal condition of a forward walk high
2 the committed quorum was checked only for non-zero and not-larger-than-the-set, so a signed 1-of-3 was accepted — two disjoint claims each reach quorum high
3 the winner was chosen with find over a HashMap before the conflict check, so the choice depended on iteration order and the divergence went unreported high (with 2)
4 the provenance read resolved the cell against the verifier's configured fleet, not the vault's committed set; the composer never checked the winner's set at all medium

The repairs

  1. Member read semantics. Only an explicit 404 from a correctly attributed member is an absence. Transport failure, timeout, 5xx, an unreadable body, a missing or wrong node id, and a member this client cannot locate are all Unavailable. Attribution is folded in at the read, so an unattributed response is never counted in either direction.
  2. Canonical quorum. q must be the strict majority of the committed set, exactly — derived, never chosen — checked as soon as V_n is decoded, before anything downstream relies on it.
  3. Deterministic observation. The tally completes before anything is selected. Two values at quorum is a Conflict, not a race. A minority disagreement beside a strict-majority winner still resolves, because write-once rows mean the loser can never gain a majority.
  4. Authority-set consistency. The set is resolved by re-deriving its id from the member ids a catalog entry lists and requiring that to reproduce the signed id; configuration says where to reach a member and can never substitute a set. The composer now also refuses a winner claimed under a different set.

Proof

Every case you named, plus the mutation controls:

case result
3 members, all reads fail → Unavailable, never EmptyAtQuorum green
3 members, 2 explicit not-found + 1 unavailable → EmptyAtQuorum green
3 members, committed q=1 → refused as noncanonical, before the cell is read green
A,A,BClaimed(A) green
A,B,unavailableConflict green
A,empty,unavailableUnavailable green
claim from a different storage_set_id → refused green
a mis-echoed or failing member is never counted as an absence green
a foreign set does not resolve from the catalog green
mutation, one per repair plus the transport classification each reddens only its own named test; restored checksum-verified
boards workspace 74 binaries, 3978 passed / 0 failed / 17 ignored (dsm lib 1699/0, dsm_sdk lib 1816/0/7 ignored), exit 0; node 11 binaries, 276/0, exit 0; root make lint exit 0; ci/production_safety_checks.sh PASS (pinned 1.98.0, incl. TLA+); scripts/ci_scan.sh PASS — all on tree stamp a5b264255c88

One honest note on the mutation work: the transport arm was green on the first attempt, because no test reached that branch — a gap in the test rather than a passing gate. The test was extended to cover a failing member and the cycle rerun, which then reddened.

A scope note you should see

beta_storage_profile::is_conformant_commitment is removed. It was the uncalled verifier-side rule this change replaces, and its test is re-rooted onto the canonical one; the two agree at the deployed three-member cardinality. The module's constants stay — one is used by the root-register profile.

The "wrong local fleet, correct birth set" case is covered by the catalog re-derivation test rather than end to end, because in beta the configured fleet and every vault's birth set are byte-identical, so the two are not behaviourally distinguishable today. Stated rather than claimed as more than it is.

Test-name reconciliation

+11 / −2. The two removals are replacements, each with a named successor covering the same proposition at the layer the property now lives: a_committed_q_is_checked_against_the_profile_not_recomputedthe_profile_threshold_is_the_canonical_quorum_for_the_deployed_fleet, and an_unattributed_read_row_is_uncountablea_mis_echoed_or_failing_member_is_never_counted_as_an_absence, which now also covers an outage.

The storage-node conformance suite is the dependent this change owed a sweep, and it caught it: that test pins the fake fleet to real node behaviour, and it now asserts against a real node that an unreachable member reads Unavailable and never an absence.

Not in this PR

The vault-lineage verifier, the depth bound, and griefing. Per the locked order, the adversarial gate review is rerun from scratch against the merged tree, and the lineage work starts only if it comes back clean.

…ell, and an error is never emptiness

Before the settlement-slot register can carry economic uniqueness, the reader
that turns member responses into an authoritative observation has to be sound.
The write-once primitive underneath already is — one writer, insert-if-absent,
same-transaction read-back, durable before the ack, nothing replicates the
table, no rollback path — but none of that survives a reader that treats "I
could not ask" as an answer. An adversarial review of that reader found two
high-severity defects and two lesser ones. All four are repaired here.

    Claimed(C) | EmptyAtQuorum | Conflict | Unavailable

1. MEMBER READ SEMANTICS. Any non-200 became "no value", and the node-identity
   echo is an outermost response layer so it is present on 5xx too. A quorum of
   members with a failing database therefore MANUFACTURED `EmptyAtQuorum` — the
   one observation the design treats as a fact, and the terminal condition of a
   forward lineage walk. Now only an explicit 404 from a correctly attributed
   member is an absence; transport failure, timeout, 5xx, an unreadable body, a
   missing or wrong node id, and a member this client cannot locate are all
   `Unavailable`, which supports no conclusion in either direction.

2. CANONICAL QUORUM. The committed `q` was checked only for being non-zero and
   not larger than the set, so a signed `1-of-3` was accepted — under which two
   disjoint claims each reach quorum and different verifiers fold different
   chains. `q` must now BE the canonical strict majority of the committed set,
   exactly, checked as soon as `V_n` is decoded and before anything downstream
   relies on it. Quorum is derived, never chosen.

3. DETERMINISTIC OBSERVATION. The old tally selected a winner with `find` over
   a `HashMap` and only then looked for disagreement, so with a nonconformant
   `q` the choice depended on iteration order and the divergence was never
   reported. The tally now completes before anything is selected; two values at
   quorum is a conflict rather than a race. A minority disagreement BESIDE a
   strict-majority winner still resolves, because write-once rows mean the
   loser can never gain a majority — `A,A,B` is `Claimed(A)`.

4. AUTHORITY-SET CONSISTENCY. The provenance read resolved the cell against
   whatever fleet the verifier was configured for; it now resolves the set the
   vault COMMITTED, by re-deriving that set's id from the member ids a catalog
   entry lists and requiring it to reproduce the signed id. Configuration says
   where to reach a member and can never substitute a set. The composer also
   now refuses a winner claimed under a different set, which the provenance arm
   already did — a member reconfigured into another set keeps serving rows
   written under the old one.

`EmptyAtQuorum` means no quorum claim is observable NOW. It has never meant the
generation can never be claimed, and the walk's break comment says so.

Proven: three failing reads are `Unavailable`, never empty; two explicit
not-founds plus one outage are `EmptyAtQuorum`; `A,A,B` resolves to `A`;
`A,B,unavailable` is a `Conflict`; `A,empty,unavailable` is `Unavailable`; a
committed `1-of-3` is refused before the cell is read; a winner claimed under
another storage set is refused; a mis-echoed or failing member is never counted
as an absence; a foreign set does not resolve from the catalog. Five mutation
cycles, one per repair plus the transport classification, each reddening only
its own named test with checksum-verified restoration. One arm was green on the
first attempt because no test reached that branch — a gap in the test, fixed
before the cycle was rerun.

`beta_storage_profile::is_conformant_commitment` is removed: it was the
uncalled verifier-side rule this change replaces, and its test is re-rooted onto
the canonical one, which agrees with the profile at the deployed cardinality.

NOT in this change: the vault-lineage verifier, the depth bound, griefing.

Boards: workspace 74 binaries, 3978 passed / 0 failed / 17 ignored (dsm lib 1699/0, dsm_sdk lib 1816/0/7 ignored), exit 0; node 11 binaries, 276/0, exit 0; make lint exit 0; production
safety PASS (pinned 1.98.0); ci_scan PASS.
@cryptskii
cryptskii merged commit de45b32 into main Sep 5, 2026
15 checks passed
@cryptskii
cryptskii deleted the fix/settlement-slot-observation-soundness branch September 5, 2026 04:01
cryptskii added a commit that referenced this pull request Sep 5, 2026
… backend proves the registers (#768)

* fix(economic): a slot observation keeps its taxonomy, and the shipped backend proves the registers

#767 gave the four-valued cell observation a sound core but let four of the
five read paths collapse it back to two answers, and the register properties
it proved were only ever executed against SQLite while the fleet deploys
Postgres. This closes both, plus the durability posture that was asserted by
comment and set nowhere.

The resolver contract no longer has a channel a caller can collapse.
`settlement_slot_observation` returns `CellObservation` — not `Option`, not
`Result` — so a transport failure IS `Unavailable` and there is no `None`
standing for four different things. `verify_transition_provenance` matches
all four arms and gives each its own verdict: `Conflict` quarantines,
`Unavailable` is retryable, `EmptyAtQuorum` is an evidence refusal saying
this settle never won the parent it names, and only `Claimed` proceeds.
None of the three refusals is an invalid-credit verdict. `SettlementSlotWin`
and `read_winning_settlement_slot` are deleted rather than deprecated: they
existed only to carry the collapsed answer.

A member's 404 now counts as emptiness only when it SAYS the cell is absent
(`x-dsm-slot-outcome: absent`). Any other 404 — a route miss reaching the
fallback, which still carries the identity echo — is `Unavailable`. An
absence must be asserted, not inferred from a status code.

Durability is set, not inherited. All three of the node's one-shot register
claims — settlement slot, faucet ticket, economic root — now open their write
transaction through one `begin_durable_write` helper that issues
`SET LOCAL synchronous_commit = on`, so the posture cannot hold for one
register and silently lapse for another; only the settlement slot had it
before. `require_durable_commit_posture` refuses to start a node whose server
reports `fsync` or `full_page_writes` off, and its refusal DECISION is now a
pure function so both directions are testable without a differently
configured server.

The write-once properties are stated once for all three registers in
`db::write_once_properties` and run against whichever backend is compiled.
On Postgres they refuse to run without `DSM_TEST_DATABASE_URL` rather than
skipping, so a job that loses its service container fails loudly instead of
reporting a green board that executed nothing. A new `Storage Node
(Postgres)` CI job runs the node's DEFAULT features against a real server and
then counts the register tests that executed, because a green board is not
evidence that these tests ran.

Mutation controls, all performed against a live Postgres 14 and restored:
- slot write-once -> `DO UPDATE`: all three settlement-slot properties red,
  16 of 16 racers accepted.
- `begin_durable_write` stops setting the posture: the transaction reports
  `off` on a session weakened to `off`; the named test red.
- the posture gate stops refusing: the weak-posture test red.

Verification (Rust 1.98.0, the version rust-toolchain.toml declares):
- root `make lint` exit 0
- workspace board 3979 passed, 0 failed
- node board (local-dev,strict / SQLite) 282 passed, 0 failed
- node board (default features / Postgres 14) 252 passed, 0 failed
- ci/production_safety_checks.sh, scripts/ci_scan.sh,
  ci/no_clock_and_no_json.sh, scripts/check_forbidden_symbols.sh all exit 0

* fix(ci): the Postgres board supplies the node's own database URL, not just the register-test one

registry_gate builds a real app through `build_app_for_tests`, which reads
DSM_DATABASE_URL and whose Postgres default is a placeholder that was never
meant to resolve. With only DSM_TEST_DATABASE_URL set, the job reached a
listening server that demanded a password and failed with 'password missing'.

This did not reproduce locally because this machine already had a `dsm_storage`
database with trust auth — the default URL happened to work here and cannot
work on a fresh runner. Re-verified against a freshly created, empty database
with both variables set, which is the shape the job actually has:
241 lib + 11 integration passed, 0 failed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant