fix(holdings): re-announce holdings when the peer pool rises from zero (#1734)

[MichaelTaylor3d]

Closes DIG-Network/dig_ecosystem#1734 (P0 / MVP).

The defect

A node that pins content while its peer pool is empty never announces it, and never re-announces when peers arrive. It holds the capsule, has recorded it as announced, and is invisible to every peer it later connects to.

reconcile_and_announce floods an opcode-222 announcement only for a NON-EMPTY InventoryDelta diffed against the node's OWN local DHT provider records. That diff answers "what changed here", never "what do my peers know", and the two diverge silently as soon as an inventory change happens with nobody connected: the pin moves the local records, the flood reaches zero peers, and every later reconcile of the same inventory is a no-op.

One correction to the ticket's mechanism. Bring-up does not flood at all. bring_up_dht calls dht::announce_inventory (a local DHT PUT) and then DhtHandle::new(service, initial_ids), seeding the remembered announced-id set from the on-disk cache before the broadcaster exists. So the empty-pool flooded … peers=0 line is a wasted frame, not the poisoning step; the poisoning is reconcile_inventory overwriting the remembered set unconditionally. The consequence is stronger than reported: a node RESTARTING with content already cached has a permanently empty delta and never floods even with peers connected.

The fix — (a) and (c) merged; (b) rejected

Implemented: the node re-states its holdings IN FULL — every held content id as an Add, no diff — whenever its connected pool rises from zero peers to some, including the first such observation after bring-up. That single rule is both the ticket's (a) 0→N re-announce and its (c) don't-announce-into-an-empty-pool: rather than skipping the bring-up announce, it DEFERS the statement until someone is listening. A pool that merely grows while already peered does not re-flood.

Why this and not the others:

• (a) alone (a delta re-announce on the edge) is the nearest wrong fix, and the added test reds for it: the pin already moved the local records, so the edge recomputes an EMPTY delta and puts nothing on the wire. Re-stating the full inventory is what makes the repair independent of the poisoned state. It is safe because an Add is idempotent at every receiver under an advancing seq.
• (c) alone fixes nothing — the poisoning is the record diff, not the flood, so suppressing the empty-pool flood leaves the no-op reconcile exactly as it was.
• (b) remote acknowledgement was rejected as both larger and unnecessary: opcode 222 is a fire-and-forget flood with no ack in the wire format, so "announced means a peer received it" would be a protocol change. An idempotent full re-statement on every 0→N reaches the same property — "I hold X" and "peers know I hold X" cannot stay diverged — at one frame per inventory batch.
• All peers drop and return is the same transition by construction, so it re-announces; the presence state machine re-arms on the fall to zero, and the test pins both directions of the edge.
• Residual, deliberately not built (hardening is postponed): a NEW peer joining an already-peered pool receives no re-statement. It discovers this node through the durable DHT provider records, which by then have been PUT with routing populated. Worth a follow-up only if measurement shows it matters.

The test, and where it asserts

holdings_pinned_before_the_first_peer_reach_that_peer_when_it_connects (crates/dig-node-core/tests/holdings_wire.rs) — two live gossip nodes over a loopback mTLS link, real NodeCerts, real signature, real frame, real inbound decode.

It asserts at the RECEIVING node's ingest — HoldingsIngress::accept returning ingested >= 1 and find_providers returning the holder — never on the sender. The sender's flooded an opcode-222 announcement line already prints on the broken path with peers=0, so any assertion keyed on it passes against the defect; that was the specific vacuity to avoid. The fixture drives the real ordering (pin at zero peers through the real reconcile_and_announce with the real pool as transport, THEN connect), because the defect only exists as an ordering across two nodes.

Proven load-bearing by two reverts, committed first:

Revert	Result
The announcer states nothing (defect behaviour restored)	FAILED (20s timeout: no frame reached the receiver)
The peer edge re-runs the delta reconcile_and_announce (the nearest wrong fix)	FAILED (empty delta, nothing on the wire)

every_zero_to_nonzero_peer_transition_re_arms_the_announce pins the edge from both sides: a rise fires, a further rise while peered does NOT, a fall to zero re-arms.

Blast radius checked

gitnexus is disabled in the loop (temp override), so this was done with ripgrep plus a direct read of every caller.

• Added, not modified: announce_all_holdings, HoldingsInventory, PoolPresence, run_first_peer_announcer (holdings.rs); NodeHoldingsInventory + one tokio::spawn at bring-up (peer.rs). No existing symbol's signature or behaviour changed — reconcile_and_announce, HoldingsBroadcaster::announce_change, DhtHandle::reconcile_inventory, sync_inventory are untouched.
• The only behavioural addition to a running node is one extra full-inventory announcement per 0→N pool transition. Risk: LOW.
• Full workspace suite green (450 + 242 + 289 unit, all integration suites), cargo fmt --check clean, cargo clippy --workspace --all-targets clean.

Version

0.64.0 → 0.65.0 (minor): new announce behaviour, additive, no API/wire/format change. dig-node-core 0.22.0 → 0.23.0 to match. SPEC.md §19.3a gains the normative peer-presence re-announce rule.

[MichaelTaylor3d]

Verdict: PASS (correctness review, independent fresh context)

Merge-clear. Both claimed reverts reproduced RED in my own worktree (C:/tmp/worktrees/rv-1734, since removed; no shared checkout touched), including the load-bearing one.

Reverts, run here. Baseline cargo test -p dig-node-core --test holdings_wire 8/8 green. (A) announce_all_holdings neutered to return 0 -> the wire test FAILS on the 20s timeout, no frame at the receiver. (B) the load-bearing one: the fixture announcer replaced with the nearest wrong fix, a re-run of reconcile_and_announce against the same DhtHandle on the peer edge -> FAILS, 20s timeout, no frame arrived. The delta really is empty because the zero-peer pin already moved the local records, so the test discriminates FULL RE-STATEMENT from ANY ANNOUNCE AT ALL. It names the property, not an outcome.

Reaches the wire on the real path. The discriminating assertion is receiver-side only (inbound_receiver -> provider_peer_id match -> HoldingsIngress::accept -> ingested >= 1 -> find_providers returns the holder); nothing on the sender, which is right because the sender flood log prints with peers=0 on the broken path. The fixture drives the real ordering: pin at zero peers through the real reconcile_and_announce with the real pool as transport, assert the records moved, then dial. Because the HOLDER is the dialing side and the test passes, an outbound connect demonstrably raises a PoolEvent, which is what a dial-out-only node depends on in production.

Parity. announce_all_holdings uses inventory_content_ids, the same helper reconcile_inventory assigns to announced (seams/dig_peer/dht.rs:505), and NodeHoldingsInventory::current calls cache_list_cached(), the same call as peer.rs:890. The re-statement cannot advertise a shape or inventory the reconcile path does not.

Edge, both directions. PoolPresence::observe is the only gate; the unit test pins rise-fires, growth-does-not, fall-re-arms. Not one-sided.

Idempotence. An Add is a sink.ingest(ProviderRecord::new(..)) refresh of the same (content_key, provider) entry: a receiver that already holds it is refreshed, a duplicate content id costs one extra map write. announce_change advances seq per batch via fetch_add.

SemVer. 0.65.0 workspace / 0.23.0 dig-node-core, Cargo.lock agrees, no package.json, dependents are path deps with no version pin. Additive public surface only; no wire, format or signature change. MINOR is correct.

SPEC 19.3a matches the shipped behaviour (all held ids as Add, no diff, on 0->N including the first post-bring-up observation, not on growth). No surviving text asserts delta-ONLY announcement; the reshare rule about reusing the one inventory-reconcile path is a different trigger, and the parity above means the two agree regardless.

Item 3, the seq / wall-clock interaction: REAL as a mechanism, REFUTED as a reason this PR fails to repair production. NOT gating.

IngressState::admit (holdings.rs:586-590) rejects seq <= highest as StaleSeq and never resets the watermark on a window elapse, and the broadcaster is seeded from now_unix_secs() (peer.rs:2590). So a node whose batch count out-ran its uptime in seconds does restart below a still-running receiver watermark. It does not gate because: (a) both the clock seed and the StaleSeq gate shipped in #1429 and are present at dd6bc46, so this PR neither introduces nor worsens it, adding one batch per 0->N transition; (b) SPEC 19.3a already documents it verbatim, naming the consequence as self-inflicted silence rather than an accepted replay, tracked to #1477; (c) the regime needs batches > uptime_seconds averaged over the whole run, i.e. sustained above one inventory-change batch per second, and idle time repays the deficit; (d) it also needs the receiver to have stayed up and retained the provider entry, whereas the production picture behind the zero-providers symptom is receivers restarting too, with empty watermarks. Belongs to #1477.

Non-gating notes

Three inline notes, posted and resolved by me. None blocks merge.

Not examined

Coverage arithmetic (taken as reported, 83.07%); rate-limit and DoS bounds and defense-in-depth (hardening postponed per the 2026-07-28 directive); the dig-node-core suite beyond holdings_wire plus the gate-reported green checks; dig-gossip internal PoolEvent emission beyond what the passing two-node fixture demonstrates.