User story: As a developer embedding d-engine in my application, when a replica silently falls behind due to an internal replication fault, I expect my application to be notified directly — not have to parse d-engine's log output — so I can decide whether to alert, restart that replica, or intervene manually.
Current state: d-engine only logs these faults (error!) and increments an internal counter. A write can still succeed via quorum while one peer is silently broken, and nothing surfaces this to the embedding application. Existing host-facing notifications (leader_change_notifier, membership_change_notifier) only report current state, not discrete events, so they don't fit this case either. For now, host apps must rely on d-engine's log output.
Additional scope discovered while reviewing #436's snapshot install path (2026-08-18):
-
Missing SnapshotResponse health event for BoundaryConflict — when a follower detects
a snapshot boundary conflict (same index, different term — a sentinel anomaly that
shouldn't occur under correct Raft operation, likely signals data corruption or a prior
bug elsewhere), it's currently reported as a generic SnapshotError::OperationFailed,
indistinguishable from an ordinary transient install failure (disk full, network drop).
The host app has no way to tell "this peer might have corrupted data" apart from
"this peer had a routine hiccup, will retry."
-
Missing notification when snapshot install fails due to I/O errors (e.g. snapshot_dir
missing/corrupted mid-transfer) — the node stays alive and the leader retries on a
backoff schedule (correct behavior), but if the root cause is persistent (permissions,
disk, misconfiguration), the node retries forever with only a repeated log line — no
program-observable signal that this peer is permanently stuck.
Both are instances of the same gap this ticket already covers (log-only, no host-facing
event) — should be designed as part of the same notification mechanism, not bolted on
separately.
User story: As a developer embedding d-engine in my application, when a replica silently falls behind due to an internal replication fault, I expect my application to be notified directly — not have to parse d-engine's log output — so I can decide whether to alert, restart that replica, or intervene manually.
Current state: d-engine only logs these faults (
error!) and increments an internal counter. A write can still succeed via quorum while one peer is silently broken, and nothing surfaces this to the embedding application. Existing host-facing notifications (leader_change_notifier,membership_change_notifier) only report current state, not discrete events, so they don't fit this case either. For now, host apps must rely on d-engine's log output.Additional scope discovered while reviewing #436's snapshot install path (2026-08-18):
Missing
SnapshotResponsehealth event forBoundaryConflict— when a follower detectsa snapshot boundary conflict (same index, different term — a sentinel anomaly that
shouldn't occur under correct Raft operation, likely signals data corruption or a prior
bug elsewhere), it's currently reported as a generic
SnapshotError::OperationFailed,indistinguishable from an ordinary transient install failure (disk full, network drop).
The host app has no way to tell "this peer might have corrupted data" apart from
"this peer had a routine hiccup, will retry."
Missing notification when snapshot install fails due to I/O errors (e.g. snapshot_dir
missing/corrupted mid-transfer) — the node stays alive and the leader retries on a
backoff schedule (correct behavior), but if the root cause is persistent (permissions,
disk, misconfiguration), the node retries forever with only a repeated log line — no
program-observable signal that this peer is permanently stuck.
Both are instances of the same gap this ticket already covers (log-only, no host-facing
event) — should be designed as part of the same notification mechanism, not bolted on
separately.