Skip to content

Establish mainnet-launch: adopt DDC work onto the Mainnet line, preserving master's pallet indices - #734

Open
jaxter03 wants to merge 8 commits into
mainnet-launchfrom
feat/port-pallets
Open

Establish mainnet-launch: adopt DDC work onto the Mainnet line, preserving master's pallet indices#734
jaxter03 wants to merge 8 commits into
mainnet-launchfrom
feat/port-pallets

Conversation

@jaxter03

@jaxter03 jaxter03 commented Aug 19, 2026

Copy link
Copy Markdown
Member

What this is

The Mainnet-targeted port of the DDC work from staging. Base is mainnet-launch, which is master unchanged.

⚠️ Migrations are deliberately not wired — and are NOT yet safe to wire

They arrive present and provably inert: migrations::Unreleased has zero references and pallet_migrations::Config::Migrations is (), so a dry run can demonstrate nothing runs. Wiring and fixing them is a separate PR.

That separation matters, because the storage-migration evaluation found the migration set is not safe to apply to Mainnet today. Dry-run against a Mainnet fork at block 26,666,018 found six blockers, including:

Pallet / storage Finding
DdcPayouts v2 skips all 58 Mainnet billing reports (decode fails, translate skips rather than deletes); v3 then reads the same bytes under a misaligned layout that is also 167 bytes, succeeds, and writes values inflated ~4.2e12× into PayoutReceipts. try-runtime's pre/post checks pass vacuously — both count 0.
DdcCustomers v5_mbm emptied 443 of 535 ledgers into free balances with no deposit registered — Mainnet has zero contracts deployed, and the deposit contract cannot exist until the clusters chain reaches v4. v4_mbm also copies rather than moves, stranding all 535 under the old prefix.
DdcClusters v3's guard is current_version == 3 against an in-code version of 6 — unsatisfiable, so the whole 2→6 chain stalls at 2.
DdcNodes The one migration that would work (v0_v2) is commented out.
Idempotency The check fails with a storage-root mismatch; culprit not yet bisected.

This PR removes three further blockers by construction — the Balances::Holds (17.16M CERE), TechComm::ProposalOf and Scheduler::Agenda value-shape breakage — by appending DdcVerification at index 54 rather than inserting at 40. Those needed no migration once the indices stop moving.

One thing this PR improves for the evaluation itself: keeping master's #[cfg(feature = "try-runtime")] type DisabledValidators = () means a runtime built from this line can drive multi-block migrations to completion. Building from staging could not — that is what stalled evaluation round 1.

Full analysis: docs/MAINNET_STORAGE_MIGRATION_DRYRUN.md, and the live memory-bank page 02-cere-protocol/blockchain/mainnet-storage-migration-evaluation.md.

Why it is a port and not a merge

master and staging are two long-lived lines, not one ahead of the other:

Merge base ffd39b4c
Commits on master not in staging 201
Commits on staging not in master 1,058

Both migrated to polkadot-sdk = 2512.3.2 independently, so a trial git merge origin/staging produced 72 conflicted files in code neither side solely owns. Every change here was adopted deliberately, file by file and hunk by hunk, with master as the default.

Selective Bulk copy would be
Files 91 133
Lines +9,269 / −9,982 +11,997 / −11,435
runtime/cere/src/lib.rs +276 / −86 +474 / −304
runtime/cere-dev/src/lib.rs +190 / −36 +360 / −236

Of 93 hunks in runtime/cere/src/lib.rs, 49 were pure import-path churn (staging drops the polkadot_sdk:: prefix) and were rejected. cere-dev: 46 of 85.

The part that matters most — master-only content preserved

Each of these would have been silently replaced by a bulk copy. The first three are Mainnet-correctness issues.

Kept Staging's value Consequence if lost
StateMachine::Polkadot(3367) StateMachine::Kusama(4009) Mainnet's ISMP bridge would address the Gargantua testnet coprocessor
EitherOfDiverse<EnsureRoot, EnsureMembers<TechCommCollective, 3>> EnsureRoot only Technical Committee loses its ismp origin capability
#[cfg(feature = "try-runtime")] type DisabledValidators = () absent try-runtime cannot author blocks against Mainnet state — this is what stalled storage-migration evaluation round 1
pallet-ismp / ismp-grandpa benchmark forwards dropped Hyperbridge crates are outside the umbrella and need them
pallet-ddc-clusters-gov/std forward dropped staging drops the forward while keeping the dep — appears to be a staging bug
sc-cli = { default-features = true } absent rocksdb, which the umbrella does not forward

Deliberate divergence from both branches

DdcVerification appended at index 54, not inserted at 40

Staging inserts it at 40, shifting every pallet below. Three storage items encode a pallet index in their SCALE-encoded values, and none change storage version — so a version sweep cannot see the damage:

Storage Mainnet exposure
Balances::Holds (RuntimeHoldReason) 24 accounts, 17,164,730.5167 CERE would read as empty
TechComm::ProposalOf (RuntimeCall) 1 proposal, permanently un-closeable
Scheduler::Agenda (OriginCaller) 1 of 3 slots silently never fires

Appending leaves every existing index untouched, so none of those need a migration. Verified programmatically: zero master indices moved.

ddc-customers call indices

Staging inserts deposit_for at index 3, shifting four extrinsics — a caller targeting unlock_deposit at 3 would instead invoke a deposit. Renumbered so deposit_for lands at 7 and Mainnet's encoding is preserved.

Versions

spec_version 73159 (master + 1), transaction_version 26 — the latter must bump, since ddc-customers gained an extrinsic and fee-handler lost one.

One genuine feature loss: fee-handler

Staging's version is taken, which drops fee_distribution_config and its storage, and moves burn_native_tokens from index 2 to 1.

This is a deliberate rework on staging's side, not an accident — it also adds a WeightInfo type, benchmarks, tests and a mock that master never had. But handle_fee differs in behaviour: master splits fees treasury/fee-pot per stored config; staging sends everything to the fee pot.

Relevant facts: the config has never been set on Mainnet (None), and master never wires FeeHandler into anything — so its splitting logic has never run in production. Staging does wire it, so this PR makes fee handling active for the first time.

Open question for whoever owns fee handling: is the configurable treasury split still wanted? If yes it is ~40 lines on top of staging's version. Where payout fees go is an economic decision, not a porting one.

Verification

cargo check --workspace          exit 0, no errors
cargo test  --workspace          38 of 39 suites green

One test failure, in pallet-ddc-clusters-gov's own mock runtime:

mock_clusters_gov::__construct_runtime_integrity_test::runtime_integrity_tests
panicked at pallet-scheduler-46.0.0/src/lib.rs:453: Must be possible to submit a small lookup

Our pallets/ddc-clusters-gov is byte-identical to staging's (empty diff), and the test exercises the pallet's own mock, not the assembled runtime — so it is very likely pre-existing. I could not confirm that directly: building that pallet standalone on a staging worktree fails earlier, on an unrelated pallet-ddc-staking unresolved module std error. Treat as unconfirmed rather than dismissed.

Review guidance

The commits are ordered and each is self-contained: dependency swap → pallets → contract → runtimes → node → docs → lock. The runtime commit (7e45a665) is where the judgement calls are and is worth the most attention.

docs/MAINNET_LAUNCH_PORT_RECORD.md records every decision with its reasoning.

Follow-ups

  • Migration wiring and fixes — separate PR
  • External crates track branches, not pinned commits; pin before Mainnet
  • ddc-payouts' PR is held pending its pallet-ddc-customers dev-dep being repointed at this branch
  • Confirm or fix the ddc-clusters-gov integrity test

🤖 Generated with Claude Code

jaxter03 and others added 7 commits August 20, 2026 03:08
`staging` moved four things out of this repo. This commit follows, and does
nothing else — the tree does not build until the pallets and runtime catch up
in the next two commits.

  primitives/            -> ddc-primitives          (external)
  pallets/ddc-payouts/   -> pallet-ddc-payouts      (external)
  (new)                  -> ddc-api, ddc-dac-host,
                            pallet-ddc-verification

All five reference `branch = "mainnet-launch"`, never "staging". Cargo keys a
git source by URL *and* branch, so one reference left on "staging" resolves the
same commit as a second, distinct crate and breaks trait bounds with errors
that do not name the cause. This was hit for real in ddc-payouts.

Also adds the workspace deps the external crates need (prost, serde_with,
scopeguard) and `ink` for contracts/customer-deposit.

Workspace version 7.5.0 — continues master's 7.4.0. Staging's 8.0.1 belongs to
a separate lineage.

Rejected from staging's version of this file: the members-list reordering and
the clap/codec line swap, both pure churn.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
Code only; these pallets stay in this repo. No
Cerebellum-Network/ddc-{customers,clusters,nodes,staking,clusters-gov}
repositories exist — only ddc-payouts and ddc-verification were extracted.

An API-surface diff across all 11 pallets found exactly one genuine removal
(fee-handler, below). Everything else flagged was a rename:
AccountsLedger->CustomerLedger, sub_account_id->cluster_vault_id,
OldCluster-> versioned v0/v1/v2::Cluster.

DELIBERATE DIVERGENCE — ddc-customers call indices.

Staging inserts `deposit_for` at index 3, shifting four extrinsics down. A
caller targeting `unlock_deposit` at index 3 would instead invoke a *deposit*.
Renumbered so Mainnet's existing encoding is preserved:

  0 create_bucket  1 deposit  2 deposit_extra  3 unlock_deposit
  4 withdraw_unlocked_deposit  5 set_bucket_params  6 remove_bucket
  7 deposit_for    <- appended

ddc-clusters needed no such fix; it appends `join_cluster` at 5 correctly.

fee-handler: staging's version taken, which drops the `fee_distribution_config`
extrinsic and its storage, and renumbers `burn_native_tokens` 2 -> 1. This is a
deliberate rework on staging's side, not an accident — it also adds a WeightInfo
associated type, benchmarks, tests and a mock that master never had. Mainnet has
never set the config (storage reads None) and master never wires FeeHandler into
anything, so its splitting logic has never run in production. See
docs/MAINNET_LAUNCH_PORT_RECORD.md for the open question this leaves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
New top-level directory and workspace member, absent from master entirely.

Its Cargo.toml carries its own `ddc-primitives` git reference, repointed to
mainnet-launch along with the rest — easy to miss, and a single reference left
on "staging" would produce a duplicate crate.

Wired in both runtimes by `CereChainExtension` (next commit): the contract calls
func_id 1 to fetch the DdcPayouts pallet account so it can authorise incoming
`charge` calls.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
Of 93 hunks in runtime/cere/src/lib.rs, 49 were pure import-path churn
(staging drops the `polkadot_sdk::` prefix) and were rejected. cere-dev: 46 of
85. Result is +276/-86 and +190/-36 where a bulk copy would be +474/-304 and
+360/-236.

DELIBERATE DIVERGENCE — DdcVerification appended at index 54, not inserted at 40.

Staging inserts it at 40, shifting every pallet below. Three storage items
encode a pallet index in their SCALE-encoded *values*, and none of them change
storage version, so a version sweep cannot see the damage:

  Balances::Holds       (RuntimeHoldReason)  24 accounts, 17,164,730.5167 CERE
  TechComm::ProposalOf  (RuntimeCall)        1 proposal, would become un-closeable
  Scheduler::Agenda     (OriginCaller)       1 of 3 slots, would silently read empty

Appending leaves every existing index untouched, so none of those need a
migration. Verified programmatically: zero master indices moved.

MASTER-ONLY CONTENT PRESERVED. Each would have been lost by a bulk copy:

  StateMachine::Polkadot(3367)     staging targets Kusama(4009), the Gargantua
                                   TESTNET coprocessor — wrong for Mainnet's bridge
  TechComm ismp origins            staging collapses EitherOfDiverse<EnsureRoot,
                                   EnsureMembers<TechCommCollective,3>> to EnsureRoot
  Babe DisabledValidators bypass   #[cfg(feature = "try-runtime")] type
                                   DisabledValidators = (); absent on staging. This
                                   is what lets try-runtime drive multi-block
                                   migrations — its absence stalled evaluation round 1
  pallet-ismp/ismp-grandpa
    runtime-benchmarks forwards    dropped by staging in both runtime Cargo.tomls
  pallet-ddc-clusters-gov/std      staging drops the forward while keeping the
                                   dependency — appears to be a staging bug

Versions: spec_version 73159 (master's 73158 + 1). transaction_version 26 — this
MUST bump, because ddc-customers gained an extrinsic and fee-handler lost one.

Migrations: staging's `migrations` module is included and wired to NOTHING —
`migrations::Unreleased` has zero references, `pallet_migrations::Config::
Migrations` is `()`. The "no migrations run" property is enforced by the type, so
a dry run can prove it. Executive's tuple keeps master's two self-gating FRAME
migrations; master's two RemovePallet cleanups are dropped, as master's own
comment instructed — they shipped with 73158 and both prefixes read empty on
Mainnet.

API changes that surfaced as compile errors, not silent breakage:
pallet-pool-withdrawal-fix gained RuntimeEvent; pallet-ddc-staking merged
NodeVisitor + NodeCreator into NodeManager.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
An API-surface diff over node/ found nothing on master that staging lacks, so
staging's sources were taken directly.

Two exceptions, hand-merged rather than taken:

  node/cli/Cargo.toml   keeps master's `sc-cli = { default-features = true }`
                        rocksdb workaround alongside staging's ddc-dac-host dep.
                        Also takes default = ["cli", "cere-native",
                        "cere-dev-native"], which builds the Mainnet native
                        runtime by default.
  node/rpc/Cargo.toml   left at master's — its entire diff was comment
                        relabelling.

What staging brings: `ocw_heap_pages` / `--ocw-heap-pages N`, giving off-chain
workers a dynamic heap up to 512 MB separate from the on-chain 128 MB. It exists
because DdcVerification runs OCWs, so it belongs with that pallet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
MAINNET_LAUNCH_PORT_INVENTORY.md — why this is a manual port rather than a
merge (merge base ffd39b4; 201 commits on master, 1,058 on staging; both
migrated to polkadot-sdk 2512.3.2 independently, so a trial merge produced 72
conflicts), what must not move, and the branch/PR plan.

MAINNET_LAUNCH_PORT_RECORD.md — what was actually taken from staging, what was
kept from master and why, the deliberate divergences, and the verification.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
All five Cere external crates resolve through `branch = "mainnet-launch"`.
No `branch = "staging"` reference remains, so there is no duplicate-crate
hazard — cargo keys a git source by URL *and* branch, and a mixed label would
resolve the same commit as two distinct crates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
@jaxter03 jaxter03 changed the title port: adopt staging's DDC work onto the Mainnet line, preserving master's pallet indices Establish mainnet-launch: adopt DDC work onto the Mainnet line, preserving master's pallet indices Aug 19, 2026
`pallet-ddc-payouts::Config` requires an `InspectorAuthority`. Testnet binds it
to `DdcVerification` -- but bringing that pallet in also forces a fifth
`SessionKeys` entry, `UpgradeSessionKeys` (a flat 50% of `max_block`), and
starts its off-chain worker on Mainnet every tenth block.

DDC payouts have not run on Mainnet since 2024-06-30. All 58 billing reports
predate that; 53 are Finalized and 5 have been stuck mid-charge for two years.
Freezing a facility that has been dormant for 781 days costs nothing, and buys
a materially smaller first release.

So `NoInspectorAuthority` denies everything: payout extrinsics reject with
`Unauthorized`. No migration calls this trait -- all 21 call sites are in the
payout processing path -- so the storage migrations are unaffected.

Removed with it:
  - DdcVerification pallet (was index 54) and its Config
  - OldSessionKeys, the ddc_verification key, transform_session_keys.
    SessionKeys is now byte-identical to master's, so validators' existing keys
    decode unchanged and no session-key migration is needed.
  - UpgradeSessionKeys, and with it the 50% of max_block that made the upgrade
    block exceed the weight limit (evaluation round 2 measured 131.20%)
  - pallet-ddc-verification from both runtime Cargo.tomls; it was only a
    dev-dependency of ddc-payouts, so nothing else required it
  - the ddc_verification session key from node/service/src/chain_spec.rs

REPLACE the stub with DdcVerification in the release that re-enables payouts.
`add_inspector` returns an error under runtime-benchmarks rather than Ok(()),
so a benchmark cannot silently succeed against an authority that denies all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FdHHPhppLJhGdBfySiJiJY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant