feat(graph): 🔍 separate the graph's flat-world slot cost from its traffic

[diagonal-hamiltonian]

🤖 AI text below 🤖

Summary

graph_memory_breakdown() reported the graph's cross-rank storage as one number,
cross_rank_bytes, which mixes two costs that scale with different things:

• one record per slot of the flat world (ranks × partitions), whether or not that slot carries
  any traffic, and
• the endpoints actually crossing.

On a partitioned run the first grows with a P the rank count never shows, so a breakdown that adds
them cannot say which of the two a graph is spending on. This splits them.

What changed

Five fields on the existing GraphMemoryBreakdown, deliberately outside total_bytes(),
because each is a count or a slice of a field already summed there:

field	what it is
slot_record_bytes	the slot-proportional slice of cross_rank_bytes
layer_cores	distinct LayerCores; slot_records / layer_cores recovers P
slot_records	summed over cores, so P per core
occupied_slots	slots carrying any traffic; / slot_records is the occupancy
cross_rank_endpoints	the traffic itself, and a P-independent ceiling on occupied_slots

Three helpers in MPGraphEncoding.cpp derive them. cross_rank_storage_bytes now calls
cross_rank_slot_record_bytes rather than repeating its expression, so the slice cannot drift from
the total it is a slice of.

Surfaced to Python as five d_-prefixed keys on graph_memory_breakdown(), matching the d_
convention operator_memory_breakdown() already uses for diagnostics excluded from the sum.

Tests

Both sides of the binding, because they fail differently.

• cpp/tests/graph_encoding_tests.cpp — five cases. occupied_slots against zeros at the
  front, the interior and the back (the three places a scan loses count); slot_record_bytes
  against a pair of storages with identical traffic and different world sizes, which is the
  whole point of the field; total_bytes() unchanged by any diagnostic; and += summing all five,
  since a partitioned propagator merges per-partition breakdowns.
• tests/test_memory_breakdown.py — the dicts are built inside nanobind lambdas with no C++
  entry point returning the map, so which struct field each string key names is observable only
  from Python. It also pins monoprop_PARTITIONS; left alone, P resolves to the host's physical
  core count and every slot-record expectation becomes machine-dependent.

d_slot_record_bytes / d_slot_records is asserted to be exactly 32 or 40 bytes —
CrossRankPartnerRange is two size_t and three TermIndex, the latter 4 B or 8 B under
monoprop_WIDE_TERM_INDEX, and nothing in between.

Scope

This PR was much larger and is now much smaller. It previously also carried the
monoprop_PROFILE instrument. That instrument is held back: whether to ship an internal profiler at
all is a separate decision from whether the graph can report its own slot cost, and the two were
only ever in one PR because they were written in one sitting. The instrument is kept as a
cherry-pickable commit for measurement work and is not part of this diff. Consequences:

• Nothing here is behind a build flag or an environment variable. Five always-on counters, and
  grep -r 'monoprop_PROF\|profile::' over this diff returns nothing.
• @robertodr's fprintf comment no longer applies to this PR — that code is not here. It was
  six call sites, not eight (two of the grep hits were the word inside a comment), and they
  became one noexcept helper over std::format on the branch that still has them.
• The earlier A/B on this branch measured a superset of this diff and should not be read as this
  PR's result. For the record it resolved 0 of 24 timing tests against 48cadcb, with one memory
  cell at 1.01x and 15 flat — but that was the instrument plus this, not this.

Reviewers who remember closed #232 pr/layer-profile: that was the instrument's ancestor, not this.

Verification

ctest -L unit 220/220, -L serial 219/219, -L mpi, and the Python MPI suite at 1x1, 1x16, 2x8
and 8x16 — 600 passed at every layout, all suites green. Extension 3abe91dcace4eef73c43526d04e95a6e.
clang-format --output-replacements-xml reports 0 replacements on every changed C++ file.

[github-actions]

Docs preview: https://pr-253.monoprop-docs.pages.dev

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.70%. Comparing base (412315e) to head (1f97ea9).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #253   +/-   ##
=======================================
  Coverage   97.70%   97.70%           
=======================================
  Files          14       14           
  Lines         742      742           
  Branches       98       98           
=======================================
  Hits          725      725           
  Misses         12       12           
  Partials        5        5           

Flag	Coverage Δ
cpp	97.70% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
88.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[diagonal-hamiltonian]

🤖 AI text below 🤖

Closing this — the five counters have moved to the profiling-tool branch rather than to main, because nothing that ships reads them.

What actually decides the memory results is kernel peak RSS from /usr/bin/time -v around each rank, measured outside the code under test. The engine's own byte ledger is a secondary diagnostic and has disagreed with the kernel by up to 23x in both directions. So the case for landing d_slot_record_bytes, d_layer_cores, d_slot_records, d_occupied_slots and d_cross_rank_endpoints in main rested on them being useful to something downstream — and checked, the only consumers are five collators in the private benchmark harness (sparse_collate.py, graph_world_report.py, graph_layout_collate.py, nocache_collate.py, sparse_stack_collate.py). No library code, no benchmark, no shipping test outside this PR's own two suites. Diagnostics whose only readers are measurement tools belong with the measurement tools.

Where the work went: branch tool/profile-instrument, tag tool/profile-instrument-v2 — one squashed commit on main 48cadcb carrying the monoprop_PROFILE instrument and these counters, with both test suites intact. It is cherry-picked onto a branch under test, never merged. Nothing is lost: that tree is byte-identical to the pre-split branch, all at tree 321e8701a8995fe20b27d6f12be5aa2ebeb99329.

If a PR later needs these fields to make its own case, it should carry them. One that already would: perf/sparse-world-slots (#238) added assertions on all five to tests/test_monoprop_smoke.py, since its whole claim is that per-rank graph memory falls as the world grows. Those counters are the natural way to show it, so they should ship with that argument rather than ahead of it.

Two review threads this closes out:

• @robertodr's fprintf point — already answered above: that code was never going to land here, and on the tool branch it is one noexcept helper over std::format with a single fwrite per line.
• The A/B figures quoted earlier on this PR measured a superset of the final diff and should not be read as anyone's result.

For the record, the diff was green throughout: ctest -L unit 220/220, -L serial 219/219, -L mpi and the Python MPI suite at 1×1, 1×16, 2×8 and 8×16 (600 passed each), 22/22 CI checks, and clang-format clean.