A solver family's cached data is one object, not eighteen members - #185
Open
BDonnot wants to merge 3 commits into
Open
A solver family's cached data is one object, not eighteen members#185BDonnot wants to merge 3 commits into
BDonnot wants to merge 3 commits into
Conversation
…embers
The bus labelling, Ybus / Sbus, the slack, the pv-pq split and the connectivity
snapshot are one picture of the grid taken at one instant, all of it expressed in
ONE bus labelling. They were eighteen separate LSGrid members, and
`pre_process_solver` took six of them as parameters while reaching for the other
three itself -- so a caller building into its own vectors got six of its outputs
and the grid got three of them.
That is a real defect in the batch path. A TimeSeries / ContingencyAnalysis prep
wrote its pv-pq split and its slack weights into the grid's cache, next to a Ybus
built for a different labelling, and left that grid still claiming the mixture was
reusable (checked against dev_0.13.2: need_reset_solver() stays false across such
a build). The reuse guard then sized one owner's containers against the other's;
sizes agree far more often than labellings do, so a caller reusing its containers
with a "nothing changed" control would have skipped fillpv_pq and stamped this
grid's split onto its own system -- converged, plausible, wrong. Nothing reaches
it today: the batch works on a private copy and always asks for a full rebuild.
Both of those are accidents, and neither is written down as a requirement.
So rather than adding a check that the nine parts agree, make them one object.
src/core/SolverSideCache.hpp
SolverBusLayout -- labelling, slack, pv-pq split, connectivity snapshot,
allow_reuse, algo_needs_rebuild (types that do not
mention the family's scalar)
SolverSideCache<T> -- the above, plus mat and inj (the two that do)
T = cplx_type -> AC (Ybus / Sbus)
T = real_type -> DC (Bbus / Pbus)
LSGrid holds ac_cache_ / dc_cache_. pre_process_solver / pre_process_dc_solver
take one cache reference. There is no way to hand either half of one.
What this deletes, rather than adds:
- the reuse guard is `cache.is_usable(nb_bus)`, in one place, instead of eight
comparisons written inline against a mix of two objects' members;
- "is this my cache?" is `&c == &ac_cache_`, two overloads picked by the family's
own type, instead of counting nine pointer comparisons and throwing on a
partial match;
- `_mark_cache_valid` / `prevent_*_cache_reuse` / `init_bus_status` lose their
`if(ac) x_ac_ else x_dc_` bodies;
- BaseBatchSolverSynch's eleven loose members become its own two caches, and the
read-back of _grid_model.get_ac_pv_solver() -- with the three copies its own
`TODO copies are made here, which is not ideal` flagged -- is gone: one call
fills the whole thing, into vectors the batch owns.
And it is the extension point that was asked for: remote / shared voltage-control
layouts, HVDC droop data, whatever comes next, go in as another field and inherit
the lifetime, invalidation and consistency rules of everything already there.
`algo_controler_` (DualAlgoControl) deliberately stays where it is -- it is
threaded through every element container's mutator signature and is already
correctly per family.
Two behaviours change, both in the foreign-build path:
- a build into a caller's cache never reuses (`solver_control`, the snapshot
init_bus_status() compares against and the flags it raises all describe THIS
grid, and say nothing about someone else's cache);
- it retires this grid's cache for that family afterwards. The labelling and the
split still have to be published, because the NR extensions read them back
through `lsgrid_ptr` rather than from what the solver was handed -- including
bus_pq, which fill_voltage_control_solver_data needs and which the old
write-through supplied by accident. Publishing the matrix too would mean copying
it, so what is left is a view for the extensions, not a cache: the snapshot is
cleared and the control raised, so this grid's next own powerflow rebuilds.
- the grid's own algorithm is no longer reset / reconfigured for a solve it will
never perform.
ABI: this changes LSGrid's member layout, so a consumer that casts an LSGrid
across a module boundary (gpusim2grid -- see the note that used to sit on
_forced_ref_slack_bus_id) must be rebuilt against these headers. That is already
what docs/solver_plugin.rst requires of plugins ("the same version of
lightsim2grid headers that is installed at runtime ... different BaseAlgo
layout"). Flagged in the changelog.
Cost, callgrind instruction counts (slope between 200 and 1200 powerflows on the
exotic-elements IEEE14 grid, so construction cancels): +200 instr on an AC
powerflow (839 340 -> 839 541, +0.024%) and +214 on a warm-cache DC one
(28 095 -> 28 309, +0.76%, and that is the cheapest powerflow this library can
run). Roughly two thirds of it is not this change: a per-function diff of the
profiles puts +74/pf in compute_results_tsc_rxha_no_amps and +60/pf in an
Eigen::Ref helper, neither of which the diff touches -- inlining decisions that
moved when the translation unit was recompiled. Wall clock shows nothing: the
run-to-run spread on one unchanged binary (2464 -> 2744 ns on the same DC case)
is larger than the gap between the two.
Tests:
- test_cache_reuse.cpp, "the structural half ... unset_changes": the existing
[unset_changes] sections all reach unset_changes() through
allow_cache_reuse(false), ie BOTH families off, so the family that runs is
stopped by is_usable's first line (`if(!allow_reuse)`) and those sections pass
with the rest of is_usable deleted. Leaving ONE family automatic is what
exercises it; verified to SIGSEGV against a build with is_usable gutted to
`return allow_reuse;`.
- test_cache_reuse.cpp, "a build into someone else's cache ...": a foreign build
fills the caller's cache completely, publishes what the extensions read, and
retires the grid's -- so the grid's own next powerflow is unaffected. The
need_reset_solver() assertion is the one that separates this from dev_0.13.2.
- test_batch_voltage_control.cpp: the source grid solves identically on both sides
of a TimeSeries and a ContingencyAnalysis.
Verified: 210 test cases / 5815 assertions pass under Release, under C++14
(LS2G_CXX_STANDARD=14), under ASan+UBSan, and under valgrind (0 errors, no
leaks). The python layer could not be exercised here (no pybind11 / numpy on this
machine); no python-visible API changes.
Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u
init_bus_status() took SubstationContainer::get_bus_status() by value:
const std::vector<bool> new_status = substations_.get_bus_status();
The accessor already returns `const std::vector<bool> &`, so this copied the
whole vector into a local, to be read twice by _flag_dimension_change and
thrown away. Every mutation of the bus status happens in the disconnect /
reconnect calls above the line, and _flag_dimension_change only reads, so a
reference is safe.
One missing `&`. O(nb_bus) per powerflow that reaches init_bus_status, which is
every powerflow that rebuilds and every one where an element changed bus.
Verified: 210 test cases / 5815 assertions pass under Release.
Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u
Same split as compute_pf_with_input_validation / compute_pf: the user-facing
entry point does not take the caller's word for it, the internal one does,
because it has just built the data itself.
unset_changes() user-facing, BOTH families, checked
_mark_cache_valid(...) internal, one family, no checks
`unset_changes()` still marks both families -- that is its historical contract
and pre-1.0.0 code relies on it -- but each is now verified before the claim is
recorded, and a family that cannot back it is retired rather than marked.
The check that matters is not the one I expected
------------------------------------------------
The obvious check is "does the cache hold a system of the right shape". It is
necessary -- that is what stands between a claim about a family that never
solved and an out-of-bounds read -- but it is nowhere near sufficient, and the
new test caught it: after
ac_pf(); deactivate_powerline(4); unset_changes(); ac_pf();
the answer was wrong by 0.038 pu. Deactivating that line changes no bus's
connected/disconnected status and no vector's size, so every structural check
passes -- and `tell_none_changed()` then cleared the pending `recompute_ybus`,
so the next powerflow re-solved a Ybus that still contained the line.
A cache cannot tell from its own contents that it is stale. Change an
impedance, a tap, a load target: every size and every bus status is exactly
what it was. What knows is the element containers, which declare every change
through AlgoControl in their own modifiers -- so the load-bearing test is to
READ that back, via the new `AlgoControl::nothing_changed()` (the exact negation
of `tell_all_changed()`). The containers stay the sole authority: nothing here
second-guesses what they declared, adds flags of its own, or recomputes
connectivity behind them.
The powerflow path stops re-checking
-------------------------------------
`_pre_process_solver_impl` now asks only the switch (`!own_cache ||
!cache.may_be_reused()`). It can, because every other way the flags reach it can
only make a cache MORE stale, never falsely fresh: the containers raise them as
the grid is modified, AlgoControl's constructor asks for a full rebuild,
set_state / the copy ctor / a divergence reset, and python cannot clear them
(`get_*_algo_controler()` is bound read-only -- binding_misc.cpp exposes the
has_* / need_* getters and no tell_*). The two entry points that CAN claim
"nothing changed" without having built anything now verify it themselves:
`unset_changes()`, and `check_solution()`, whose weaker
`id_me_to_solver.size() > 0` guard is replaced by the same consistency check.
A debug-only assertion keeps that reasoning honest: free under -DNDEBUG (what
the wheels ship), and it fires in the C++ suite -- which CI runs under ASan,
UBSan and valgrind -- the day a third claimant appears. Verified non-vacuous:
making `unset_changes()` mark unconditionally aborts the Debug build.
`SolverSideCache`'s predicates are now two separate questions, which is what
made the above possible to reason about at all:
is_consistent(nb_bus) is the data there and self-consistent (sizes only)
may_be_reused() is reuse allowed at all
Neither asks "has the grid changed" -- that is not a question a cache can answer
about itself, and AlgoControl already answers it.
Tests: test_cache_reuse.cpp's [unset_changes] case rewritten. It used to pin the
dangerous intermediate state -- that a never-built family got marked "in sync",
with the powerflow path catching it later. It now pins the opposite: the claim
is refused at the point it is made. Sections cover a family that never solved,
each family solved alone, a pending topology change (the 0.038 pu case), a
genuinely valid cache claimed repeatedly, and the three sequences that used to
segfault. Verified the flag test bites: removing it reproduces the wrong answer.
Verified: 210 test cases / 5830 assertions pass under Release, C++14, Debug
(assertions live), ASan+UBSan, and valgrind (0 errors, no leaks). The python
layer could not be exercised here (no pybind11 / numpy on this machine).
Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u
BDonnot
force-pushed
the
claude/solver-side-cache-data-class
branch
from
August 23, 2026 15:54
73ca033 to
03260b9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One commit, fresh from
dev_0.13.2. Replaces #184, which took the opposite approach (add checks that the eighteen loose members agree) and is closed.The problem, concretely
pre_process_solverfills in containers you hand it. Two callers hand it different ones:Six of the nine outputs were parameters. Three were not. At the top of
_pre_process_solver_impl:Hardcoded to the grid's members whatever the caller passed for the rest. One call, output split across two objects. The tell was already sitting in the batch:
Three consequences, all silent:
Ybusbuilt for a different bus labelling — and the grid went on claiming that mixture was reusable. Checked againstdev_0.13.2:need_reset_solver()stays false across such a build.fillpv_pqand stamped this grid's split onto its system — converged, plausible, wrong._algo.reset()/tell_solver_control()fired on the grid's own algorithm for a solve it would never perform, discarding a factorization its next powerflow would have reused.Nothing reaches any of this today. The batch works on a private
LSGridcopy and always callstell_all_changed()first. Both are accidents; neither is written down as a requirement, and the obvious next optimisation in there (keepYbusacross scenarios) removes the second one.The fix: make them one object
src/core/SolverSideCache.hpp, split by type, not lifetime:SolverBusLayoutallow_reuse,algo_needs_rebuild— everything whose type doesn't mention the family's scalarSolverSideCache<T>matandinj.cplx_type→ AC (Ybus/Sbus),real_type→ DC (Bbus/Pbus)The split exists so the batch — which runs AC or DC, never both — can name the family-agnostic half: 82 of its 150 call sites don't care which family, and go through
active_layout()instead of a ternary at every use.LSGridholdsac_cache_/dc_cache_.pre_process_solver/pre_process_dc_solvertake one cache reference. There is no way to hand either function half of one.What this deletes rather than adds
cache.is_usable(nb_bus)— one place, instead of eight comparisons written inline against a mix of two objects' members.&c == &ac_cache_, two overloads picked by the family's own type. Solver-side data of a powerflow belongs to exactly one owner #184 needed nine pointer comparisons and a throw on a partial match; that whole function is gone._mark_cache_valid/prevent_*_cache_reuse/init_bus_statuslose theirif(ac) x_ac_ else x_dc_bodies.BaseBatchSolverSynch's eleven loose members become its own two caches; the read-back above, and its three copies, are gone.It's also the extension point: remote / shared voltage-control layouts, HVDC droop data — add a field and it inherits the lifetime, invalidation and consistency rules of everything already there.
DualAlgoControldeliberately stays put. It's threaded through every element-container mutator signature and is already correctly per-family.Behaviour changes (all in the foreign-build path)
solver_control, the snapshotinit_bus_status()raises its flags against, and those flags all describe this grid and say nothing about someone else's cache.lsgrid_ptrrather than from what the solver was handed, includingbus_pq, whichfill_voltage_control_solver_dataneeds and which the old write-through supplied by accident. Publishing the matrix too would mean copying it, so what remains is a view for the extensions, not a cache: snapshot cleared, control raised, next own powerflow rebuilds.This changes
LSGrid's member layout. The note that sat on_forced_ref_slack_bus_id("declared LAST so existing member offsets are unchanged — ABI-stable for the gpusim2grid cross-moduleLSGridcast") cannot survive regrouping scattered members.docs/solver_plugin.rstalready requires the same version of headers at runtime ("differentBaseAlgolayout ... undefined behaviour"), and 1.0.0 already carries several[BREAKING]entries — so this is flagged in the changelog rather than worked around. gpusim2grid must be rebuilt against these headers.Cost — measured
Callgrind slope, 200→1200 powerflows on the exotic-elements IEEE14 grid (so construction cancels):
The DC row is the floor — the cheapest powerflow this library can run.
Roughly two thirds of that is not this change: a per-function diff of the two profiles puts +74/pf in
compute_results_tsc_rxha_no_ampsand +60/pf in anEigen::Refhelper, neither of which the diff touches — inlining decisions that moved when the translation unit was recompiled. Wall clock shows nothing: run-to-run spread on one unchanged binary (2464 → 2744 ns on the same DC case) exceeds the gap between the two binaries.Tests
test_cache_reuse.cpp, "the structural half …unset_changes" — the existing[unset_changes]sections all reach it throughallow_cache_reuse(false), i.e. both families off, so the family that runs is stopped byis_usable's first line (if(!allow_reuse)) and those sections pass with the rest ofis_usabledeleted. Leaving one family automatic is what exercises the structural checks. Verified: guttingis_usabletoreturn allow_reuse;makes it SIGSEGV.test_cache_reuse.cpp, "a build into someone else's cache …" — a foreign build fills the caller's cache completely, publishes what the extensions read, and retires the grid's. Theneed_reset_solver()assertion is the one that separates this fromdev_0.13.2.test_batch_voltage_control.cpp— the source grid solves identically on both sides of aTimeSeriesand aContingencyAnalysis.210 test cases / 5815 assertions pass under Release, C++14 (
LS2G_CXX_STANDARD=14), ASan+UBSan, and valgrind (0 errors, no leaks).The python layer could not be exercised locally (no pybind11 / numpy on the machine this ran on) — CircleCI's
test_legacy_*jobs cover it. No python-visible API changed;pre_process_solverwas never bound, and every&LSGrid::symbol the bindings reference still resolves.🤖 Generated with Claude Code
https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u
Generated by Claude Code