docs: add inlineDemo tracing example (inline showcase)#232
Open
gnidan wants to merge 15 commits into
Open
Conversation
The TCO back-edge JUMP previously emitted a gather wrapper around its invoke and return contexts. Multiple discriminator keys can coexist on a single context object without gather wrapping, so the JUMP now carries a flat combined context with both `invoke` and `return` keys directly. Updates the countCallSites helper in optimizer-contexts.test to check invoke and return independently rather than as an either/or, so flat multi-discriminator contexts get counted in both buckets. The TCO-specific assertion now finds the back-edge JUMP by the presence of both discriminators rather than by a gather wrapper.
* format: make invoke.target optional for internal calls Internal calls via JUMP normally carry a code pointer to the callee's entry point. When the compiler inlines a function, the JUMP is elided — there is no physical call instruction and no code target to point at. The callee identity (identifier, declaration, type) remains meaningful, but the target pointer does not. Same pattern as #211 (making return.data optional). Unblocks inlining: bugc can emit invoke contexts on inlined first instructions without fabricating a target pointer. - Schema: drop target from InternalCall.required, expand description, add worked example for inlined case - TS types: mark target optional; guard relaxed - Spec page: document optionality and point at transform + gather for inlining annotation - bugc: guard target access in patchInvokeInContext; tests assert target defined before dereferencing * format: prefer flat form for invoke + transform composition Pair with #212's flat-form guidance: when an inlined body's first instruction carries both an invoke and a transform, those belong as sibling keys on a single context — gather isn't needed because `invoke` and `transform` don't collide.
* format: add transform context for compiler optimizations
Adds a new context type annotating instructions with the
compiler transformations that produced them. The value is an
array of short identifiers; the list may repeat the same
identifier when the transformation has been applied multiple
times (e.g., ["inline", "inline"] for doubly-inlined code).
Transform is *additional* annotation. The invoke/return contexts
for the logical call are still emitted at the call boundary so
debuggers see the source-level call stack; the transform context
tells debuggers how the call was physically realized. Consumers
that ignore transform contexts get a sound source-level view
from the semantic contexts alone.
v1 identifiers:
- "inline": marked instruction is part of an inlined function
body; surrounding invoke/return contexts name the inlined
callee.
- "tailcall": marked instruction is a tail-call-optimized
back-edge JUMP or continuation, where the call was realized
without pushing/popping a full activation.
The identifier set is extensible. Debuggers unfamiliar with a
given identifier should preserve it as an opaque label. Order
in the array is not semantically significant — the multiset is
what matters.
Unblocks the final shape of TCO back-edge annotations in
bugc (#210): a tail-call-optimized JUMP can now carry
`gather: [return, invoke, transform: ["tailcall"]]`.
Includes:
- schemas/program/context/transform.schema.yaml
- schemas/program/context.schema.yaml: wire into the if/$ref
union.
- packages/format/src/types/program/context.ts: Context.Transform
interface, isTransform guard, and Transform.Identifier union
preserving autocomplete for known values.
- packages/format/src/types/program/context.test.ts: register
Context.isTransform with the schema guard test harness.
- packages/web/spec/program/context/transform.mdx: spec page
covering role, v1 identifiers, repetition/composition, and
interaction with gather.
* format: expand transform v1 vocabulary with fold and coalesce
Adds two more identifiers to the v1 transform context
vocabulary, based on bugc optimizer's audit of transformations
the compiler currently performs or will perform:
- "fold" — compile-time constant folding. The marked
instruction carries the result (typically a PUSH) replacing
a compute sequence that appeared in source.
- "coalesce" — read-write merging. The marked instruction is
part of a SHL/OR sequence (or similar) introduced by the
compiler to combine adjacent source-level reads or writes,
such as packing narrower fields into a single storage slot.
Together with the previously-defined "inline" and "tailcall",
this covers the four transformations bugc emits today or will
emit in the near term (inline once a function inlining pass
lands). Propagate was considered for v1 and deferred as
borderline.
Updates:
- transform.schema.yaml: description enumerates the four v1
identifiers; examples include single-identifier cases for
each plus combinations ["inline", "fold"], ["coalesce",
"coalesce"].
- context.ts: Transform.Identifier union extended with "fold"
and "coalesce" (still keeps `string & {}` for extensibility
and autocomplete).
- transform.mdx: subsection for each identifier with a concrete
EVM-level example, updated repetition/composition section
with new combinations.
* format: prefer flat context composition, document gather scope
The context schema's discriminator keys combine via allOf of
if/then rules, so a single context object can carry multiple
keys at once (e.g., `invoke`, `return`, and `transform` all
side by side). Use gather only when two contexts would collide
on the same key.
- transform spec: switch the TCO back-edge example from gather
to the flat form; revise the tailcall bullet accordingly
- transform schema: note in the description that flat
composition is preferred; gather is for key collisions
- gather spec: add a "When to use" section flagging the flat
form as the default and listing the canonical collision
cases (multiple frames, multiple variables blocks)
The TCO back-edge JUMP already carries a flat context with both invoke (the new iteration's call) and return (the previous iteration's return). Add a third sibling key, transform: ["tailcall"], marking the instruction as a tail-call-optimized back-edge. This is an additive annotation: it does not replace the invoke/return pair (which state the source-level facts) but tells debuggers the pair was realized as a TCO back-edge rather than a real frame push/pop, so they can avoid inventing a spurious frame. Consumers that ignore transform contexts still get a sound source-level view from invoke/return alone. Widens the emitted context type to Return & Invoke & Transform and extends the optimizer-contexts test to assert the back-edge JUMP carries transform containing "tailcall".
Add tailcall (transform context) support to the trace widgets: - extractTransformFromInstruction: gather/pick-aware collector for compiler transform identifiers (duck-typed until #212's guard lands) - extractCallInfoFromInstruction: attach isTailCall when a tailcall transform is present alongside the invoke/return - buildCallStack: a TCO back-edge carries both return and invoke on one instruction; replace the top frame in place (reuse) instead of popping to empty, and mark it isTailCall. Fixes a real call-stack correctness bug for tail-recursive loops. - CallStackDisplay: tail-call chip on the reused frame - CallInfoPanel: tail-call banner variant - Propagate isTailCall through ResolvedCallFrame / ResolvedCallInfo - CSS (+ web theme copies) for the transform/tailcall styling Tested: 9 new unit tests in mockTrace.test.ts covering extraction, the isTailCall flag, and frame replacement. Does not touch the docs TraceDrawer opt level or examples (held for product decisions).
Add two self-tail-recursive BUG programs (accumulator sum and factorial) and a Tail-call optimization section to the tracing page. Both programs fold under bugc's level-2 optimizer (verified: the recursive call terminator is eliminated and replaced with a loop trampoline), so they exercise the new tailcall transform context in the tracer widget. The section explains how a TCO back-edge JUMP composes return, invoke, and transform: ["tailcall"] as sibling keys on one context (the flat form), and how a debugger can reconcile that with the source-level call stack. Also register program/context/transform in the web schemaIndex; it was missing (gather was present), which broke the docs build for #212's transform spec page.
In transform.mdx and gather.mdx, the first reference to frame contexts now links to /spec/program/context/frame, matching the existing [`gather`](...) link precedent. Frame is the one composition concept a reader reaching these pages may not have met yet.
cloneFunction dropped the optional Ir.Function.loc and sourceId
fields, returning only { name, parameters, entry, blocks }. Since
the first optimization pass clones the module, every function lost
its declaration source info from optimization level 1 upward.
evmgen gates declaration emission on func.loc && func.sourceId, so
all invoke/return contexts lost their declaration source ranges at
optimized levels — measurably 3/3 declared at level 0, 0/3 at
levels 1-3 on the widget's runtimeInstructions path.
Copy loc/sourceId in cloneFunction's return so declarations survive
optimization. Adds a regression test asserting invoke/return
contexts still carry declaration at levels 1, 2, and 3 (with a
level-0 baseline).
…n panels (#222) * web: fix tracer-drawer opcodes/state panels not filling height The trace panels live in a flex:1 grid whose implicit row was auto- sized to content, so dragging the drawer taller left dead space below the panels instead of growing them. Give the grid an explicit 1fr row and min-height:0 (on the grid and its items) so both panels absorb the added vertical space and scroll internally. * web: add optimizer-level selector (O0/O2) to tracer drawer The drawer hardcoded optimizer level 0. Add an O0/O2 toggle in the drawer header that recompiles + retraces at the chosen level, so readers can flip to level 2 and watch optimizer transforms (e.g. the tailcall annotation on TCO back-edges) appear. compileAndTrace now takes the level explicitly; a ref mirrors the state so the example- load effect reads the current level without re-running on toggle. * programs-react: cover the flat (production) TCO back-edge shape bugc #217 emits the TCO back-edge as a single flat context object (return + invoke + transform keys together), but the existing tests only exercised the gather shape. Add flat-shape variants for transform extraction, the isTailCall flag, and frame replacement, plus a guard that stripping the marker (the #10 failure mode) drops tail-call handling. * web: dedupe call stack + tailcall render + right-column panels - Reuse the shared, tailcall-aware buildCallStack / extractCallInfo / extractTransform from @ethdebug/programs-react via a thin adapter (bugc .debug.context -> ethdebug format shape); drop the drawer's inline call-stack builder and local extractCallInfo. - Render the tail-call chip on the reused call-stack frame and a tail-call variant on the call-info banner. - Right column (gnidan's picks 1/2/4): resolved variable values (name: value via pointer resolution), gas remaining + per-step delta, and a transform annotations panel with per-tag glosses. Sections are now collapsible. * web: widen optimizer selector to O0/O1/O2/O3 gnidan's call: expose all four bugc optimizer levels (each distinct — L1 fold/prop/DCE, L2 +CSE/TCO/jump-opt, L3 +merging) rather than a two-state O0/O2 toggle, future-proofing for other transforms. The recompile+retrace already took the level; just widen the control from two buttons to four (mapped over OPT_LEVELS, per-level tooltips). The tailcall demo still lands on O2.
Update the tail-call-optimization walkthrough to reference the tracer drawer's Opt optimization-level selector (O0-O3), telling readers to compare O0 with O2 (TCO kicks in at level 2), instead of the generic "optimizer control" wording.
* bugc: emit fold transform on constant folding Marks constant-folded values with transform:["fold"] so debuggers can show that a value is a compile-time-evaluated constant rather than source the user wrote. Adds Ir.Utils.addTransform(debug, ...ids), which composes transform markers as a flat sibling key on a debug context (per the flat- composition convention) and appends to any existing transform array — so an instruction touched by multiple passes accumulates the multiset (e.g. ["fold","coalesce"]). ConstantFoldingStep applies it to folded binary and hash results. Although a folded const instruction is typically dissolved by constant propagation + DCE, ConstantPropagationStep already carries the folded const's operationDebug into the consuming instruction, so the marker survives to the emitted bytecode's runtimeInstructions (the tracer widget's path) at levels 1-3. Adds an end-to-end test asserting the fold marker reaches the bytecode at levels 1/2/3 and is absent at level 0. * bugc: route tailcall transform through addTransform helper Refactor buildTailCallJumpOptions to emit its transform:["tailcall"] marker via Ir.Utils.addTransform instead of a hand-written key, so all transform emission (fold/tailcall/coalesce/...) routes through the one helper. Behavior is identical — the back-edge JUMP still carries return + invoke + transform:["tailcall"] — but composition stays consistent (e.g. a folded-then-tailcall instruction would accumulate the multiset) and there's no divergent hand-rolled site. Existing #217 tailcall tests pass unchanged.
Marks the SHL/OR field-packing sequence produced by ReadWriteMerging (level 3) with transform:["coalesce"], so debuggers can show that a packed-storage write is compiler-synthesized rather than source the user wrote. Every instruction the merge produces (shifts, ORs, and the merged write) routes its debug through Ir.Utils.addTransform, which appends coalesce to any existing transform array — so a folded value packed into a word composes as ["fold","coalesce"]. Per #212, coalesce = read-write (SHL/OR) packing specifically; the CFG-merging passes (block-merging, return-merging) are left unmarked for v1. Adds an end-to-end test: coalesce marker present at level 3, absent at levels 0-2.
* format: spec the inlined-call virtual activation contract Document how an inlined internal call is represented and how a debugger reconstructs its virtual activation, per the inlining design (gnidan rulings: reuse jump:true, omit arg/return value pointers in v1, adopt DWARF "activation" naming). - invoke.mdx: add "Inlined internal calls" (jump:true = call kind not a literal JUMP; target omitted; transform:["inline"]) and a "Reconstructing activations" section defining the real-vs-virtual activation distinction, annotation-driven per-instruction membership (robust to non-contiguous inlined bodies), graceful identity degradation, and v1 value resolution (structure + locals; no arguments/data pointers). - transform.mdx: enrich the inline id (virtual activation, cross- link) and add an inlined-call-site worked example noting that a helper inlined at N sites yields N virtual activations. No schema change. Scope-agnostic re: the single- vs multi-site heuristic (bugc pass scope, decision #3, still open). * format: describe inlining eligibility qualitatively (decision #3) Reflect gnidan's #3 heuristic (leaf or small, non-recursive, internal, at all call sites) as compiler-typical, keeping the threshold qualitative — the format is invariant to the eligibility rule. * format: sharpen activation discriminator + membership wording Apply debugger review precision notes on #229: - Real-vs-virtual: the reliable discriminator is the inline transform marker, not target-absence — a real internal call may also omit target (#213), so no-target does NOT imply virtual. Reword the real/virtual clauses accordingly. - Membership: reword "iff it carries transform:["inline"]" (read as exact array equality) to "carries an inline identifier in its transform list" so composed markers like ["inline","fold"] still confer membership. * format: frame virtual-activation structure as compiler-typical Apply debugger note (a): "reliably carries ... identity" contradicted the graceful-degradation paragraph (identity is optional). Reword to "an inlining compiler typically preserves declaration and per-instruction source ranges; identity fields remain optional and degrade gracefully" — a compiler-behavior expectation, not a format guarantee.
Adds InliningStep (level 2, first — after L1 fold, before CSE/TCO/ JumpOpt) that replaces calls to eligible internal functions with a copy of the callee body spliced into the caller. Every inlined instruction is annotated transform:["inline"] via addTransform, and the body is bracketed by a virtual invoke (jump:true, identity, no target — the #213 optional-target signal) / virtual return, so a debugger can reconstruct a virtual activation. This lights up `inline` in the tracer, completing the transform set. v1 eligibility (correctness over coverage; follow-ups noted): - internal, non-recursive, single-return, LEAF callee; - applied at all call sites; callee deleted once fully inlined; - NOT inlined into self-recursive / TCO'd callers: inlining a helper into a tail-recursive call's arguments rewrites count(succ(n)) -> count(n+1), which the tail-call optimizer mishandles (pre-existing bug, tracked separately). Guarding this keeps the tailcall demos pristine. Return values use dest-substitution (not a continuation phi), which is robust to L3 block-merging; deep clones use structuredClone to preserve bigint const values. Updates optimizer-contexts tests: leaf helpers (add/dbl/addThree) now inline at L2+ (no real caller JUMP; inline activation instead), while recursive/multi-return functions (fact, isEven/isOdd, count) are untouched. Deferred to follow-ups: non-leaf/nested callees, multi-return, size-threshold (non-leaf) inlining.
A small leaf helper square(x) called from two sites with storage-read args. At O2/O3 both sites inline into the caller -> two virtual activations in the tracer (the inline showcase, as sum/factorial were for tailcall). Args are non-constant (storage reads) so L1 fold can't collapse the inlined bodies. Verified against bugc: correct result (3^2 + 4^2 = 25) at O0-O3; zero inline marks at O0/O1, both sites inlined at O2 (108 -> 50 runtime instrs) and O3. Prose + TraceExample wiring to follow (writer/docs).
Contributor
|
gnidan
force-pushed
the
transform-context
branch
from
883d0f5 to
94a800e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the
inlineDemotracing example — a dedicated showcase for theinlinetransform, the waytailRecursiveSum/tailRecursiveFactorialshowcase
tailcall.The program
A leaf helper
square(x) -> x * xcalled from two sites:Args are storage reads (
a,b), not constants — so L1 fold can'tcollapse the inlined bodies, keeping the two virtual activations visible
in the tracer at O2/O3.
Verified against bugc (this branch)
3^2 + 4^2 = 25.runtime instructions) and O3.
target) / return contexts emitted per the format: spec the inlined-call virtual activation contract #229contract.
Scope of this PR
Data only — the
inlineDemoexport. Prose + the<TraceExample>blockin
tracing.mdxwill follow from writer/docs (coordinating placementwith the #227 Explore/trace-playground reorg).