[FEATURE] Static Model Cost and Performance Analysis

[LucaSforza]

Overview

Add static model cost analysis to the editor using inferred tensor types, stereotype parameters, and declarative operator-cost metadata. Users should see model size and likely hotspots before conversion or training.

The analysis should report useful deterministic estimates, not claim hardware-specific latency accuracy.

Metrics

Provide totals and per-node values for:

• total and trainable parameter count;
• parameter memory by dtype;
• MACs and FLOPs for supported operators;
• input/output activation size;
• estimated peak live-activation memory for the compiled graph schedule;
• optional optimizer-state and gradient memory estimates;
• unsupported or unresolved metrics with an explicit reason.

Batch-dependent estimates accept a user-provided batch size when the type system retains a symbolic batch dimension.

Architecture

Implement a CostEngine separate from TypeEngine. It consumes TypeResult, resolved node parameters, topology, and stereotype cost declarations, and returns structured per-node and aggregate results.

Cost formulas should be declarative and colocated with stereotypes where practical, reusing the safe arithmetic parser rather than hardcoding module names throughout TypeScript. Unknown custom modules remain supported by returning unknown, not by guessing.

Subflow totals aggregate their internal graph. Repeat multiplies sequential cost and parameter count by its number of independent copies. HorizontalRepeat accounts for parallel copies and concatenated activation shape. Shared tensors on forks must not be double-counted in peak-memory liveness analysis.

Python may provide an optional verification path using instantiated modules or meta/fake tensors, but frontend results must remain available without starting training.

UX

• Add a model analysis panel with aggregate cards and a sortable per-node table.
• Overlay a selectable heatmap on graph nodes for parameters, MACs, or activation memory.
• Show formula inputs and unresolved dimensions in a node detail view.
• Allow batch size and dtype assumptions to be changed without modifying the diagram.
• Highlight the largest contributors and warn when an estimate exceeds a selected device-memory budget.
• Keep estimates clearly labeled and distinguish MACs from FLOPs.

MCP and export

• Add a browser RPC/MCP query for full-model or selected-node cost analysis.
• Include analysis assumptions and unresolved metrics in the response.
• Optionally export a machine-readable report; generated model behavior must not depend on the report.

Acceptance criteria

• CostEngine returns structured aggregate and per-node metrics without mutating the diagram.
• Linear, Conv1d/2d, normalization, pooling, embedding, attention, joins, and current subflow operations have tested cost behavior or an explicit unsupported result.
• Parameter counts distinguish total and trainable values where statically known.
• Activation memory uses resolved shapes, dtype byte widths, and configurable batch assumptions.
• Peak-memory estimation respects graph fan-out and tensor liveness instead of simply summing all activations.
• Symbolic or invalid values propagate as unknown with a reason rather than producing misleading numbers.
• Repeat, HorizontalRepeat, nested subflows, joins, and skip connections aggregate correctly.
• The editor provides summary, per-node inspection, and heatmap views.
• Browser RPC/MCP exposes the same analysis.
• Tests compare known small networks against hand-calculated parameter, MAC, and memory values.
• Optional Python verification tests detect material disagreement for supported PyTorch modules.
• Documentation defines formulas, assumptions, MAC/FLOP convention, and limitations.

Out of scope

• Exact wall-clock latency or throughput prediction.
• Kernel-, compiler-, and hardware-specific performance modeling.
• Automatic architecture mutation or optimization.
• Profiling a full training epoch.