Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions anyplotlib/FIGURE_ESM.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ Rule 5 – Text never clips. Optional gutters earn real layout space:
| `drawScaleBar2d` / `drawColorbar2d` | 1360 / 1436 |
| `_drawAxes2d` (ticks, labels, title) | 1491 |
| `drawOverlay2d` / `drawMarkers2d` | 1629 / 1685 |
| **Image layers**: `_layerBytes` / `_layerBitmap` / `_drawLayers2d` | 1553 / 1577 / 1629 |
| Binary-bytes splice: `_spliceBinaryBytes` / `_registerBinaryPixelListeners` | 675 / 706 |
| **3D drawing**: `draw3d` | 1833 |
| Event emission `_emitEvent` | 2031 |
| 3D event handlers `_attachEvents3d` | 2059 |
Expand Down Expand Up @@ -196,6 +198,82 @@ inverses of the blit geometry.

---

## Image layers (multi-image overlay)

`Plot2D.add_layer(data, cmap=, alpha=, clim=, visible=)` composites a second
(third, …) scalar image OVER the base image in the same panel, each with its own
colormap / clim / alpha. Distinct from `set_overlay_mask` (single-colour boolean
mask). `Layer.set(...)`, `Layer.set_data(frame)`, `Layer.remove()`,
`Plot2D.layers`, `Plot2D.remove_layer(layer)`. **Layers and tile mode are mutually
exclusive** (guard raises in both directions: `add_layer` on a tiled plot, and
`enable_tile` / `set_data(tile=True)` / auto-tile on a layered plot).

### State + transport (dynamic per-layer pixel keys)

The layer *metadata* lives in `st.layers` (a list of small dicts on the light view
trait):

```
st.layers = [{ id, cmap, clim_min, clim_max, alpha, visible,
width, height, colormap_data, image_b64 }, …] # z-order
```

`image_b64` in each entry is the layer's pixels: a base64 string (Jupyter /
standalone / `save_html`) OR a `"\x00bin:<adler32>"` change-token (Electron binary
transport). The JS reads pixels from this entry field on the base64 path.

The *heavy pixel bytes* additionally ride a **DYNAMIC geometry key**
`layer_<id>_b64` — one per layer — mirroring how the base image `image_b64` rides
the geom channel. The dynamic-key mechanism:

- **`Plot2D._GEOM_KEYS` is a PROPERTY** (not a plain frozenset): it returns the
fixed base set (`image_b64`, `colormap_data`, `overlay_mask_b64`, `detail_b64`)
UNION the current `layer_<id>_b64` keys. So `Figure._push` splits every layer's
pixels off the light view trait onto `panel_<id>_geom` and dedup-caches them
exactly like the base image; a removed layer's key drops out automatically.
- **`_electron._route_change`** ships each layer key as its own PLOTBIN frame:
`_is_binary_pixel_key(k)` matches `k in _BINARY_KEYS` OR `layer_*_b64`. The
binary frame header carries `{"geom": "panel_<id>_geom"}` and `key=layer_<id>_b64`,
so the receiver builds slot `panel_<id>_geom::layer_<id>_b64` (the same
`awi_state_binary` handler as the base image — already generic on `hdr.geom` +
`e.data.key`, no change needed there).
- **`resolve_pixel_tokens`** (cold path: `save_html` / standalone) materialises
real base64 for every `layer_<id>_b64` key AND the entry `image_b64` mirror, so a
snapshot is self-contained.
- **JS `_spliceBinaryBytes`** scans `__apl_pixbytes` by the `panel_<id>_geom::`
PREFIX (not the old hardcoded 3-key list) so it splices any `layer_<id>_b64_bytes`
into `p2._geomCache`. The per-slot binary listeners are registered only for the
fixed keys (`_registerBinaryPixelListeners`); dynamic layer bytes are consumed by
the **geom-JSON change handler**, which now also calls `_spliceBinaryBytes` — the
geom trait always re-pushes when layers change, so a layer's bytes converge into
the cache regardless of trait arrival order.

### JS compositing (`_drawLayers2d`, called from `draw2d`)

After the base image (Canvas2D blit OR WebGPU) and the overlay mask, and BEFORE
`_drawAxes2d` / markers / widgets, `_drawLayers2d(p, st, imgW, imgH, ctx, iw, ih)`
draws each **visible** layer bottom-up on `plotCanvas`:

- `_layerBytes(st, layer)` prefers `layer_<id>_b64_bytes` (binary) over the entry
`image_b64` base64;
- `_layerBitmap(p, st, layer)` builds a LUT-colormapped RGBA `OffscreenCanvas`,
**cached per layer id** by `(pixel key, cmap, clim)` — rebuilt only when the
layer's data or appearance changes (a live scrub that only swaps one layer's
data rebuilds just that layer);
- it blits with the SAME fit-rect + zoom/pan transform as the base blit
(`_imgFitRect` + the `zoom>=1` window math) at `ctx.globalAlpha = layer.alpha`,
so zoom/pan track the base exactly.

Because layers draw on `plotCanvas`, they sit UNDER `markersCanvas` /
`overlayCanvas` (z-order) and are captured by `exportPNG` for free (plotCanvas is
z1 in the composite). Over a WebGPU base the layers still composite in Canvas2D on
`plotCanvas` (which sits above the transparent `gpuCanvas`) — verified by
`test_layers_playwright.py::TestGpuBaseWithLayer`. Per-move perf: only the changed
layer's LUT bitmap is rebuilt (the box-loop is ~one pass over H×W uint8 → uint32,
comparable to the base image's `_buildLut32` blit).

---

## 3D drawing (line ~1840)
Orthographic projection; geometry b64-decoded and cached. `draw3d` sorts
triangles, draws axes with per-axis `_drawTex` labels (`x/y/z_label_size`).
Expand Down
17 changes: 15 additions & 2 deletions anyplotlib/_electron.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,20 @@
# transport on its _encode_pixels "\x00bin:<checksum>" token stays in the geom JSON
# and the real bytes are never shipped, so the renderer can't decode it and the crisp
# zoom tile never displays (you only ever see the downsampled overview base).
#
# Image LAYERS add DYNAMIC pixel keys ``layer_<id>_b64`` (one per layer) that also
# ride the geom channel and must ship as binary — they can't be enumerated in a
# fixed frozenset, so ``_is_binary_pixel_key`` matches them by name pattern.
_BINARY_KEYS = frozenset({"image_b64", "overlay_mask_b64", "detail_b64"})


def _is_binary_pixel_key(k: str) -> bool:
"""True for a geom key whose value is raw image pixels worth shipping as a
PLOTBIN binary frame — the fixed base-image/mask/detail keys OR a dynamic
per-layer key ``layer_<id>_b64``."""
return k in _BINARY_KEYS or (k.startswith("layer_") and k.endswith("_b64"))


def _route_change(fig_id: str, name: str, value) -> None:
"""Forward ONE trait change to the host — as a raw PLOTBIN binary frame for a
large image pixel trait (when binary transport is enabled), else as a
Expand All @@ -61,13 +72,15 @@ def _route_change(fig_id: str, name: str, value) -> None:
geom = json.loads(value)
except Exception:
geom = None
if isinstance(geom, dict) and any(k in geom for k in _BINARY_KEYS):
if isinstance(geom, dict) and any(
_is_binary_pixel_key(k) for k in geom):
panel_id = name[len("panel_"):-len("_geom")]
fig = _figures.get(fig_id)
raw_tbl = getattr(fig, "_raw_pixels", None)
sent_binary = False
for k in list(geom.keys()):
if k not in _BINARY_KEYS or not isinstance(geom[k], str) or not geom[k]:
if (not _is_binary_pixel_key(k)
or not isinstance(geom[k], str) or not geom[k]):
continue
raw = None
if geom[k].startswith("\x00bin:") and raw_tbl is not None:
Expand Down
179 changes: 159 additions & 20 deletions anyplotlib/figure_esm.js
Original file line number Diff line number Diff line change
Expand Up @@ -676,8 +676,15 @@ function render({ model, el }) {
const tbl = globalThis.__apl_pixbytes;
if (!tbl) return false;
let spliced = false;
for (const pixelKey of ['image_b64', 'overlay_mask_b64', 'detail_b64']) {
const b = tbl[`${geomTrait}::${pixelKey}`];
// Fixed base-image / mask / detail keys PLUS every DYNAMIC layer pixel key
// (`layer_<id>_b64`) currently present in the side-table for this panel's
// geom trait — layer keys aren't enumerable ahead of time, so scan by the
// `<geomTrait>::` prefix. Splices each into the cache as `<pixelKey>_bytes`.
const prefix = `${geomTrait}::`;
for (const slot in tbl) {
if (!slot.startsWith(prefix)) continue;
const pixelKey = slot.slice(prefix.length);
const b = tbl[slot];
if (!b) continue;
if (b.__aplSeq === undefined) {
try { b.__aplSeq = ++_pixArrivalSeq; } catch (_) {}
Expand All @@ -689,6 +696,25 @@ function render({ model, el }) {
return spliced;
}

// Register the per-pixel-key binary-bytes change listeners for a panel's geom
// trait. The FIXED base-image / mask / detail keys each get a dedicated
// listener (their slot fires `change:<slot>` when new bytes arrive). DYNAMIC
// layer keys (layer_<id>_b64) are NOT enumerable here — their bytes are picked
// up by the geom-JSON re-splice (the geom trait always re-pushes when layers
// change) and by _spliceBinaryBytes's prefix scan, so they need no per-slot
// listener. Shared by _createPanelDOM and _createInsetDOM.
function _registerBinaryPixelListeners(id, geomTrait) {
for (const pixelKey of ['image_b64', 'overlay_mask_b64', 'detail_b64']) {
const slot = `${geomTrait}::${pixelKey}`;
model.on(`change:${slot}`, () => {
const p2 = panels.get(id);
if (!p2) return;
_spliceBinaryBytes(p2, geomTrait);
if (p2.state) { _applyGeom(p2, p2.state); _redrawPanel(p2); }
});
}
}

// ── layout application ───────────────────────────────────────────────────
function applyLayout() {
if (_suppressLayoutUpdate) return;
Expand Down Expand Up @@ -963,22 +989,19 @@ function render({ model, el }) {
const rev = (p2.state && p2.state._geom_rev !== undefined)
? p2.state._geom_rev : ((p2._geomRev || 0) + 1);
_loadGeom(p2, model.get(_geomTrait), rev);
// Re-splice any binary pixel bytes present for this panel — including
// DYNAMIC layer keys (layer_<id>_b64) that have no dedicated per-slot
// listener. The geom JSON always re-pushes when layers change, so this
// guarantees a layer's bytes are consumed regardless of arrival order.
_spliceBinaryBytes(p2, _geomTrait);
if (p2.state) { _applyGeom(p2, p2.state); _redrawPanel(p2); }
});
// Binary transport: raw pixel bytes for this panel arrive on a companion
// trait `panel_<id>_geom::<pixelKey>` (a Uint8Array). Merge them into the
// geomCache under `<pixelKey>_bytes` so _imageBytes uses them (no atob), and
// redraw. Fires INDEPENDENTLY of the geom JSON trait (which now carries only
// the small LUT/flags), so pixels + LUT converge in the cache.
for (const pixelKey of ['image_b64', 'overlay_mask_b64', 'detail_b64']) {
const slot = `${_geomTrait}::${pixelKey}`;
model.on(`change:${slot}`, () => {
const p2 = panels.get(id);
if (!p2) return;
_spliceBinaryBytes(p2, _geomTrait);
if (p2.state) { _applyGeom(p2, p2.state); _redrawPanel(p2); }
});
}
_registerBinaryPixelListeners(id, _geomTrait);
}

model.on(`change:panel_${id}_json`, () => {
Expand Down Expand Up @@ -1119,22 +1142,16 @@ function render({ model, el }) {
const rev = (p2.state && p2.state._geom_rev !== undefined)
? p2.state._geom_rev : ((p2._geomRev || 0) + 1);
_loadGeom(p2, model.get(_geomTrait), rev);
// Re-splice binary bytes incl. dynamic layer keys — see _createPanelDOM.
_spliceBinaryBytes(p2, _geomTrait);
if (p2.state) { _applyGeom(p2, p2.state); _redrawPanel(p2); }
});
// Binary transport: raw pixel bytes for this panel arrive on a companion
// trait `panel_<id>_geom::<pixelKey>` (a Uint8Array). Merge them into the
// geomCache under `<pixelKey>_bytes` so _imageBytes uses them (no atob), and
// redraw. Fires INDEPENDENTLY of the geom JSON trait (which now carries only
// the small LUT/flags), so pixels + LUT converge in the cache.
for (const pixelKey of ['image_b64', 'overlay_mask_b64', 'detail_b64']) {
const slot = `${_geomTrait}::${pixelKey}`;
model.on(`change:${slot}`, () => {
const p2 = panels.get(id);
if (!p2) return;
_spliceBinaryBytes(p2, _geomTrait);
if (p2.state) { _applyGeom(p2, p2.state); _redrawPanel(p2); }
});
}
_registerBinaryPixelListeners(id, _geomTrait);
}

model.on(`change:panel_${id}_json`, () => {
Expand Down Expand Up @@ -1528,6 +1545,120 @@ function render({ model, el }) {
return Math.max(0, Math.min(1, (now - b.t0) / _DETAIL_BLEND_MS));
}

// ── Image layers (multi-image overlay) ─────────────────────────────────────
// Raw single-channel pixel bytes for ONE layer. Prefers the BINARY bytes
// (`layer_<id>_b64_bytes`, spliced into the geomCache under `<key>_bytes`) over
// the base64 string in the layer entry's `image_b64`. Returns {bytes, key} where
// `key` is a cheap identity for the "unchanged → skip rebuild" check.
function _layerBytes(st, layer) {
const pk = `layer_${layer.id}_b64`;
const raw = st[pk + '_bytes'];
if (raw && (raw instanceof Uint8Array || raw.byteLength !== undefined)) {
const u8 = raw instanceof Uint8Array ? raw : new Uint8Array(raw);
let key = layer.image_b64; // the "\x00bin:<adler>" content token
if (!key && u8.__aplSeq !== undefined) key = `binseq:${u8.__aplSeq}`;
if (!key) key = `layerbin:${u8.length}`;
return { bytes: u8, key };
}
const b64 = layer.image_b64 || '';
if (!b64 || b64.charCodeAt(0) === 0) return { bytes: null, key: '' }; // token, no bytes yet
try {
const bin = atob(b64);
const u8 = new Uint8Array(bin.length);
for (let i = 0; i < bin.length; i++) u8[i] = bin.charCodeAt(i);
return { bytes: u8, key: b64 };
} catch (_) { return { bytes: null, key: '' }; }
}

// Build (and cache on the panel) the LUT-colormapped RGBA OffscreenCanvas for
// one layer. Cached per layer id by (pixel key, cmap, clim) so a live scrub only
// rebuilds when the layer's data/appearance actually changes. Returns null when
// the bytes aren't available or the size is wrong.
function _layerBitmap(p, st, layer) {
const d = _layerBytes(st, layer);
const lw = layer.width || st.image_width;
const lh = layer.height || st.image_height;
if (!d.bytes || !lw || !lh || d.bytes.length < lw * lh) return null;
const cmap = layer.cmap || 'gray';
const dMin = layer.clim_min, dMax = layer.clim_max;
const lutKey = [cmap, dMin, dMax].join('|');
const cache = (p._layerBlit ||= {});
const c = cache[layer.id];
if (c && c.key === d.key && c.lutKey === lutKey && c.w === lw && c.h === lh)
return c.bitmap;
// Build a 256-entry uint32 LUT from the layer's own colormap + clim. The layer
// bytes are quantised over [clim_min, clim_max] (Python side), so code i maps
// linearly through that window; the LUT bakes clim→colour like the base path.
const cmapData = layer.colormap_data || st.colormap_data || [];
let cmapFlat = null;
if (cmapData.length === 256) {
cmapFlat = new Uint8Array(256 * 4);
for (let i = 0; i < 256; i++) {
cmapFlat[i*4] = cmapData[i][0]; cmapFlat[i*4+1] = cmapData[i][1];
cmapFlat[i*4+2] = cmapData[i][2]; cmapFlat[i*4+3] = 255;
}
}
const lut = new Uint32Array(256);
const buf = new ArrayBuffer(4); const dv = new DataView(buf);
const u32 = new Uint32Array(buf);
for (let raw = 0; raw < 256; raw++) {
// Bytes are already the normalised index over the layer's clim (Python
// _normalize_image), so idx == raw — identity into the colormap.
const idx = raw;
if (cmapFlat) {
dv.setUint8(0, cmapFlat[idx*4]); dv.setUint8(1, cmapFlat[idx*4+1]);
dv.setUint8(2, cmapFlat[idx*4+2]); dv.setUint8(3, 255);
} else { dv.setUint8(0, idx); dv.setUint8(1, idx); dv.setUint8(2, idx); dv.setUint8(3, 255); }
lut[raw] = u32[0];
}
const imgData = new ImageData(lw, lh);
const out32 = new Uint32Array(imgData.data.buffer);
for (let i = 0; i < lw * lh; i++) out32[i] = lut[d.bytes[i]];
const oc = new OffscreenCanvas(lw, lh);
oc.getContext('2d').putImageData(imgData, 0, 0);
cache[layer.id] = { bitmap: oc, key: d.key, lutKey, w: lw, h: lh };
return oc;
}

// Composite every VISIBLE layer over the base image on plotCanvas, bottom-up,
// using the SAME zoom/pan/fit-rect transform as the base blit (so zoom/pan track
// exactly) at each layer's own globalAlpha. Layers draw AFTER the base + overlay
// mask and BEFORE axes/markers/widgets, so they sit under annotations + widgets.
// Works identically over a WebGPU base (which lives on the gpuCanvas beneath the
// transparent plotCanvas): the layers just paint onto plotCanvas above it.
function _drawLayers2d(p, st, imgW, imgH, ctx, iw, ih) {
const layers = st.layers;
if (!layers || !layers.length) { if (p._layerBlit) p._layerBlit = {}; return; }
// Drop cache entries for layers that were removed.
if (p._layerBlit) {
const live = new Set(layers.map(l => l.id));
for (const k in p._layerBlit) if (!live.has(k)) delete p._layerBlit[k];
}
const { x, y, w, h } = _imgFitRect(iw, ih, imgW, imgH);
const z = st.zoom, cx = st.center_x, cy = st.center_y;
for (const layer of layers) {
if (layer.visible === false) continue;
const bmp = _layerBitmap(p, st, layer);
if (!bmp) continue;
const alpha = layer.alpha != null ? layer.alpha : 1.0;
const bw = bmp.width, bh = bmp.height;
const kx = bw / iw, ky = bh / ih; // logical→layer-texel scale (==1 here)
ctx.save();
ctx.globalAlpha = alpha;
ctx.imageSmoothingEnabled = false;
if (z >= 1.0) {
const visW = iw / z, visH = ih / z;
const sx = Math.max(0, Math.min(iw - visW, cx * iw - visW / 2));
const sy = Math.max(0, Math.min(ih - visH, cy * ih - visH / 2));
ctx.drawImage(bmp, sx * kx, sy * ky, visW * kx, visH * ky, x, y, w, h);
} else {
const dw = w * z, dh = h * z;
ctx.drawImage(bmp, 0, 0, bw, bh, x + (w - dw) / 2, y + (h - dh) / 2, dw, dh);
}
ctx.restore();
}
}

function _blit2d(p, bitmap, st, pw, ph, ctx, detailBitmap) {
const { x, y, w, h } = _imgFitRect(st.image_width, st.image_height, pw, ph);
const zoom = st.zoom, cx = st.center_x, cy = st.center_y;
Expand Down Expand Up @@ -1769,6 +1900,14 @@ function render({ model, el }) {
ctx.restore();
}
}
// ── Image layers ─────────────────────────────────────────────────────────
// Composited over the base image (+ overlay mask) at each layer's own
// colormap/clim/alpha, using the same zoom/pan transform, UNDER axes/markers/
// widgets. For a WebGPU base the layers paint on the transparent plotCanvas
// that sits above the gpuCanvas. iw/ih == the logical image size here (tiling
// is forbidden when layers exist, so base_width is 0 and iw == image_width).
_drawLayers2d(p, st, imgW, imgH, ctx, iw, ih);

// Axes / scalebar / colorbar
_drawAxes2d(p);
drawScaleBar2d(p);
Expand Down
5 changes: 3 additions & 2 deletions anyplotlib/plot2d/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""anyplotlib.plot2d — 2-D image plot panel classes (Plot2D, PlotMesh)."""
"""anyplotlib.plot2d — 2-D image plot panel classes (Plot2D, PlotMesh, Layer)."""
from anyplotlib.plot2d._plot2d import Plot2D
from anyplotlib.plot2d._plotmesh import PlotMesh
from anyplotlib.plot2d._layer import Layer

__all__ = ["Plot2D", "PlotMesh"]
__all__ = ["Plot2D", "PlotMesh", "Layer"]
Loading
Loading