Every interactive suggest click converts the model's mask into nested Python lists at the
asset's own resolution before any of the post-processing runs. On a large frame that is the
single biggest thing the request does - measured at roughly the same order as the whole
pipeline behind it, and larger than any individual step of it.
It happens on the decode side of the embedding cache, so it is paid in full on every click
including the ones that are meant to be cheap. That is what stops "the first click is the slow
one" from being true in practice, whatever the cache does.
Where it is
_segments in the SAM adapter hands the mask across the port with .tolist(). The port's own
type is a sequence of sequences of booleans, and the pipeline above it reads the mask a row at
a time, so nothing downstream needs a Python list in particular - it needs something with
len, indexing and a fast per-row scan.
Why it is not simply fixed
The obvious answers each cost something real:
- Handing the array across the port unchanged puts the numeric stack's semantics behind a
kernel-pure type, and the row scan the pipeline relies on is not available on it.
- Tracing at the model's native mask resolution and scaling the contour up changes what the
geometry is, at a precision the current tests pin.
So this is a design question rather than a patch, which is why it is filed rather than taken
in passing.
Measurement
The pipeline's own cost is already bounded and countable (cf. the post-processing work). The
conversion is not, and it scales with the asset rather than with the object that was clicked -
a 4K frame is eight million booleans whether the thing clicked is a lorry or a bottle cap.
Wall-clock numbers should come from a machine with a GPU and no other load; this was measured
on a box where the same benchmark varied four-fold between runs, so the honest statement here
is the shape of the cost rather than a duration.
Every interactive suggest click converts the model's mask into nested Python lists at the
asset's own resolution before any of the post-processing runs. On a large frame that is the
single biggest thing the request does - measured at roughly the same order as the whole
pipeline behind it, and larger than any individual step of it.
It happens on the decode side of the embedding cache, so it is paid in full on every click
including the ones that are meant to be cheap. That is what stops "the first click is the slow
one" from being true in practice, whatever the cache does.
Where it is
_segmentsin the SAM adapter hands the mask across the port with.tolist(). The port's owntype is a sequence of sequences of booleans, and the pipeline above it reads the mask a row at
a time, so nothing downstream needs a Python list in particular - it needs something with
len, indexing and a fast per-row scan.Why it is not simply fixed
The obvious answers each cost something real:
kernel-pure type, and the row scan the pipeline relies on is not available on it.
geometry is, at a precision the current tests pin.
So this is a design question rather than a patch, which is why it is filed rather than taken
in passing.
Measurement
The pipeline's own cost is already bounded and countable (
cf.the post-processing work). Theconversion is not, and it scales with the asset rather than with the object that was clicked -
a 4K frame is eight million booleans whether the thing clicked is a lorry or a bottle cap.
Wall-clock numbers should come from a machine with a GPU and no other load; this was measured
on a box where the same benchmark varied four-fold between runs, so the honest statement here
is the shape of the cost rather than a duration.