fix: one setting, one wait, and a box over the whole object - #562
Merged
Conversation
Two of the three suggestion settings did nothing to an ordinary single clean mask — every position produced an identical shape — so they read as controls wired to nothing, and could only be got wrong on the unusual mask. Both come off the wire. Both behaviours stay, at fixed defaults beside the pipeline. - `SuggestParameter` is `detail` alone, declared for polygons only. A box class declares nothing, which is how a client is told to render no adjustments. - Specks under a twentieth of the largest piece are dropped unconditionally, and `components` now leads with the piece the prompt points at. - A box is the union of every surviving piece. A point prompt means *this object*, and a mask in several pieces is nearly always one object seen around an occlusion; largest-only cuts it off and one-box-per-piece doubles it. - The closing reach is capped, and a box skips the close entirely because it cannot move an extent. Per click at 4K: 44 morphological passes become 12 for a polygon and 0 for a box. cf. #557
…vertices - The ring at the click point and the busy cursor are gone. An indicator sitting on the picture beside the pointer reads as the machine having seized rather than as work in progress, and the card already says so in words. The panel's message is the only report; the cold-start sentence keeps its clock. - A suggestion preview is a third visual state, not a shape marked selected: its vertices are drawn the whole time it is up, undecimated at every step, because where precision was gained or lost is what the setting is about. - `detail` becomes a three-position slider with a live `Fine · 41 pts` label, replacing three text labels that did not read as pressable. Pressing it never moves focus off the canvas, so the chords keep working. - The two mask settings and the re-ask path they drove are deleted. That path sent one inference request per input event of a drag. cf. #557
The skip is a performance rule with no behavioural signature — a close cannot move an extent, which is what makes it safe to skip and what makes it invisible to any assertion about the box. Asserted structurally instead. cf. #557
JArmandoAnaya
force-pushed
the
fix/autolabel-corrections
branch
from
August 12, 2026 18:57
4ead019 to
1af10df
Compare
Contributor
Author
|
Rebased onto
Left open, as the dispatch asks. |
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seven items were reported against the suggest tool after dogfooding. Three of the reported
mechanisms turned out to be wrong, and the diagnosis is the more useful half of this change:
two of the three "regressions" are behaviours that already work, and the thing actually making
the tool slow and unpredictable was a control nobody could see was firing requests.
Closes #557.
What was actually wrong, mechanism by mechanism
A. A spinner at the cursor — real, and removed
A ring at the click point and a
progresscursor, both from #543. Removed:SuggestHalo, its constants, the cursor and the prop plumbing that carried it, anduseReducedMotion, which had no other caller. The panel'sLooking at that…card is now theonly report of a wait. The 1.5s cold-start sentence keeps its clock, which is what that clock
is now for.
B. "Every click pays full inference cost" — the stated mechanism is falsified
There is an embedding cache, and nothing removed or bypassed it: a
BoundedCache[UUID, …]ofcapacity 8 on the process-wide provider pool, keyed on
asset_id, with anencodescounterthat exists for exactly this question and a test that asserts it
(
test_a_second_click_on_the_same_asset_decodes_without_encoding_again). Every line of it is a+fromb89c88b(#448) with no-in any later commit:49a3765's whole change toproviders.pyis a return-type widening. The cache is untouchedby this PR.
What did get heavier on the decode side is post-processing, which
49a3765moved above theport. Two costs were addressable and both are taken here:
which is 44 bitset passes to bridge gaps that were never there. Now capped.
is what makes skipping it safe and what makes it invisible to any assertion about the box.
The geometry branch moved ahead of the close, and a polygon now runs it on the one piece it
is about to trace rather than on every piece that survived.
Morphological passes per click, counted rather than timed — the numbers are deterministic and
hardware-independent, which wall clock on this machine emphatically is not (see the gate note):
This does not make repeat clicks light, and the PR does not claim it does. The dominant
per-click term is the adapter marshalling a full-resolution mask into Python lists — measured
at roughly the same order as the whole pipeline at 4K — and it predates #551. It is out of this
change's scope (the pipeline's math is), and it is filed as #561.
C. "Adjusting
detailre-runs inference" — falsified both waysNeither variant holds. The contour reaches the client (
AnnotationPage.tsxreadsregion.contour), andapplyDetail → withDetail → polygonAtis local arithmetic with nomutateanywhere in it.annotate.spec.tsalready asserted a flat network log across[and]and still does.The control that re-ran the model is
Close gaps. It is a range input whose ReactonChangefires on every step of a drag, and each one sent a fresh suggest request — up to tenper drag, undebounced. That control is removed by E, which is the fix; the diagnosis is
recorded here rather than a debounce added to a path that no longer exists.
D. The preview drew no vertices — half-shipped
The dashed outline already shipped (
SUGGESTION_DASH = "10 6", opacity 0.6). What was missingwas the vertices, so
detailmoved a number and nothing anybody could see. The preview is nowa third visual state with its own rule — not a shape marked selected, because selection
carries the panel row, the delete key and keyboard behaviour a proposal must not have.
DESIGN.md's "vertices render only while selected" is amended in the same change, so the prosecontract cannot age into contradiction with the code.
E. Two settings out, and a box becomes a union
fill_holesandfragmentsleave the wire and the editor; both behaviours stay at fixeddefaults beside the pipeline. A box class now declares no parameters, which is how a client
is told to render no adjustments section — the editor still contains no condition mentioning a
box.
The bbox rule changed: after noise filtering the box is the union of every surviving piece.
A point prompt means this object, and a mask arriving in pieces is nearly always one object
seen around an occlusion; largest-only cuts it off at the occlusion and one-box-per-piece
annotates it twice.
F.
detailas a sliderA three-position native range with a live
Fine · 41 ptslabel. Pressing it never moves focusoff the canvas — every chord in the editor is a
keydownon the annotator's root — while Tabstill reaches it, so keyboard operation stays a deliberate arrival rather than a side effect of
pointing at it.
G. No pointer leak exists, and the guard passed on the first run
The card is
pointer-events-autoinside apointer-events-nonecolumn and is a sibling ofthe canvas, so there is no DOM path from a press on it into the
<svg>. A new browser specpresses every control on the card and asserts zero suggest requests and an unmoved seed point:
it passed before any change was made to that path, and it is a regression guard rather than a
red-before-green reproduction. Said plainly because a green test presented as a fix is worse
than no test.
What the report describes is C seen from the other side: pressing
Close gapsdid fire a newsuggest, legitimately, because that is what the control did. Removing it removes that too.
Red-before-green
Acceptance criterion 7 asks for A, B, C and G to fail on pre-fix HEAD. Only A can, and the
reason is the diagnosis above: B's cache and C's local re-simplification already work, and G's
leak does not exist. Claiming three red baselines here would mean writing tests against
mechanisms that are not there.
says so on the card and puts nothing at all on the canvasfails on pre-fix HEAD,and the mutation below restores the busy cursor to prove it.
is asserted structurally (
test_a_box_never_pays_for_the_close_at_all) and measured by passcount. The cache half needs no test; it has one already, passing, on
main.Mutation verification
Work committed first. One mutation at a time, anchor asserted present-and-unique before and
replacement asserted present after, reverted by its own recorded patch, each step an
unconditional statement, runner output to a file. Every row turned a named test red.
_union_of'srighttakes the first boxtest_a_box_is_one_union_over_every_surviving_piece0.0test_the_union_leaves_out_the_specks_the_noise_filter_droppedtest_the_reach_stops_at_the_cap_however_large_the_piecedetaildeclared forBBOXtootest_a_box_class_is_told_nothing_appliestest_the_piece_under_the_click_leads_even_when_it_is_not_the_biggesttest_a_box_never_pays_for_the_close_at_allbalancedreports a step through the door that needs no requestonMouseDowndropped from the sliderdoes not let a press on the slider take focus off the canvasnames the step and what it costs in one labelsays so on the card and puts nothing at all on the canvasthe preview draws its vertices, and a committed shape does notThree of these lied before they told the truth, and the harness caught each one.
_union_of'sleft = min(...)mutated toboxes[0].xcame back green: the union hasfour sites, and in a fixture whose pointed-at piece is already the leftmost that spelling is
a no-op. Mutating
rightinstead reddens the named test. Stopping at the first green wouldhave reported the rule unverifiable.
preview-verticesmutation deleted the whole element, which orphaned its importand failed
tsc, so the browser suite never started — a broken harness, not a covered rule.Redone type-validly (
points={[]}) it reddens exactly the named test.cds intofrontend/app, sogit apply -Rran from the wrong directory andsilently did not revert. The harness's own dirty-tree check caught it on the next case;
reverted by hand and the tree verified clean before continuing.
The gate
Staged, because the harness kills any command at about ten minutes. Every stage's exit code.
#550's two flaky scenarios passed this time, at the same worker count. Nothing here touches
them; noted so the run is not read as evidence they are fixed.
A note on the timing numbers. This machine's wall clock swung 4× between identical runs
(the same benchmark on unmodified
mainmeasured 296 ms and then 1227 ms), and CPU time swungwith it under SMT contention. That is why the performance claim above is a count of
morphological passes rather than a duration: it is exact, reproducible and independent of the
box. A real wall-clock number wants your GPU machine, and the command is in the flags below.
Found, not fixed
the largest single per-click term at 4K — larger than the whole post-processing pipeline —
and predates feat: post-inference mask post-processing and adjustable suggestion parameters #551. Out of scope here; filed as The suggest adapter converts a full-resolution mask into Python lists on every click #561.
regionsnow always carries 0 or 1 shapes, since both geometries answer singly. The pluralshape is kept deliberately rather than collapsed, because accepting part of a plural proposal
is tracked work (Accepting part of a plural suggestion #548) and collapsing it would be a second breaking wire change to undo later.
Flags for Armando
(D). Worth knowing where the reports came from, because the pattern is consistent: all three
are things a person would infer from watching the tool rather than from reading it, and the
single real cause underneath two of them was one undebounced range input.
box cannot produce it — no torch installed, no CUDA, and the load makes timing meaningless.
On your GPU machine:
uv run pytest tests/inference/test_sam_provider.py -qfor the encoder count, then twoconsecutive suggest clicks on one 4K frame with the network panel open.
MAXIMUM_CLOSING_RADIUS = 6is the one number here without a prior calibration behind it.It is chosen so a 4K object closes gaps of a few pixels rather than 22, which is the scale of
the artefact rather than of the object. If real masks want more reach, it is one constant.
the halo (
TransientLayer.tsx, feat(annotator): in-flight indicator for interactive auto-labeling #543), the busy cursor (AnnotatorCanvas.tsx, feat(annotator): in-flight indicator for interactive auto-labeling #543), thepanel's
askingcard (SuggestPanel.tsx, Interactive auto-labeling — click-to-suggest in the editor canvas #424) and its cold-start sentence (feat(annotator): in-flight indicator for interactive auto-labeling #543). Thefirst two are gone; the last two stay and are the whole report now. A fifth, the
checkingspinner on the connection-list blocker, is about a different request and is untouched.