docs: fragments: one is the piece you pointed at, not the biggest one - #554
Merged
Conversation
Decision (Armando, 2026-08-12): prompt-anchored selection is kept; the wording follows it. The shipped rule was already this — masks._pointed_at picks the piece under the points, falls back to the largest of several that are pointed at, and to the nearest one where a mask does not cover the exact pixel clicked. What was missing was anywhere a reader could learn it: the table said "`one` piece or `all` of them", which invites the guess that `one` means the biggest. docs/inference.md now states the three cases and why negatives never select, and says out loud what the rule is not — picking whichever piece owns the topmost-leftmost lit pixel is a fact about where the speckle fell rather than about what was asked for. docs/ui.md gets the short form. The Fragments docstring is widened too, because that one is the client-facing contract: pydantic does not publish a StrEnum's per-member comments, so the spec's description was the generic sentence alone and a caller reading it had no way to learn any of this. openapi.json and the generated client follow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The suggestion pipeline's
fragmentssetting has two values, and the single case picks the pieceyour points are on rather than the biggest piece on the frame. That has been the behaviour since
it shipped and it is argued in the code, but nowhere a reader would find it: the table in
docs/inference.mdsaid "onepiece orallof them", which invites exactly the wrong guess.Decision (Armando, 2026-08-12): prompt-anchored selection is kept; the wording follows it. This is
the flag raised in #551's "Flags for Armando" section, settled.
What changed
docs/inference.mdnow states the rule and its three cases — a point inside a piece picks thatpiece, several points across several pieces pick the largest of those because two positives
describe one object, and a point inside none of them picks the nearest, since a mask need not cover
the exact pixel clicked. Negative points never select: they say what the shape is not, and a piece
is chosen before its shape is known. It also says what the rule is not, because that is the
guess worth heading off — picking whichever piece owns the topmost-leftmost lit pixel is a fact
about where the speckle fell rather than about what was asked for.
docs/ui.mdgets the short form beside the control.The
Fragmentsdocstring is widened too, and that is the half that matters most. Pydantic doesnot publish a
StrEnum's per-member comments, so the spec carried only the generic sentence and aclient reading
openapi.jsonhad no way to learn any of this. The description now names the rule.openapi.jsonand the generated client follow; the only functional line in the diff is four linesof comment in
api.ts.No behaviour changes.
masks._pointed_atis untouched.The gate, and a declared fallback
Static gates, the generated-artifact drift gates, the script gates and every Python module this
change can reach, all green:
@visionset/ui-core's suite did not run clean here, and it does not run clean on unmodifiedmaineither. This machine is at a load average of 139 on eight cores, with several othersessions and a run of clang compilations on it:
On a clean, freshly built checkout of
mainat567f848with none of this branch's changes:On this branch the same suite failed 5 then 3 tests across two runs, with a different set each
time, and all of them pass when their files are run on their own:
A shifting failure set that reproduces on unmodified
mainis the machine, not the diff — and adiff of one docstring and two prose paragraphs has no mechanism to reach a schema-draft test.
So this is the sanctioned fallback, declared rather than taken silently. Not run locally: the
full
@visionset/ui-coresuite and both browser suites (annotator e2e (chromium),browser cycle (chromium)). CI on clean runners is the arbiter, and this will not be merged until everyrequired check is green there.