You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Under machine load, pnpm --filter @visionset/ui-core test fails with a handful of tests
timing out at vitest's 5000ms default. Which tests fail varies run to run, and how many
tracks the load average rather than anything in the diff. Every failure observed so far has
been Error: Test timed out in 5000ms, plus one variant where userEvent.type delivered
only part of a string before the assertion ran.
The files involved are consistently the heaviest ones — the ones mounting a screen under a
real ApiProvider with a stubbed fetch and driving it through userEvent:
src/screens/schemaDraft.test.tsx
src/screens/screens.test.tsx (the schema editor and version history groups)
Four consecutive runs on one machine over about twenty minutes, as load fell from a load
average of roughly 90 to roughly 7, produced 41, 19, 11 and 5 failures. The set is
nested rather than random: the same files fail, and fewer of their tests fail as the machine
frees up. Running the same three files alone, immediately after a run in which all three had
failed, passes every one of their 75 tests.
CI does not see it. The suite is green on clean runners, which is why this has gone
unnoticed — it costs nothing there and costs a local gate its meaning here.
Why it matters
A local bash scripts/check.sh is what the contribution protocol asks for before a push,
and a gate that fails for reasons unrelated to the diff trains people to read red as noise.
That is the same failure mode #550 and #511 describe, one suite over, and the cost is the
same: the run that finally is about the change looks like the four before it.
It is also actively misleading during a task. A branch under this condition can show fewer
failures than main does, which is what happened while this was being diagnosed.
Likely direction
The 5000ms default is vitest's, and these tests are near it even when they pass: the same
run reports cumulative test time far in excess of wall-clock, so several workers are
contending for one machine. Two things worth measuring before choosing:
Raise testTimeout for the suite, which is the smallest change and the one that
admits what these tests are: several of them mount a screen, wait on TanStack Query, and
drive a dozen userEvent interactions. A timeout tuned for a pure function is not a
meaningful bound on that.
Whichever is chosen, the check is the same and it is cheap: reproduce under synthetic load,
apply, and confirm the suite is stable across several consecutive runs — the repetition
discipline tests/kernel/test_concurrency.py already sets for this repository.
Do not fix it by deleting or skipping the affected tests. They cover the schema draft's
survival across a tab switch and a version published underneath it, which is exactly the
class of defect #389 was.
What happens
Under machine load,
pnpm --filter @visionset/ui-core testfails with a handful of teststiming out at vitest's 5000ms default. Which tests fail varies run to run, and how many
tracks the load average rather than anything in the diff. Every failure observed so far has
been
Error: Test timed out in 5000ms, plus one variant whereuserEvent.typedeliveredonly part of a string before the assertion ran.
The files involved are consistently the heaviest ones — the ones mounting a screen under a
real
ApiProviderwith a stubbedfetchand driving it throughuserEvent:src/screens/schemaDraft.test.tsxsrc/screens/screens.test.tsx(the schema editor and version history groups)src/screens/inference.test.tsxsrc/annotator/addClassProvenance.test.tsxsrc/annotator/addClassDialog.test.tsx,src/annotator/drawingClass.test.tsxIt is not a change anybody made
Measured on
49a3765, unmodified, on an Apple M-series laptop, with the same command in thesame environment:
Four consecutive runs on one machine over about twenty minutes, as load fell from a load
average of roughly 90 to roughly 7, produced 41, 19, 11 and 5 failures. The set is
nested rather than random: the same files fail, and fewer of their tests fail as the machine
frees up. Running the same three files alone, immediately after a run in which all three had
failed, passes every one of their 75 tests.
CI does not see it. The suite is green on clean runners, which is why this has gone
unnoticed — it costs nothing there and costs a local gate its meaning here.
Why it matters
A local
bash scripts/check.shis what the contribution protocol asks for before a push,and a gate that fails for reasons unrelated to the diff trains people to read red as noise.
That is the same failure mode #550 and #511 describe, one suite over, and the cost is the
same: the run that finally is about the change looks like the four before it.
It is also actively misleading during a task. A branch under this condition can show fewer
failures than
maindoes, which is what happened while this was being diagnosed.Likely direction
The 5000ms default is vitest's, and these tests are near it even when they pass: the same
run reports cumulative test time far in excess of wall-clock, so several workers are
contending for one machine. Two things worth measuring before choosing:
testTimeoutfor the suite, which is the smallest change and the one thatadmits what these tests are: several of them mount a screen, wait on TanStack Query, and
drive a dozen
userEventinteractions. A timeout tuned for a pure function is not ameaningful bound on that.
contention is between workers rather than with the rest of the machine, this is the fix
that keeps the timeout honest.
Whichever is chosen, the check is the same and it is cheap: reproduce under synthetic load,
apply, and confirm the suite is stable across several consecutive runs — the repetition
discipline
tests/kernel/test_concurrency.pyalready sets for this repository.Do not fix it by deleting or skipping the affected tests. They cover the schema draft's
survival across a tab switch and a version published underneath it, which is exactly the
class of defect #389 was.