Skip to content

perf(ci): split the serialized integration collection to lower the suite floor #863

Description

@mforce

Split out of #839, which merged as a variance fix (#861). This is the lever that moves the floor.

The problem

dotnet test overlaps collections but runs tests within a collection serially. The shared integration collection is one collection holding 95 classes and 1,024 of 1,815 tests — 316.1s of test work that cannot overlap itself.

That 316s serial block is the suite's wall-clock floor. Measured on the #861 run: 331.5s total wall clock, of which the shared collection occupied 316.1s end to end. #861 stops the block landing at the end of the timeline (the 434s outlier); it cannot shrink the block.

Everything else — 142 classes, 852.4s of test work — already runs in parallel across 12 cores and fits inside the block's shadow. Splitting the shared collection is the only change that lowers the floor.

First task: measure, do not split

#839's own lesson applies. Its stated premise (per-class container startup) was already solved before anyone looked. Before designing a split, establish:

  1. What state the 95 classes actually share. The default account, base reference data (grades, roles, packed-unit conversions), Simulation:* config, and any advisory-lock assumptions. The shared fixture exists because they share something; find out what, per class, before assuming two classes are separable.
  2. Which classes are already isolated in practice — those that never write, or write only rows nobody else reads. Those are the candidates.
  3. What the floor would become at 2, 3, and 4 collections of the resulting groups. A split that produces two 158s blocks halves the floor; one that produces 300s + 16s does nothing.

Constraints

Explicitly not this issue

Test deletion and higher worker counts. #775 decision 1 argues against deletion; #776 landed coverage to answer "what is untested", which is never "what is redundant".

Known remaining cost, for whoever sizes this

The parallel work is lumpy: six classes hold 374s of the 852s. OtlpSubprocessExporterTests 80.5s for 11 tests, ProcessRoleGuardTests 71.7s, SeedCommandTests 61.0s, OneShotVerbMinimalConfigTests 58.7s. These repeatedly launch the app as a subprocess. Lowering the collection floor below roughly that height makes this the next floor, so measure both before committing to a split width.

Refs #839

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions