test: run full suite only on latest PG, essential subset on 14-17#489
Conversation
Re-running the entire suite on every supported PostgreSQL version mostly re-tests version-agnostic logic and forces a growing skip list for cosmetic differences (pg_get_viewdef() column qualification before PG 16) and version-gated features (NULLS NOT DISTINCT, temporal constraints). New strategy: - LatestPostgresVersion (currently 18) is the single point of indirection. The full suite runs only against it; bumping it is a one-line switch. - Older versions (14..latest-1) run only essentialTests, a representative smoke-test subset (one or two cases per object type) whose golden files are byte-stable across all versions. Enough to catch gross version-specific breakage without re-running everything. - The default test version (no env var) now derives from LatestPostgresVersion, so PR CI keeps running the full suite. Replaces the per-version skipList* tables with this model; extension-required skips are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates the cross-version CI testing strategy so the full test suite runs only on the latest supported PostgreSQL major version (via a single LatestPostgresVersion constant), while older supported versions run a curated essential smoke-test subset to reduce version-specific golden drift and skip-list maintenance.
Changes:
- Introduces
LatestPostgresVersionand anessentialTestsallowlist, and rewritesShouldSkipTestto enforce “full suite on latest, essential subset on older versions”. - Removes the per-version skip lists (PG14/15 viewdef drift, version-gated feature skips) in favor of the essential subset approach.
- Makes the default embedded Postgres version (when
PGSCHEMA_POSTGRES_VERSIONis unset) derive fromLatestPostgresVersion.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
testutil/skip_list.go |
Replaces version-specific skip lists with LatestPostgresVersion + essential subset gating logic. |
testutil/postgres.go |
Defaults test Postgres version selection to LatestPostgresVersion when env var is unset. |
Comments suppressed due to low confidence (1)
testutil/postgres.go:174
getPostgresVersionfalls back toembeddedpostgres.V18for any unrecognizedPGSCHEMA_POSTGRES_VERSION. This silently ignores misconfiguration and also means bumpingLatestPostgresVersionalone would not update the default version (e.g., ifLatestPostgresVersionbecomes 19, the empty env-var path would still return V18 via thedefaultcase). Prefer failing fast (or mapping strictly) on unknown versions instead of defaulting to V18.
case "18":
return embeddedpostgres.V18
default:
return embeddedpostgres.V18
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Greptile SummaryThis PR replaces the growing per-version skip-list approach with a two-tier test strategy: the full suite runs only against
Confidence Score: 4/5Safe to merge; the new strategy is correct in its current state (LatestPostgresVersion=18, case "18" present), and the essential subset is well-chosen. The only concern is that testutil/postgres.go — the Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[ShouldSkipTest called] --> B{In skipListRequiresExtension?}
B -- Yes --> C[t.Skipf: extension not available]
B -- No --> D{majorVersion >= LatestPostgresVersion?}
D -- Yes --> E[Return: run full suite]
D -- No --> F{In essentialTests?}
F -- Yes --> G[Return: run test]
F -- No --> H[t.Skipf: only essential subset on older versions]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[ShouldSkipTest called] --> B{In skipListRequiresExtension?}
B -- Yes --> C[t.Skipf: extension not available]
B -- No --> D{majorVersion >= LatestPostgresVersion?}
D -- Yes --> E[Return: run full suite]
D -- No --> F{In essentialTests?}
F -- Yes --> G[Return: run test]
F -- No --> H[t.Skipf: only essential subset on older versions]
|
Summary
Changes the cross-version test strategy: the full suite runs only against the latest PostgreSQL version (currently 18), while older versions (14–17) run only an essential smoke-test subset.
Why
Re-running the entire suite on every supported version mostly re-tests version-agnostic logic and forced a steadily growing skip list for:
pg_get_viewdef()qualifies view columns differently before PG 16 (tb_users.idvsid).NULLS NOT DISTINCT(PG 15+), temporal constraintsWITHOUT OVERLAPS(PG 18+).How
LatestPostgresVersion(currently18) is the single point of indirection. The full suite runs only against it; the older versions run onlyessentialTests. Bumping to a new major version is a one-line change (plus adding the new version to the CI matrix and itsembedded-postgresenum case).essentialTestsis a representative subset — one or two cases per object type (tables, columns, constraints, indexes, functions, procedures, sequences, types, domains, triggers, policies, aggregates, comments, privileges) — chosen so their golden files are byte-stable across all supported versions (no view definitions, no version-gated syntax). Enough to catch gross version-specific breakage (e.g. a catalog query that fails on an older release) without re-running everything.PGSCHEMA_POSTGRES_VERSION) now derives fromLatestPostgresVersion, so PR CI keeps running the full suite.skipListPG14*tables; the extension-required skip list is unchanged.Verification
TestDiffFromFilesandTestPlanAndApply.🤖 Generated with Claude Code