Skip to content

Consolidate on the index access method; auto-provision opclasses#1

Merged
bitner merged 2 commits into
mainfrom
consolidate-on-index-am
Jun 22, 2026
Merged

Consolidate on the index access method; auto-provision opclasses#1
bitner merged 2 commits into
mainfrom
consolidate-on-index-am

Conversation

@bitner

@bitner bitner commented Jun 22, 2026

Copy link
Copy Markdown
Owner

Makes CREATE INDEX ... USING table_range the single interface and removes the SQL function interface (table_range_create/refresh/drop/summary_count), the registration table, and the staleness triggers.

Highlights

  • Automatic operator-class provisioning (answers "reuse BRIN" without the catch). Opclasses can't be shared across access methods, but we mirror the coverage: a table_range_sync_opclasses() routine + a CREATE EXTENSION event trigger create matching table_range opclasses for every btree-ordered type, every range type, and PostGIS geometry/geography. Any btree-comparable or range type works out of the box, and PostGIS geometry registers the moment CREATE EXTENSION postgis runs — no manual step.
  • Simpler engine. summary_build.rs keeps only the ambuild path (scalar min/max + range/geometry extent). Staleness is maintained by aminsert; recompute via REINDEX; the sql_drop event trigger still cleans up dropped indexes/tables.
  • Tests rewritten onto CREATE INDEX (26 pass on pg18; production build, clippy -D warnings, and fmt all clean). Dropped two function-only tests (refresh needs REINDEX, which can't run inside a pg_test transaction; drop is covered by the AM drop test).

Performance — corrected, and now honestly benchmarked

An earlier revision of this PR claimed the index interface added ~85 ms of per-partition planning overhead. That was a measurement artifact — it compared two separate benchmark sessions under different machine load. Controlled, same-session A/B measurements show:

  • The index-validity effect on planning is ~3 ms on 500 partitions (warm), not ~85 ms. So the "mark indexes invalid" event trigger I had briefly added is reverted — it bought ~3 ms for a confusing INVALID display in \d. The ineffective get_relation_info_hook is reverted too.
  • The previous "210→100 ms planning win" was a relcache-warmth artifact (the off case ran first/cold, the on case second/warm). Pruning actually adds a small per-plan overhead.
  • The real benefit is execution time. On 100 partitions × 30k rows (3M rows, warm), a selective non-key predicate runs in ~18 ms with pruning vs ~125 ms without (~7×) because it scans 1 partition instead of 100. Pruning wins when the eliminated partitions are large enough to outweigh its overhead, and can be a slight net cost on tiny partitions.

bench/planning_benchmark.sql now measures total (plan + exec) time, warm, on realistic partition sizes, and the README performance section matches.

Follow-up (separate PR)

The ~20 ms per-plan overhead is mostly text→Datum conversions during per-partition evaluation. Caching parsed min/max as Datums would lower the break-even partition size and widen where pruning pays off.

bitner and others added 2 commits June 22, 2026 16:07
Make CREATE INDEX ... USING table_range the single interface and remove the
SQL function interface (table_range_create/refresh/drop/summary_count), the
registration table, and the staleness triggers.

- Operator classes are now provisioned automatically by mirroring the types that
  already have a btree or range operator class, re-running on every CREATE EXTENSION.
  So any btree-comparable type and any range type work out of the box, and PostGIS
  geometry registers the moment `CREATE EXTENSION postgis` runs — no manual step.
- summary_build.rs keeps only the ambuild path (scalar min/max + range/geometry
  extent); staleness is maintained by aminsert, recompute via REINDEX; the sql_drop
  event trigger still cleans up dropped indexes/tables.
- Tests rewritten onto CREATE INDEX (26 pass on pg18); dropped the two function-only
  tests (refresh needs REINDEX which can't run in a pg_test txn; drop is covered by
  the AM drop test).
- README/benchmark updated. Honest planning-time note: owning summaries in a real
  index adds per-partition index-metadata overhead during planning (~85 ms flat on a
  bare 1000-partition table); pruning still removes ~110 ms of child-path planning.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rtifact

While investigating the "per-partition index planning overhead," controlled
same-session A/B measurements showed my earlier numbers were confounded by
relcache warmth and cross-session machine load:

- The index-validity effect on planning is ~3 ms on 500 partitions (warm), not the
  ~85 ms I previously reported. So the invalid-index event trigger I added is not
  worth its cost (a confusing INVALID display) and is reverted, along with the
  ineffective get_relation_info_hook.
- The real, clean benefit is at EXECUTION: on 100 partitions x 30k rows (3M rows,
  warm), a selective non-key predicate runs in ~18 ms with pruning vs ~125 ms
  without (~7x). Pruning adds a small per-plan overhead and can be a slight net cost
  on tiny partitions; it wins when eliminated partitions are large.
- Rewrote bench/planning_benchmark.sql to measure total (plan+exec) time, warm, on
  realistic partition sizes, and rewrote the README performance section to match.
- Gated the test-only cache-load diagnostics so production builds are warning-free.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bitner bitner merged commit 5cd6672 into main Jun 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant