Filter compression schemes by serialized IDs instead of array IDs

[mhk197]

Summary

Compression schemes are filtered against the IDs a writer may emit. Today that filter mixes two ID spaces. Schemes declare their in-memory encoding ID, while most callers pass the serialized IDs from their enabled editions. The two only line up because every encoding currently uses the same string for both.

DecimalByteParts stops doing that in #9834. Its in-memory ID becomes vortex.decimal_byte_parts.v2 while the frozen wire ID stays vortex.decimal_byte_parts. With the current filter, every caller that passes edition IDs would silently drop the decimal scheme and write decimals canonical.

This PR makes the contract explicit. Schemes declare the serialized IDs they write, and every caller filters with serialized IDs.

Changes

• The file writer no longer maps permitted serialized IDs through the registry to plugin in-memory IDs before filtering schemes. new_array_context returns only the array context, which is seeded with exactly the permitted IDs, and the writer reads them back with to_ids into a named set for the compressor builder. This matches the Python, TUI, CUDA, bench, and golden-test callers, which already pass serialized IDs.

Behavior

No scheme changes behavior. In-memory and wire IDs coincide for every scheme on develop, so the same schemes are retained from the same inputs and no snapshot changes.

Follow-ups

• Support i128 and i256 decimals in DecimalByteParts encoding  #9834 declares vortex.decimal_byte_parts from the decimal scheme on top of this.
• Tell the compressor which serialized IDs the writer may emit #9770 builds on this with a permitted-ID snapshot in CompressorContext, so a scheme can choose a wire format when it compresses. If the method names should say serialized IDs, renaming produced_encodings and retain_allowed_encodings together belongs there.

[codspeed]

Merging this PR will degrade performance by 9.72%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 2 improved benchmarks
❌ 5 regressed benchmarks
✅ 2195 untouched benchmarks
⏩ 218 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	arrow_checked_add_u32_neon[16384]	12.3 µs	20.4 µs	-39.91%
❌	WallTime	subtract_shapes_neon[(128, PerRowPerRow)]	1.8 µs	2 µs	-10.77%
❌	Simulation	allocate_drop_vortex_minimal_alignment[0]	913.3 ns	1,021.7 ns	-10.6%
❌	WallTime	multiply_shapes_neon[(128, PerRowPerRow)]	1.9 µs	2.1 µs	-10.34%
❌	Simulation	allocate_drop_vortex[0]	969.2 ns	1,077.5 ns	-10.05%
⚡	WallTime	filtered_sink_i64_avx2[OneNullInEight]	26 µs	22.9 µs	+13.65%
⚡	WallTime	filtered_sink_i64_avx512[OneNullInEight]	25.6 µs	23 µs	+11.23%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/scheme-serialized-ids (d308072) with develop (96b2684)²}

Footnotes

1. 218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on develop (a8e949e) during the generation of this report, so 96b2684 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[robert3005]

I was trying very hard to not create it twice but I recognise this is over optimising