Skip to content

feat(file_index): add Java-compatible Bitmap V2 writer - #759

Open
QuakeWang wants to merge 1 commit into
apache:mainfrom
QuakeWang:bitmap-v2-writer
Open

feat(file_index): add Java-compatible Bitmap V2 writer#759
QuakeWang wants to merge 1 commit into
apache:mainfrom
QuakeWang:bitmap-v2-writer

Conversation

@QuakeWang

Copy link
Copy Markdown
Member

Purpose

Related issue: #34

The Bitmap file-index module was read-only. It could consume Java V1/V2 payloads but could not produce the V2 format used by Java Paimon 2.0, leaving Rust without a compatible Bitmap write path.

Brief change log

  • Add a crate-private Java-compatible Bitmap V2 writer.
  • Write row/null and per-value RoaringBitmap32 data.
  • Support singleton offsets, V2 metadata, secondary index blocks, and index-block-size.
  • Preserve Java-compatible value encoding.
  • Reject invalid configuration and unsupported data types.
  • Add Java golden and Rust reader tests for nulls, singletons, multiple bitmap bodies, multiple blocks, and variable-length block boundaries.

Tests

  • cargo fmt --all -- --check
  • cargo clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings
  • cargo test --locked -p paimon --all-targets --features fulltext,vortex
  • Java release-2.0 golden and reader interoperability probes

API and Format

No public API change. This adds Java-compatible Bitmap V2 payload writing. V1 writing remains unsupported.

Documentation

No documentation changes.

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant