Skip to content

refactor(ivf): use turbo for distance calculation - #773

Open
richyreachy wants to merge 17 commits into
alibaba:mainfrom
richyreachy:refactor/ivf_on_turbo
Open

richyreachy wants to merge 17 commits into
alibaba:mainfrom
richyreachy:refactor/ivf_on_turbo

Conversation

@richyreachy

Copy link
Copy Markdown
Collaborator

Refactor IVF to use the Turbo quantizer APIs adopted by HNSW and Flat. Keep centroid training and preprocessing separate from posting encoding, and use per-query Turbo distance state for batched posting scans.

  • Persist quantizer metadata and training state for reload.
  • Preserve legacy index loading and converter fallbacks.
  • Decode vectors for fetch and providers, including merges between legacy and Turbo indexes.
  • Correct cosine centroid preprocessing and support batched centroid-query transforms.
  • Reject incompatible column-major layouts and packed-only quantizers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant