Add Int8Rescored: int8 graph walk with exact float re-ranking

[matengo]

Summary

Adds VectorQuantization.Int8Rescored: the HNSW graph is built and walked on int8 codes exactly as in Int8, but the original floats are kept in the file and the efSearch candidates are re-ranked against them. Returned scores and top-k are the float ones; GetByGuid returns the original vector.

This is the "float refiner" the Cohere 1M benchmark write-up identified as the remaining lever, and the equivalent of zvec's --is-using-refiner.

Format

No version bump, no new QuantizationMode. A rescored file is an int8 file (QuantizationMode = 2) that additionally carries section 1 (Vectors) as an optional, non-Required section in slot 8, with HasOptionalSections set. Readers that predate this change open it as plain int8. QvecDatabase.Quantization reports Int8Rescored when the section is present; reopening with the wrong mode throws in both directions. Header validation checks the float section's shape when present. Float and plain int8 files are unchanged (existing tests pass untouched).

Implementation

• _vectorSectionOffset split into _floatVectorSectionOffset / _codesSectionOffset; _rescore flag.
• WriteVectorToDisk writes codes + params and, when rescored, the floats too — so AddEntry, AddEntries, UpdateVector and Vacuum all keep both sections in step.
• ReadVectorInto returns the exact floats when available.
• FinalScore / RescoreCandidates: exact float scoring after SearchLayerNearest / SearchLayerFiltered, and in every exhaustive path. No-ops for float and int8 files.
• Benchmark: --quantization int8rescored.

Tests

RescoredQuantizationTests (16): layout, Open reports the mode, mismatch both ways, exact GetByGuid, scores equal float brute force (1e-4) in all three metrics, recall >= 97 % on tight cosine clusters where int8 alone measured 89.8 %, filtered search, Update/Delete/Vacuum, Grow, parallel AddEntries. Fast suite + Slow suite green, 0 warnings.

Measurements (Cohere 1M, 768-d cosine, k=100, M=15, 12 threads)

mode	build	file	ef	recall@100	QPS 12 thr	QPS 1 thr
float	419 s	3,376 MiB	180	94.8 %	3,764	565
int8	181 s	1,194 MiB	180	93.2 %	6,841	839
int8 rescored	170 s	4,124 MiB	180	94.7 %	6,778	630
float			320	97.4 %	2,207	351
int8			320	95.3 %	4,714	502
int8 rescored			320	97.4 %	3,884	421

Float recall at int8 throughput, paid for in storage (~1.25x float). Honest caveat, in the docs: at efSearch == k re-ranking cannot add candidates, so the ef 100 row stays at int8 recall (89.2 %). Float/int8 indexes re-queried the same day were within 6 % of the table.

siftsmall: rescored 99.1 / 99.8 % recall@10 at ef 20 / 80 vs float 98.6 / 99.5 and int8 98.3 / 99.0.

Docs

docs/design-quantization-rescoring.md (new), docs/design-quantization-int8.md, docs/design-format-v4.md, benchmarks/README.md, README.md.