Skip to content

fix(retriever): skip FAISS padding slots instead of returning the last document - #518

Merged
xhd0728 merged 1 commit into
OpenBMB:mainfrom
winter-street:fix/faiss-search-negative-index
Sep 19, 2026
Merged

xhd0728 merged 1 commit into
OpenBMB:mainfrom
winter-street:fix/faiss-search-negative-index

Conversation

@winter-street

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

FAISS pads its result with -1 when the index holds fewer vectors than the
requested top_k. BaseIndexBackend.contents is a plain list
(index_backends/base.py:30), so self.contents[-1] wraps around to the last
document instead of signalling a miss: the padding came back as duplicated
passages, indistinguishable from real hits downstream.

The change skips those slots:

for doc_id in doc_ids:
    if doc_id == -1:
        continue
    cur_ret.append(self.contents[doc_id])

MilvusIndexBackend and QdrantIndexBackend are unaffected — they resolve
content from the payload the store returns rather than by positional index into
contents.

How did you test it?

Ran it, with faiss-cpu 1.15.1. A 3-passage index queried with top_k=5:

FAISS raw indices search() returns
before [0, 2, 1, -1, -1] ['DOC_A', 'DOC_C', 'DOC_B', 'DOC_C', 'DOC_C']
after [0, 2, 1, -1, -1] ['DOC_A', 'DOC_C', 'DOC_B']

This PR adds tests/servers/retriever/test_faiss_search_padding.py, four cases
covering the padding, top_k below the index size, top_k equal to it, and a
single-passage index:

uv sync --extra retriever
uv run pytest tests/servers/retriever/test_faiss_search_padding.py -v
# 4 passed

With the fix reverted, the two padding cases fail
(expected 3 real hits, got 5: ['DOC_A', 'DOC_C', 'DOC_B', 'DOC_C', 'DOC_C'])
and the two normal-path cases still pass, so the tests pin the behaviour rather
than the implementation.

uv run pytest tests/ servers/retriever/tests servers/generation/tests could
not be collected in my environment (openai, litellm not installed), so I
have not run the rest of the suite — the change is confined to one module.

Notes for the reviewer

PR #425 proposed the same guard, bundled with five other fixes across five
files, and was closed. I opened this as a single-file change with a regression
test in case the bundling was what sank it. If the team's reason for closing
that PR applies to this fix as well, please say so and I will close this one
rather than resubmit.

Checklist

  • One concern per PR, no unrelated reformatting.
  • Regression test added under tests/servers/retriever/.
  • ruff check and ruff format --check clean on the new test file. (faiss_backend.py reports seven pre-existing rules — I001, UP035, UP045, UP006 — on lines this change does not touch; I left them alone rather than bundle unrelated edits.)

…t document

FAISS fills the result with -1 when the index holds fewer vectors than the
requested `top_k`. `BaseIndexBackend.contents` is a plain list, so reading
`contents[-1]` for those slots returned the LAST document rather than a miss:
a 3-passage index queried with `top_k=5` came back as 5 passages, two of them
copies of the last one. Downstream that padding is indistinguishable from a
real hit, so it is fed to the generator as retrieved evidence.

Skip the -1 slots. The other backends are unaffected because they look the
content up by the payload the store returns, not by position.

Verified with a 3-vector index and top_k=5: before, FAISS returned
[0, 2, 1, -1, -1] and the backend produced
['DOC_A', 'DOC_C', 'DOC_B', 'DOC_C', 'DOC_C']; after, it produces
['DOC_A', 'DOC_C', 'DOC_B'].
@xhd0728 xhd0728 self-assigned this Sep 19, 2026
@xhd0728
xhd0728 merged commit a763d34 into OpenBMB:main Sep 19, 2026
3 checks passed
@xhd0728

xhd0728 commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the focused fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FAISS backend returns the last document for result slots it could not fill

2 participants