feat(core): add optional cross-encoder reranker stage to search

README.md

@@ -37,7 +37,8 @@ search.
 - **Local-first.** Plain text on your disk. Forever.
 - **Two-way.** AI and humans write to the same files; sync keeps them in step.
 - **A real knowledge graph.** Observations and wikilinks compound into context.
-- **Semantic search.** Find notes by meaning, not just keywords.
+- **Semantic search.** Find notes by meaning, not just keywords, with optional
+  cross-encoder reranking for higher-quality vector and hybrid results.
 - **MCP-native.** Works with every major AI client and IDE.
 - **Progressive tool discovery.** Every tool is tagged with behavior hints
   (read-only, destructive, idempotent) so agents pick the right tool on
@@ -357,6 +358,8 @@ Try a prompt:
 - **Semantic vector search.** Find notes by meaning, not just keywords.
   Hybrid full-text + vector ranking with FastEmbed embeddings, on SQLite or
   Postgres.
+- **Optional search reranking.** Rescore the strongest vector and hybrid
+  candidates with a local FastEmbed cross-encoder or a LiteLLM-backed provider.
 - **Schema system.** Infer, validate, and diff the structure of your
   knowledge base with `schema_infer`, `schema_validate`, `schema_diff`.
 - **Per-project cloud routing.** Route individual projects through the cloud
@@ -374,6 +377,36 @@ Try a prompt:
 
 Full [CHANGELOG](CHANGELOG.md) for v0.18 → v0.20.
 
+## Optional cross-encoder reranking
+
+Reranking adds a second relevance pass after vector or hybrid retrieval. It is
+disabled by default because it adds inference latency and, for the local
+provider, a first-run model download. Text, title, and permalink searches keep
+their existing ranking.
+
+Enable the default local FastEmbed reranker:
+
+```bash
+export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
+export BASIC_MEMORY_RERANKER_ENABLED=true
+```
+
+The default model is `jinaai/jina-reranker-v1-tiny-en`. To use a hosted
+reranker through LiteLLM instead:
+
+```bash
+export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
+export BASIC_MEMORY_RERANKER_ENABLED=true
+export BASIC_MEMORY_RERANKER_PROVIDER=litellm
+export BASIC_MEMORY_RERANKER_MODEL=cohere/rerank-v3.5
+export COHERE_API_KEY=...
+```
+
+The feature fails fast on invalid configuration and does not silently fall
+back to retrieval order when an enabled provider fails. See the
+[semantic search guide](docs/semantic-search.md#cross-encoder-reranking) for
+provider setup, all settings, tuning, pagination, and failure behavior.
+
 ## Why Basic Memory
 
 Most LLM conversations are ephemeral. You ask a question, get an answer, then

docs/semantic-search.md

@@ -322,6 +322,117 @@ Score-based fusion uses the formula `max(vec, fts) + bonus * min(vec, fts)` to p
 | `vector` | Conceptual queries, paraphrase matching, exploratory searches |
 | `hybrid` | General-purpose search combining precision and recall |
 
+## Cross-Encoder Reranking
+
+Reranking is an optional second stage for vector and hybrid search. Initial
+retrieval finds a candidate pool efficiently; a cross-encoder then reads each
+query and candidate together and replaces the leading candidates' scores with
+more precise relevance scores.
+
+Reranking is disabled by default. It requires semantic search and does not
+change `text`, `title`, or `permalink` search behavior.
+
+### Quick Start
+
+Use the default local FastEmbed provider:
+
+```bash
+export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
+export BASIC_MEMORY_RERANKER_ENABLED=true
+```
+
+The first reranked search downloads the model when it is not already cached,
+then loads `jinaai/jina-reranker-v1-tiny-en`. Later searches reuse the
+process-wide model instance and cache.
+
+To use a hosted reranker through LiteLLM:
+
+```bash
+export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
+export BASIC_MEMORY_RERANKER_ENABLED=true
+export BASIC_MEMORY_RERANKER_PROVIDER=litellm
+export BASIC_MEMORY_RERANKER_MODEL=cohere/rerank-v3.5
+export COHERE_API_KEY=...
+```
+
+LiteLLM model names must use explicit `provider/model` routing. Standard
+provider environment variables, such as `COHERE_API_KEY`, work normally. You
+can instead set `BASIC_MEMORY_RERANKER_API_KEY` to pass a credential directly
+to LiteLLM.
+
+### Providers
+
+| Provider | Runs | Default model | Tradeoff |
+|---|---|---|---|
+| `fastembed` | Locally with ONNX | `jinaai/jina-reranker-v1-tiny-en` | No API key or per-query cost; downloads a model on first use and adds local inference latency. |
+| `litellm` | Hosted or self-hosted API | No implicit hosted default | Supports LiteLLM rerank providers such as Cohere, Jina, and Voyage; adds network latency, provider cost, and credential requirements. |
+
+### Configuration Reference
+
+All settings use the `BASIC_MEMORY_` environment prefix:
+
+| Config Field | Environment Variable | Default | Description |
+|---|---|---|---|
+| `reranker_enabled` | `BASIC_MEMORY_RERANKER_ENABLED` | `false` | Enable reranking for vector and hybrid search. Requires semantic search. |
+| `reranker_provider` | `BASIC_MEMORY_RERANKER_PROVIDER` | `fastembed` | `fastembed` for a local ONNX cross-encoder or `litellm` for an API provider. |
+| `reranker_model` | `BASIC_MEMORY_RERANKER_MODEL` | `jinaai/jina-reranker-v1-tiny-en` | Model identifier. LiteLLM requires explicit `provider/model` routing. |
+| `reranker_candidates` | `BASIC_MEMORY_RERANKER_CANDIDATES` | `20` | Number of leading retrieval results rescored on every page. Larger values can improve recall but increase latency and provider usage. |
+| `reranker_max_document_chars` | `BASIC_MEMORY_RERANKER_MAX_DOCUMENT_CHARS` | `0` | Maximum characters sent per candidate. `0` sends the full matched text; a positive cap bounds latency and request size. |
+| `reranker_api_base` | `BASIC_MEMORY_RERANKER_API_BASE` | Unset | Optional custom endpoint for the LiteLLM provider. |
+| `reranker_api_key` | `BASIC_MEMORY_RERANKER_API_KEY` | Unset | Optional credential passed directly to LiteLLM. When unset, LiteLLM resolves provider credentials from its normal environment variables. |
+
+Configuration is validated at startup. Basic Memory rejects unsupported
+providers, blank models, unavailable FastEmbed models or dependencies, a
+LiteLLM model without a provider prefix, and reranking without semantic search.
+
+### Ranking and Pagination Behavior
+
+- Basic Memory reranks the same fixed leading candidate window on every
+  non-empty page. Results outside that window retain retrieval order and are
+  calibrated at or below the reranked floor. When that floor is zero, stable
+  prefix-before-tail ordering breaks the tie.
+- The matched chunk is placed before optional title context in the reranker
+  document. A positive document-character cap therefore preserves the passage
+  that caused the retrieval match.
+- Reranker scores replace retrieval scores for the reranked prefix and remain
+  within the public `[0, 1]` search score range.
+- Multi-project MCP search reranks inside each project's search service, then
+  merges the returned project-owned scores. It does not perform a second global
+  rerank in the MCP process.
+
+These rules keep independently fetched pages stable while still allowing
+larger requests to expand the untouched retrieval tail.
+
+### Failure Behavior
+
+An enabled reranker is part of the requested ranking contract, so Basic Memory
+does not silently return retrieval order when it fails:
+
+- Temporary provider, rate-limit, connection, timeout, or first-download
+  failures return HTTP 503. Retry the search after the provider recovers.
+- Malformed or incomplete provider responses return HTTP 502.
+- Authentication, model, dependency, and permanent configuration failures
+  surface directly instead of being treated as transient.
+- In multi-project search, a retryable failure from any project aborts the
+  aggregate page instead of returning a partial result set whose ordering could
+  change on retry.
+
+### Tuning
+
+Start with the defaults, then tune only if measurements justify it:
+
+- Increase `reranker_candidates` when relevant results enter the retrieval set
+  but remain outside the desired cutoff. This increases local inference time or
+  hosted provider usage.
+- Set `reranker_max_document_chars` to a positive value such as `1000` to
+  bound latency and hosted request size for long notes. The matched chunk comes
+  first, so a modest cap retains the strongest retrieval signal.
+- Keep reranking disabled when retrieval latency matters more than the
+  additional ranking pass.
+
+Changing reranker providers, models, candidate counts, or document caps does
+not change stored embeddings, so it does not require `bm reindex --embeddings`.
+
 ## The Reindex Command
 
 The `bm reindex` command rebuilds search indexes without dropping the database.

src/basic_memory/api/v2/routers/search_router.py

@@ -11,6 +11,8 @@
 import logfire
 from basic_memory.api.v2.utils import to_search_results
 from basic_memory.repository.semantic_errors import (
+    RerankProviderContractError,
+    RerankTransientError,
     SemanticDependenciesMissingError,
     SemanticSearchDisabledError,
 )
@@ -91,6 +93,15 @@ async def search(
             raise HTTPException(status_code=400, detail=str(exc)) from exc
         except SemanticDependenciesMissingError as exc:
             raise HTTPException(status_code=400, detail=str(exc)) from exc
+        except RerankTransientError as exc:
+            # Returning raw retrieval order would make pagination inconsistent with
+            # earlier reranked pages. Preserve ordering semantics and make the outage
+            # explicitly retryable instead.
+            raise HTTPException(status_code=503, detail=str(exc)) from exc
+        except RerankProviderContractError as exc:
+            # Upstream reranker returned a malformed response — an upstream fault, not a
+            # client error and not a transient outage (those map to a retryable 503).
+            raise HTTPException(status_code=502, detail=str(exc)) from exc
         except ValueError as exc:
             raise HTTPException(status_code=400, detail=str(exc)) from exc
 

src/basic_memory/config_models.py

@@ -56,6 +56,11 @@ class DatabaseBackend(str, Enum):
     POSTGRES = "postgres"
 
 
+# Default reranker model — a small local fastembed cross-encoder.
+# LiteLLM cannot route this name, so the litellm provider requires an explicit override.
+DEFAULT_FASTEMBED_RERANK_MODEL = "jinaai/jina-reranker-v1-tiny-en"
+
+
 def _default_semantic_search_enabled() -> bool:
     """Enable semantic search by default when required local semantic dependencies exist."""
     required_modules = ("fastembed", "sqlite_vec")
@@ -375,6 +380,47 @@ def __init__(self, **data: Any) -> None: ...
         "When unset, defaults to 'hybrid' if semantic search is enabled, otherwise 'text'.",
     )
 
+    # Reranker configuration (cross-encoder rescoring of the top vector/hybrid candidates)
+    reranker_enabled: bool = Field(
+        default=False,
+        description="Enable cross-encoder reranking of vector/hybrid search candidates. "
+        "Off by default: adds latency and a first-run model download. Requires semantic search.",
+    )
+    reranker_provider: str = Field(
+        default="fastembed",
+        description="Reranker provider: 'fastembed' (local ONNX cross-encoder) or 'litellm' "
+        "(Cohere/Jina/Voyage/etc. via API).",
+    )
+    reranker_model: str = Field(
+        default=DEFAULT_FASTEMBED_RERANK_MODEL,
+        description="Reranker model identifier. For litellm use the 'provider/model' form, "
+        "e.g. 'cohere/rerank-v3.5'.",
+    )
+    reranker_max_document_chars: int = Field(
+        default=0,
+        description="Max characters of each candidate's text passed to the cross-encoder. "
+        "0 (default) sends the full matched text — the model still truncates to its own token "
+        "limit. Set a positive cap (e.g. ~1000) to bound rerank latency on long notes; the "
+        "most-relevant matched chunk leads the text, so a modest cap keeps most of the signal.",
+        ge=0,
+    )
+    reranker_api_base: str | None = Field(
+        default=None,
+        description="Optional custom API base URL for the litellm reranker provider "
+        "(self-hosted OpenAI-compatible rerank endpoints).",
+    )
+    reranker_api_key: str | None = Field(
+        default=None,
+        description="Optional API key passed to the litellm reranker provider. When unset, "
+        "litellm resolves credentials from provider environment variables.",
+    )
+    reranker_candidates: int = Field(
+        default=20,
+        description="Number of top retrieval candidates to rescore with the reranker before "
+        "returning the requested page. Larger widens recall at the cost of latency.",
+        gt=0,
+    )
+
     # Database connection pool configuration (Postgres only)
     db_pool_size: int = Field(
         default=20,
@@ -865,6 +911,66 @@ def project_list(self) -> List[ProjectConfig]:  # pragma: no cover
             for name, entry in self.projects.items()
         ]
 
+    @model_validator(mode="after")
+    def validate_reranker_config(self) -> "BasicMemoryConfig":
+        """Fail fast on reranker configs that cannot work.
+
+        - Reranking runs only on vector/hybrid retrieval, so it needs semantic search;
+          accepting reranker_enabled=True without it would silently never rerank.
+        - FastEmbed exposes a finite registered model catalog, so reject typos while
+          loading config instead of deferring them to the first search request.
+        - The default model is a local fastembed cross-encoder that litellm cannot
+          route, so the litellm provider needs an explicit provider/model model id.
+        """
+        if not self.reranker_enabled:
+            return self
+        if not self.semantic_search_enabled:
+            raise ValueError(
+                "reranker_enabled=True requires semantic_search_enabled=True "
+                "(reranking operates on vector/hybrid search results)."
+            )
+        provider = self.reranker_provider.strip().lower()
+        if provider not in {"fastembed", "litellm"}:
+            raise ValueError("reranker_provider must be one of: fastembed, litellm")
+        model = self.reranker_model.strip()
+        if not model:
+            raise ValueError("reranker_model must not be blank when reranker_enabled=True")
+        self.reranker_provider = provider
+        self.reranker_model = model
+        if provider == "fastembed":
+            try:
+                from fastembed.rerank.cross_encoder import TextCrossEncoder
+            except ImportError as exc:
+                raise ValueError(
+                    "reranker_provider='fastembed' requires the fastembed package. "
+                    "Install/update basic-memory to include semantic dependencies."
+                ) from exc
+
+            supported_models = {
+                entry["model"] for entry in TextCrossEncoder.list_supported_models()
+            }
+            if model not in supported_models:
+                supported_names = ", ".join(sorted(supported_models))
+                raise ValueError(
+                    f"Unsupported FastEmbed reranker model {model!r}. "
+                    f"Supported models: {supported_names}"
+                )
+        if provider == "litellm" and model == DEFAULT_FASTEMBED_RERANK_MODEL:
+            raise ValueError(
+                "reranker_provider='litellm' requires an explicit reranker_model in "
+                "'provider/model' form (e.g. 'cohere/rerank-v3.5'); the default "
+                f"{DEFAULT_FASTEMBED_RERANK_MODEL!r} is a local fastembed model "
+                "litellm cannot route."
+            )
+        if provider == "litellm":
+            provider_name, separator, model_name = model.partition("/")
+            if not separator or not provider_name or not model_name:
+                raise ValueError(
+                    "reranker_provider='litellm' requires an explicit reranker_model in "
+                    "'provider/model' form (e.g. 'cohere/rerank-v3.5')."
+                )
+        return self
+
     @model_validator(mode="after")
     def ensure_project_paths_exists(self) -> "BasicMemoryConfig":  # pragma: no cover
         """Ensure project paths exist.