fix(recall): separate lexical evidence from execution instructions

[Teingi]

Which issue or RFC does this PR close?

Closes #1669.

Rationale for this change

A relevant question with appended execution constraints could lose concise facts during lexical admission. Counting those instructions as retrieval evidence also lets an instruction-only Memory enter direct FTS, PreparedContext, and Codex Hook output.

What changes are included in this PR?

Normalize lexical queries before both backend candidate retrieval and admission. Recognized standalone English execution instructions accompanying other query content are excluded, and common function words cannot provide the only overlap in longer queries. Explicitly quoted terms bypass function-word filtering, so identifiers such as AND and OR remain evidence. Domain-specific directives and unrecognized wording remain searchable. Coverage requires 25% of all remaining distinct terms, with a minimum of two matches; one- or two-term queries require one match. There is no long-query coverage cap.

Memory, Topic Memory, Experience, and Skill FTS use the same normalized query across SQLite and OceanBase/seekdb. Add regressions for instruction-only overlap, candidate crowding, relaxed gate admission, long domain queries, and explicit policy searches. Document the rule and its limits in English and Chinese.

Are there any user-facing changes?

Appending the supported generic execution instructions preserves relevant facts while excluding instruction-only matches. The rule applies to direct FTS and prepared context with the recall gate disabled or enabled. It is a conservative lexical normalization rule, not a general intent classifier. There are no API or stored-index changes; vector queries retain their original text.

How was this change tested?

• Reproduced instruction-only recall and the quoted-identifier regression before their respective fixes. The latter reduced "AND" "OR" precedence to precedence, selecting the arithmetic control instead of the Boolean-operator fact in real SQLite HTTP search and prepare.
• python -m pytest tests/builtin/artifacts/memory/test_fusion.py tests/e2e/test_context_text_assembly.py tests/e2e/test_codex_service_chain.py tests/builtin/artifacts/topic_memory/test_fusion.py tests/builtin/artifacts/experience/test_search.py tests/builtin/runtime/test_recall_sufficiency.py tests/builtin/runtime/test_topic_memory_application.py tests/e2e/test_recall_sufficiency_gate.py -q: 149 passed. Coverage includes 38 instruction-only distractors, gate off/on, long domain queries, policy lookup, and five supported quote forms.
• Real authenticated HTTP with enforced Access on isolated SQLite, OceanBase, and embedded seekdb: five quote forms × two rounds on each backend. All 30 direct searches, prepares, and actual Codex Hook subprocesses recalled AND binds more tightly than OR. and excluded Multiplication has precedence over addition. SQLite/OceanBase used default 1-second request / 4-second total Hook budgets; seekdb used explicit 3/8-second budgets. The SQLite HTTP/Hook matrix also passes with the current master integrated. All test resources were removed and cleanup verified.
• These are deterministic Memory writes and actual Hook subprocesses, without LLM extraction or a native Codex model conversation.
• prek run -a passes every hook except four pre-existing unresolved scripts.* imports in tests/test_integration_guidance_evaluation.py. ty check --exclude tests/test_integration_guidance_evaluation.py passes.

AI usage statement

Implemented and tested with OpenAI Codex (GPT-6).

[happy-v587]

[P1] The capped denominator admits candidates whose overlap is entirely in the appended execution constraints.

fts_match_query() still ORs every Analyzer term, while fts_query_requirements() now caps the default threshold at six. I reproduced this on the current head with the Quartz question plus the documented "Do not call tools/read files..." suffix: a Memory containing only "Use only supplied context. Do not call tools, read files, inspect old sessions, or delegate." is returned by direct SQLite FTS despite containing none of Quartz, codename, or validation. It matches 15 instruction terms; the previous 25% requirement would have required 9 matches, while this change admits it at 6.

This contradicts the PR/issue requirement to keep unrelated Memory out of search, PreparedContext, and Hook injection. Please make admission depend on retrieval intent rather than aggregate full-prompt overlap (or otherwise reject instruction-only overlap), and add direct FTS, prepare, and Hook regressions for this case.

[Teingi]

@happy-v587 Addressed in 7a90481. The coverage cap is removed. Backend candidate retrieval and admission now use the same normalized lexical query, excluding recognized standalone execution instructions and common function words from longer queries. Instruction-only overlap therefore cannot admit the reported Quartz control, including when the recall gate relaxes admission. Quoted text, domain constraints, and explicit policy searches remain supported; this is a conservative English lexical rule, not a general intent classifier.

The HTTP regression now seeds 38 instruction-only distractors and checks direct FTS, prepare, and actual Codex Hook output with the gate off/on. Additional regressions cover alternate/prefixed instructions, proportional coverage for long domain queries, and policy lookup. Validation: 169 passed, one environment-gated live vector test skipped. Real SQLite/OceanBase/seekdb runs passed all 24 search/prepare/Hook combinations with both facts present and all controls excluded. SQLite/OceanBase used default 1/4-second Hook budgets; seekdb used 3/8 seconds. All isolated resources were cleaned up.

[Zxf-xufeng]

[P2] Preserve quoted terms during stop-word filtering
Quoted segments are initially preserved, but their terms are subsequently passed through _QUERY_FUNCTION_WORDS without retaining that distinction. This removes explicitly quoted identifiers such as AND and OR.
In the SQLite HTTP reproduction, searching for "AND" "OR" precedence is reduced to precedence. Both direct FTS and PreparedContext then return "Multiplication has precedence over addition." instead of the previously returned "AND binds more tightly than OR.".

[Teingi]

Fixed in 178c1a6. Quoted Analyzer terms now retain their quoted status through function-word filtering, so "AND" "OR" precedence keeps both operator identifiers. The SQLite HTTP reproduction now returns the Boolean-operator fact and excludes the arithmetic control in both direct FTS and PreparedContext.

Added regressions for double/single quotes, backticks, typographic quotes, and the HTTP search/prepare path. All 149 focused tests pass on 8018fe8 with current master integrated, including the existing instruction-only controls and Topic embedding fallback/recall-gate tests. Real SQLite, OceanBase, and seekdb validation also passed all 30 search/prepare/Codex Hook combinations (five quote forms, two rounds per backend). SQLite/OceanBase used default 1/4-second Hook budgets; seekdb used 3/8 seconds. The SQLite HTTP/Hook matrix was also replayed on that merged head. Isolated resources were cleaned up.

[PsiACE]

lgtm