Skip to content

fix(#742): match keywords as regex, matching the documented tutorial - #753

Open
bosd wants to merge 1 commit into
masterfrom
fix/742-regex-keyword-matching
Open

fix(#742): match keywords as regex, matching the documented tutorial#753
bosd wants to merge 1 commit into
masterfrom
fix/742-regex-keyword-matching

Conversation

@bosd

@bosd bosd commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #742.

The template-authoring tutorial has always documented keywords and exclude_keywords as regex patterns — e.g. "Company US and Company\s+US both work". But InvoiceTemplate.matches_input used a plain in substring test, so any keyword that leaned on a regex metachar silently under-matched.

An audit of the bundled templates shows ~16 already write regex-shaped keywords that were effectively broken:

  • (?i)Accor, (?i)Digital\sRiver
  • BE\s0673\s923
  • Express\s+\w+
  • Wystawił\(a\)
  • NL8152[.]54[.]295[.]B01
  • …and more.

Change

  • Route both keywords and exclude_keywords checks through a new module-local _keyword_matches helper that calls _regex.search (the compile-once cache used everywhere else in extract/).
  • If a keyword doesn't compile as valid regex, fall back to a literal substring check and DEBUG-log the fallback — so a stray unescaped [ or ( in a simple-string keyword like Company [Ltd keeps matching literally instead of breaking.
  • Re-export _regex.error (the active engine's error type — re.error by default, regex.error under INVOICE2DATA_REGEX_ENGINE=regex) so the fallback doesn't need to know which engine is active.

Compatibility

  • Plain-string keywords with no metacharacters (Acme Corp) behave exactly as before — regex.search(\"Acme Corp\", text) is the same as \"Acme Corp\" in text.
  • Keywords that already were regex (bundled templates) now actually work.
  • Keywords that were string-but-look-like-regex (rare — Company [Ltd with an unclosed [) still match via the literal fallback.

Test plan

  • New unit tests for regex \s+, (?i), alternation, exclude-as-regex, and invalid-regex-falls-back-to-substring.
  • Existing test suite passes (only the pre-existing env-related failures remain — test_gvision, test_copy needing pdftotext/tesseract, and QualityHosting.pdf cascade, all red on master too).
  • test_default_templates_are_loaded still loads every bundled template (nothing rejected as invalid regex).

https://claude.ai/code/session_01YLnqiMRXGtQQsM7TGRQx6v

The template tutorial has always documented ``keywords`` and
``exclude_keywords`` as regex patterns (``Company\s+US``, ``(?i)Accor``,
``Wystawił\(a\)`` ...), and ~16 bundled templates already wrote them
that way -- but ``matches_input`` used plain ``in`` substring matching,
so those regex-shaped keywords silently under-matched. Route both
keyword checks through :func:`regex.search` (via the module-local
``_regex`` compile-once cache) so they honour the documented contract.

A pattern that doesn't compile as valid regex falls back to a literal
substring check with a DEBUG log line, so a stray metachar in a
simple-string keyword like ``Company [Ltd`` keeps matching literally
instead of breaking. Re-exports ``_regex.error`` so the fallback
does not need to know which engine is active.

Closes #742.

Claude-Session: https://claude.ai/code/session_01YLnqiMRXGtQQsM7TGRQx6v
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mismatch between docs and code

1 participant