Iterative repair loop: 1:1 conversion, --postfix N, --strip-chrome - #3
Open
MatMatt wants to merge 14 commits into
Open
Iterative repair loop: 1:1 conversion, --postfix N, --strip-chrome#3MatMatt wants to merge 14 commits into
MatMatt wants to merge 14 commits into
Conversation
- Merge postfix + review into single iterative repair loop - --postfix N (default 3 iterations), --no-postfix to disable - --review flag removed (vision repair built into loop) - 1:1 conversion by default, --strip-chrome for optional chrome removal - repair_report.md with per-iteration summary table - 439 tests passing
Append 3 new pages (sections 10-13) to pdf2md_edgecases.pdf covering 6 previously untested scenarios: - Clickable hyperlink annotations (PDF link annotations) - \DATA / <sub> / <sup> / <br> raw patterns (_sanitise_for_typst) - 130171 display math + ambiguous notation (math false-positive guards) - Running header variants per page (chrome.py detection) - Page-N-of-M footer pattern (lint.py chrome-leak guard) - Nested image paths + doubled URL schemes (lint.py) Regeneration: python3 src/pdf2md/tests/gen_edgecase_supplement.py
Remove 1990-2018_PUM_v1_short.pdf (real CLMS PUM snippet) and port the 2 pagefilter regression tests to pdf2md_edgecases.pdf. The synthetic edgecases PDF covers both scenarios: - recall: pages 9 (aerial photo raster) and 11 (vector figure) detected - skip rate: 7 text-only pages correctly skipped
- --main-llm <slug> replaces --model (--model kept as deprecated alias) - --figure-llm <slug> for Phase 1 figure detection - pdf2md --setup now fetches model catalogue from OpenRouter API and presents a numbered pick list (multimodal first, sorted by price) for both conversion LLM and figure-detection LLM - Config saved to ~/.pdf2md/config.json under 'model' and 'figure_llm'
# Conflicts: # src/pdf2md/app_cli.py # src/pdf2md/postfix.py
…lint gate
Root cause of Quarto/Typst emitting #box(image(""))
instead of #box(image("img.png")): the .qmd's markdown image reference
was itself nested inside another image wrapper ()), a
known artifact from phase2/rescue/tablefix. Quarto's typst writer does
not detect this hazard — it just stringifies the whole nested markdown
as the literal image path.
lint.py already had _unwrap_nested_images() to fix exactly this, but it
only ran in Phase 4.6 (the *final* emit-time gate), which executes AFTER
render (Phase 3), verify (Phase 4), and postfix (Phase 4.5). So the
render always saw the un-sanitized .qmd.
Fix: run the same deterministic sanitize pass right before render
(Phase 2.9), in addition to keeping the final Phase 4.6 gate (postfix
can still reintroduce the hazard after render, so both passes are
needed).
Reproduced by isolating the pattern with:
quarto render test.qmd --to typst --metadata-file src/pdf2md/render_assets/_typst.yml -M keep-typ:true
Confirmed against /tmp/eea-out real-run output, which had this exact
artifact for 2 figures (img-1174619926e23b52307165b105d52675.png,
img-ed92374ebbd81963432e40b81edd8e25.png) that had been nested-image
wrapped upstream.
… and auto-repair - Add YAML validity check to normalize_frontmatter (invalid YAML blocks are discarded and replaced with a proper skeleton) - Add title: '' and subtitle: '' fallbacks for PR-gate compliance - Fix date fallback to always emit today's date as last resort - Add per-phase model reporting (cover/detect/convert/repair) to verify report - Rename --cover-model to --cover-llm for consistency with --main-llm/--figure-llm - Add 3 new verify checks: yaml_boundary (frontmatter leaks), duplication (near-duplicate paragraphs), artifacts (Office ref errors) - Add artifact auto-repair to the deterministic postfix pass - Fix duplication check to exclude tbl-caption boilerplate (false positives)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges postfix + review into a single iterative repair loop, switches to 1:1 conversion by default, and adds optional chrome stripping.
Changes
Tests
439 passed, 1 skipped.