feat(extract): tree-sitter-based Dart extraction with source_location line anchors - #2114
feat(extract): tree-sitter-based Dart extraction with source_location line anchors#2114cornwe19 wants to merge 2 commits into
Conversation
|
@safishamsi FYI - this is a more recent attempt at what it looks like #406 was attempting back in April. Would love to see either one of them merged when possible. |
… line anchors
Dart was the one mainstream language whose extractor emitted
source_location: None on every node and edge — a ~520-line regex
extractor predating the generic tree-sitter framework — so Dart symbols
couldn't be jumped to or read by line range downstream.
extract_dart now parses with the community tree-sitter-dart grammar
(optional [dart] extra, prebuilt wheels for all platforms):
declarations (classes, mixins, enums, extension types, extensions,
typedefs), imports/exports, `part of` redirection, annotations, and
functions come from the AST with L{n} anchors on nodes and edges. The
Flutter-framework heuristics (Bloc/Riverpod/navigation) and Dart 3
pattern-destructured variables — which the young grammar still parses
with ERROR nodes — stay on the proven regexes, now line-anchored via
match offsets and scoped by the AST (declarations inside string
literals and multi-line named-parameter lists no longer produce
phantom nodes).
Without the extra installed, extraction falls back to the previous
regex extractor, preserved verbatim as _extract_dart_regex — the same
optional-grammar pattern as extract_pascal (Graphify-Labs#781).
Validated on a large production Flutter codebase (2,780 files):
100% of locally-defined nodes and all edges anchored, anchor spot-check
100% (2,547 sampled symbols on their anchored line), node counts within
0.5% of the regex baseline, and extraction slightly faster than the
regex path. All existing dart tests pass on both paths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
f2c7dac to
efd02b1
Compare
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This PR appears to be primarily a changelog rollback and accompanying code/test changes. Based on the diff, it removes several changelog entries (versions 0.9.26 through 0.9.29) from CHANGELOG.md, suggesting a reversion or restructuring of released version notes. The changed symbols span a wide surface area including extractors (C#, SQL, JS module resolution, C++, XML), hooks (merge driver registration, Windows path handling), path/detection utilities, watch/incremental logic, export (Obsidian), call-flow HTML, and their corresponding tests, along with README and changelog files for earlier versions. Note: the provided diff was truncated, so I can only summarize the changelog removals and the list of touched symbols; the actual implementation changes behind those symbols aren't visible in the excerpt.
No blocking issues surfaced.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 4549 functions depend on the 2929 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
_make_id()— 166 callers, 1 callees - worse:
extract_vue()— 10 callers, 7 callees - worse:
extract_astro()— 6 callers, 6 callees - worse:
extract_dart()— 13 callers, 2 callees - new:
_apply_annotations()— 4 callers, 4 callees - worse:
_resolve_tsconfig_alias()— 5 callers, 3 callees - worse:
extract_svelte()— 2 callers, 6 callees - new:
_sweep_class_body()— 1 callers, 6 callees - …and 3 more
Verification — 4549 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 4454 function(s) in the blast radius were not formally verified this run
· 6 grounded finding(s) anchored inline below; 5 more finding(s) on lines outside this diff (see the check run).
| } | ||
|
|
||
|
|
||
| def extract_dart(path: Path) -> dict: |
There was a problem hiding this comment.
extract_dart()
13 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| add_edge(owner_nid, bloc_nid, "references", line, context="bloc_lookup") | ||
| return handle | ||
|
|
||
| def _sweep_class_body(body_node: Any, class_nid: str) -> None: |
There was a problem hiding this comment.
_sweep_class_body()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| (_DART_TYPED_LOOKUP_RE, _typed_lookup_handler(class_nid)), | ||
| ]) | ||
|
|
||
| def _sweep_function_body(body_node: Any, fn_nid: str) -> None: |
There was a problem hiding this comment.
_sweep_function_body()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| (_DART_NAV_OBJECT_RE, on_nav_object), | ||
| ]) | ||
|
|
||
| def _apply_annotations(target_nid: str, target_name: str, is_class: bool, annotations: list[Any]) -> None: |
There was a problem hiding this comment.
_apply_annotations()
high coupling complexity (Ca·Ce = 16).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| if member.type != "annotation": | ||
| pending_annotations = [] | ||
|
|
||
| def _handle_class_family(node: Any, annotations: list[Any]) -> None: |
There was a problem hiding this comment.
_handle_class_family()
fans out to 9 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| _sweep_class_body(body_node, class_nid) | ||
| _handle_members(body_node, class_name) | ||
|
|
||
| def _handle_extension(node: Any, annotations: list[Any]) -> None: |
There was a problem hiding this comment.
_handle_extension()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
There was a problem hiding this comment.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This PR reworks the Dart extractor (graphify/extractors/dart.py) to use tree-sitter as the primary extraction path—pulling declarations, imports/exports, part-of redirection, annotations, and functions from the AST with source_location line anchors—while keeping the existing regex-based logic for Flutter-framework heuristics (Bloc/Riverpod/navigation) and variable extraction as sweeps over the source. When the optional tree-sitter-dart dependency isn't installed or parsing fails, extraction falls back to the preserved regex implementation. The README is updated to document that .dart files use tree-sitter via the graphifyy[dart] extra with regex fallback. The test suite for Dart is also updated, including additions covering line anchors, language features, generic syntax extraction, and advanced Dart features.
Worth a look
- byte offsets vs character offsets mismatch in _line_of —
graphify/extractors/dart.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 104 functions depend on the 103 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
extract_dart()— 13 callers, 2 callees - new:
_apply_annotations()— 4 callers, 4 callees - new:
_sweep_class_body()— 1 callers, 6 callees - new:
_sweep_function_body()— 1 callers, 6 callees - new:
_handle_class_family()— 0 callers, 9 callees - new:
_handle_extension()— 0 callers, 6 callees
Verification — 104 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 104 function(s) in the blast radius were not formally verified this run
· 6 grounded finding(s) anchored inline below.
| } | ||
|
|
||
|
|
||
| def extract_dart(path: Path) -> dict: |
There was a problem hiding this comment.
extract_dart()
13 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| add_edge(owner_nid, bloc_nid, "references", line, context="bloc_lookup") | ||
| return handle | ||
|
|
||
| def _sweep_class_body(body_node: Any, class_nid: str) -> None: |
There was a problem hiding this comment.
_sweep_class_body()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| (_DART_TYPED_LOOKUP_RE, _typed_lookup_handler(class_nid)), | ||
| ]) | ||
|
|
||
| def _sweep_function_body(body_node: Any, fn_nid: str) -> None: |
There was a problem hiding this comment.
_sweep_function_body()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| (_DART_NAV_OBJECT_RE, on_nav_object), | ||
| ]) | ||
|
|
||
| def _apply_annotations(target_nid: str, target_name: str, is_class: bool, annotations: list[Any]) -> None: |
There was a problem hiding this comment.
_apply_annotations()
high coupling complexity (Ca·Ce = 16).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| if member.type != "annotation": | ||
| pending_annotations = [] | ||
|
|
||
| def _handle_class_family(node: Any, annotations: list[Any]) -> None: |
There was a problem hiding this comment.
_handle_class_family()
fans out to 9 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| _sweep_class_body(body_node, class_nid) | ||
| _handle_members(body_node, class_name) | ||
|
|
||
| def _handle_extension(node: Any, annotations: list[Any]) -> None: |
There was a problem hiding this comment.
_handle_extension()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
What
Dart is currently the one mainstream language whose extractor emits
source_location: Noneon every node and edge —extract_dartis a ~520-line regex extractor predating the generic tree-sitter framework, so Dart symbols can't be jumped to or read by line range downstream, unlike Java/Kotlin/Swift/etc.This PR makes tree-sitter the primary Dart path, using the community
tree-sitter-dartgrammar as a new optional[dart]extra (prebuilt wheels for win/macOS/Linux, x86+arm):part ofredirection, annotations, and functions/factory constructors all come from the AST withL{n}anchors on nodes and edges._extract_dart_regex) — the same optional-grammar pattern asextract_pascal(feat(extract): add Pascal/Delphi and Lazarus IDE support #781). No new required dependency.Fixes along the way
The AST also closes two regex false-positive classes:
'''...''') no longer produce phantom class/function nodes.required String id,sits at the 0–2-space indent the variable regex scans) no longer leak parameters as variable nodes, including a case that produced labels containing newlines.Validation
tests/test_dart.pyexpectations pass on both paths (grammar installed and absent); new tests cover anchor stamping,part ofredirection with anchors, the extension family (named/anonymous/generic), named/optional parameters, factory vs named constructors, and generators..dartfiles): 100% of locally-defined nodes and all edges anchored, 100% anchor spot-check (2,547 sampled symbols found on their anchored line), node counts within 0.5% of the regex baseline (the delta is the string-literal phantoms the AST correctly drops), and the tree-sitter path ran slightly faster than the regex path (~5s vs ~5s for the whole repo, same order).uv.lockregenerated withuv lock(addstree-sitter-dartto theall/dartextras; the surrounding marker churn is current uv re-normalizing environment markers).🤖 Generated with Claude Code