Skip to content

fix(sync): refresh the watcher's scope when codegraph.json or a .gitignore changes (#1590) - #1594

Open
colbymchenry wants to merge 1 commit into
mainfrom
fix/1590-watcher-scope-refresh
Open

fix(sync): refresh the watcher's scope when codegraph.json or a .gitignore changes (#1590)#1594
colbymchenry wants to merge 1 commit into
mainfrom
fix/1590-watcher-scope-refresh

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Fixes #1590.

What was wrong

The live file watcher built its scope matcher — built-in defaults + .gitignore + the codegraph.json exclude/include rules — once in start() and kept it for the watcher's lifetime. The MCP server is long-lived, so a codegraph.json created or edited after it started was invisible to the watcher, while codegraph sync (a fresh process with a fresh matcher) honoured it immediately. From the user's side: the CLI removed a newly excluded file, and the daemon re-indexed it a few seconds later, which reads as "exclude doesn't work". As the report points out, extensions on the very same config file was read live (its loader is mtime-cached), so two fields of one file behaved differently.

There was a second half to it. The watcher's scoped fast path hands the exact edited paths to sync, and that path stat'ed and re-parsed them without consulting the scope matcher at all — so the stale view of scope leaked straight into the index.

What this does

Watcher — rebuild on a scope change, then reconcile in full. An event for the root codegraph.json or .gitignore rebuilds the matcher, marks the next sync as a full reconcile, and schedules it. A scope change has no per-file events: newly excluded files must be removed from the index and newly included ones added, and only the scan-diff (which builds its own fresh matcher) knows which those are. Two ordering details are deliberate:

  • the two root files are checked before the matcher is consulted, so a user pattern that happens to cover them (*.json, .*) can't hide their own edits;
  • a nested .gitignore (an embedded child repo's own rules, or a subdirectory rule the git-backed scan honours) is checked after the matcher, so the thousands of package-local .gitignores an npm install writes under an ignored node_modules/ can never trigger a rebuild storm.

Rebuilding runs embedded-repo discovery (one git ls-files), which is fine per config edit and never happens per event. Replacing the field serves both watch strategies: the recursive handler and the per-directory shouldIgnoreDir walk read it on every call.

Scoped sync — re-check the paths it was handed. The orchestrator now runs scoped paths through the same scope matcher and source-extension gate the full walk applies. An out-of-scope path is treated as absent: removed if tracked, never parsed on trust. The matcher is memoized on the mtimes of the two root files it derives from (two stats per sync while nothing changed), so the scoped path keeps skipping O(repo) work — paying embedded-repo discovery per sync would defeat its whole point.

Tests

  • watcher.test.ts — a codegraph.json edit schedules a full sync, after which an edit inside the newly excluded tree is dropped by the live matcher (not pending, no sync) while an in-scope edit still syncs scoped; a root .gitignore edit behaves the same; a nested .gitignore forces a full sync; a .gitignore under node_modules/ schedules nothing; dropping the exclude again readmits the tree.
  • sync.test.ts — end-to-end through CodeGraph: a scoped sync of a path that codegraph.json now excludes removes it (filesRemoved: 1, nothing parsed — the symbol added to the file never appears), stays out on a repeat, and is re-added through the same scoped path once the exclude is dropped.
  • All five new tests fail on main; the node_modules guard passes both ways as expected.
  • Full suite: 189 files, 3184 passed / 9 skipped.
  • CLI half of the issue's repro (init with exclude, edit the config + the file, codegraph sync): the newly excluded file is removed and its new symbol never enters the index.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK

…gnore changes (#1590)

The live file watcher built its scope matcher (defaults + .gitignore +
codegraph.json exclude/include) once in start() and kept it for the
watcher's lifetime. In the long-lived MCP daemon that meant a
codegraph.json created or edited after startup was invisible to the
watcher while `codegraph sync` — a fresh process with a fresh matcher —
honoured it immediately: the CLI removed a newly excluded file and the
watcher re-indexed it seconds later. `extensions` on the same config
file WAS read live (mtime-cached loader), so two fields of one file
disagreed.

Two layers:

- watcher: an event for the root `codegraph.json` / `.gitignore` (checked
  before the matcher, so an exclude covering them can't hide their own
  edits) or for a nested `.gitignore` inside the current scope (checked
  after it, so `npm install` writing package-local `.gitignore`s under an
  ignored `node_modules/` can't trigger a rebuild storm) rebuilds the
  matcher and forces the next sync to be a FULL reconcile — a scope
  change has no per-file events, so only the scan-diff can find the
  files the new scope drops or admits.
- orchestrator: a scoped sync now passes the paths it was handed through
  the same scope matcher + source-extension gate the full walk applies,
  memoized on the mtimes of the two root files it derives from so the
  scoped fast path keeps skipping O(repo) work. An out-of-scope path is
  absent, hence removed if tracked and never parsed on trust.

Tests cover the config edit (full sync, then the excluded file dropped
by the live matcher and an in-scope edit still syncing scoped), root and
nested .gitignore edits, the node_modules churn guard, re-admission when
the exclude is removed, and the orchestrator-side gate end-to-end
(scoped sync removes a newly excluded file and re-adds it once the
exclude is dropped). All five new tests fail on main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Watcher keeps a stale codegraph.json exclude matcher for the daemon's lifetime — sync and the live watcher disagree after a config change

1 participant