fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581)

[colbymchenry]

Fixes #1581.

What was wrong

codegraph init / codegraph index died with Segmentation fault — the whole CLI
process, not a parse worker — on a C/C++ file with very deep brace nesting (llvm's
clang/test/Parser/parser_overflow.c, 16,384 nested {). The reporter's diagnosis is
exactly right: tree-sitter's parser is iterative, so the file parses fine, and then the
native kernel's recursive walker (visit_node → visit_for_calls_and_structure → …,
one frame per AST level) overflowed the thread's stack. A native overflow can't be caught
the way a wasm abort can, and a parse worker is a thread of the codegraph process, so
the SIGSEGV took the entire indexer down — no message, no per-file fallback, no partial
index.

Two things made "just give the worker a bigger stack" the wrong fix:

• it only moves the cliff — reproduced here: the reporter's 16,384-deep file kills a
  default 4 MiB worker (rc=132 on macOS / 139 on Linux), and a 100k-deep file kills the
  8 MiB main thread too;
• the walkers are shared by every kernel-routed language (20 of them), and each has
  several recursion points with different frame sizes, so no single stack size is a
  provable bound.

Meanwhile the wasm path already handles this shape gracefully: its JS walker catches its
own RangeError per file and stores a partial result with a parse_error. The kernel
just needed a way to get there instead of dying.

What this does

The kernel guards its own recursion against the calling thread's real stack bounds and
defers a too-deep file to wasm — the same defer: routing signal it already uses for
files with parse errors, which src/extraction/kernel/index.ts treats as "take the wasm
path for this file", silently.

• codegraph-kernel/src/stack.rs: per-thread stack bounds from the OS, computed once per
  thread and cached — glibc/musl pthread_getattr_np + pthread_attr_getstack, macOS
  pthread_get_stackaddr_np + pthread_get_stacksize_np, Win32
  GetCurrentThreadStackLimits (a hand-declared kernel32 extern; no windows-sys).
  exhausted() is one thread-local load and one compare: true once the stack pointer is
  within a 256 KiB red zone of the limit, and it latches a flag. Where the OS can't report
  bounds it falls back to a fixed 1 MiB descent budget measured from the entry stack
  pointer — safe on anything from Node's 4 MiB worker default up. So the guard is exact on
  the 4 MiB worker, the 8 MiB main thread, and any resourceLimits.stackSizeMb alike.
• stack_guard!() (defined in lib.rs) is the first statement of every recursive walker
  function — all 150 self-recursive or on-cycle functions across the 15 walker modules,
  found by script (every cycle in the call graph, not just direct self-calls). It returns
  Default::default() ((), false, None, "") so an exhausted walk simply stops
  descending; a hook returning false sends its caller down the generic child walk, whose
  own guard returns at once.
• extract_file runs the whole walk under stack::run_guarded: if the flag is set
  afterwards the (truncated) result is discarded and replaced by
  defer: nesting too deep for the native walker — wasm recovery handles it.
• parse-pool.ts: a comment at new Worker(scriptPath) records why there is deliberately
  no resourceLimits.stackSizeMb bump.
• No new crates beyond libc as a direct unix dependency (already in Cargo.lock
  transitively). No wire/ABI change.

Net effect for the reporter's repo: deep.c goes to the wasm path, lands as
function foo plus a recorded parse warning, and the other 31,607 files index normally.
CODEGRAPH_KERNEL=0 and the exclude workaround are no longer needed.

Tests

Rust unit tests (cargo test, 21 passed — 7 new in stack.rs): the walkers for
C, C++, Rust, TypeScript and Python are driven on a 1 MiB thread (a quarter of Node's
worker default) with 30k-deep nesting and must return defer: instead of crashing;
shallow files are untouched; the latch resets between runs; the OS bounds are sane on the
main thread and describe a small thread's own stack.

__tests__/kernel-deep-nesting.test.ts (new, 8 tests — skips without a staged .node,
fails under CODEGRAPH_KERNEL_EXPECT=1 if the kernel is missing, like the other kernel
suites):

• every default-routed language (all 20) survives a 60k-deep expression on the main thread
  — clean result or the wasm fallback's partial result, never a crash;
• the reporter's exact 16,384-brace C file is indexed (partial) on the main thread;
• 200-deep expressions in every language still take the kernel path clean (the guard never
  trips on normal code);
• inside a default-sized 4 MiB worker_threads Worker through dist/: the reporter's
  deep.c and a 60k-deep expression in every language come back deferred with exit 0,
  and a normal file still extracts natively;
• end-to-end through the built CLI: codegraph init on a repo holding deep.c + ok.c
  exits 0 and records both files, with both functions.

Existing kernel suites: all 15 (kernel-*-parity, kernel-scaffold,
kernel-retry-materialize, kernel-grammar-parity) pass unchanged, 147 tests — the guard
never fires on the parity fixtures.

Reporter's probes (one.js from the issue, default 4 MiB worker, this build):
deep.c → deferred, exitCode=0 (was rc=132/139); deep100k.c → deferred, exitCode=0.
Main thread: deep.c / deep100k.c → wasm partial with
Parse error: Maximum call stack size exceeded; a 6,000-term binary expression and a
3,000-branch else if chain stay on the kernel path with clean results.

Perf (same dist/, only the .node swapped via CODEGRAPH_KERNEL_PATH; interleaved
main/new ×3, codegraph init, macOS arm64):

repo	main (median)	guarded (median)	nodes / edges
express (141 files)	0.60 s (0.58–0.65)	0.61 s (0.58–0.61)	1,084 / identical
redis (786 C/H files)	4.44 s (4.39–4.66)	4.49 s (4.41–4.70)	19,942 / 76,446 identical

Within run-to-run noise, as expected for one TLS load + compare per recursion entry.

Linux (Docker, node:22-bookworm, kernel built in-container, docker run --rm --init) —
the reporter's platform and the glibc pthread_getattr_np bounds path:

=== platform ===
Linux efe3cc86947b 6.12.54-linuxkit #1 SMP Tue Nov  4 21:21:47 UTC 2025 aarch64 GNU/Linux
v22.22.3
-rwxr-xr-x 1 root root 35332288 Aug 22 18:02 codegraph-kernel/prebuilds/linux-arm64/codegraph-kernel.node

=== reporter repro (issue #1581): 16,384-brace deep.c, codegraph init ===
│
└  Done

init exit code: 0
  file: deep.c
  file: deep100k.c
  file: ok.c
  function: add
  function: bar
  function: foo

=== worker probe: kernel raw extract in a default 4 MiB worker ===
deep.c: deferred
deep.c: worker exitCode=0
deep100k.c: deferred
deep100k.c: worker exitCode=0
ok.c: kernel nodes=2
ok.c: worker exitCode=0

=== cargo test stack:: (glibc pthread_getattr_np bounds path) ===
test stack::tests::os_bounds_are_sane_on_this_platform ... ok
test stack::tests::small_stack_reports_its_own_bounds ... ok
test stack::tests::normal_files_are_untouched_by_the_guard ... ok
test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok
test stack::tests::latch_resets_between_runs ... ok
test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.23s

=== vitest: kernel-deep-nesting + kernel-scaffold ===
✓ __tests__/kernel-scaffold.test.ts (10 tests) 30ms
✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 36989ms
Test Files  2 passed (2)
Tests  18 passed (18)

(The pre-fix crash was reproduced on macOS — rc=132 in a default worker, rc=139 on the main thread at 100k depth — not re-run inside this container; the reporter's Linux x86_64 trace is the SIGSEGV form of the same overflow.)

Windows (Parallels ARM64 VM, MSVC 14.44, cargo 1.97, kernel built on the VM,
GetCurrentThreadStackLimits path):

head: cbf8485 fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581)
=== cargo build --release (win32-arm64) ===
    Finished `release` profile [optimized] target(s) in 2m 04s
staged: 35086848 bytes
=== cargo test (stack guard unit tests) ===
test stack::tests::normal_files_are_untouched_by_the_guard ... ok
test stack::tests::os_bounds_are_sane_on_this_platform ... ok
test stack::tests::small_stack_reports_its_own_bounds ... ok
test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok
test stack::tests::latch_resets_between_runs ... ok
test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.49s
=== reporter repro: codegraph init on a 16,384-brace deep.c ===
└  Done
init exit code: 0
=== vitest: deep-nesting + scaffold (CODEGRAPH_KERNEL_EXPECT=1) ===
✓ __tests__/kernel-scaffold.test.ts (10 tests) 55ms
✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 67239ms
   ✓ every default-routed language survives a 60k-deep expression on the main thread 52801ms
   ✓ inside a default-sized (4 MiB) parse worker, through dist/ > defers a 60k-deep expression in every default-routed language 13050ms
   ✓ end-to-end: codegraph init on a repo holding the deep file > exits 0 and records deep.c alongside the normal files 936ms
Test Files  2 passed (2)
Tests  18 passed (18)

(The end-to-end test is what reads the Windows index back through node:sqlite — files = deep.c, ok.c; functions add, foo.)

Full npm test on this branch (macOS arm64, kernel staged): 190 files passed, 3,185 tests passed, 10 skipped, 0 failed.

Clippy note: cargo clippy on the current toolchain (1.92) reports 18 pre-existing lints
(manual_contains, unnecessary_to_owned, …) in walker code this PR only touched by
inserting guard lines; none are in stack.rs/lib.rs. Left alone to keep the diff
reviewable.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK