Skip to content

fix(value): [OBE-10732] bound the depth a VRL program can nest a Value to - #16

Open
JuanMantica45 wants to merge 1 commit into
Sentinel-One:mainfrom
JuanMantica45:obe-10732-value-depth-cap
Open

fix(value): [OBE-10732] bound the depth a VRL program can nest a Value to#16
JuanMantica45 wants to merge 1 commit into
Sentinel-One:mainfrom
JuanMantica45:obe-10732-value-depth-cap

Conversation

@JuanMantica45

Copy link
Copy Markdown
Contributor

OBE-10732 was closed by mistake and reopened — PR #9's description carries the correction that it was never fixed there.

The problem

Value's Clone, PartialEq, Hash and drop glue are all structurally recursive, and a program can build an arbitrarily deep Value without a deeply-nested program: v = push([], v) inside for_each adds a level per iteration. The next traversal then walks off the native stack — SIGSEGV, not a catchable panic, taking every co-tenant pipeline down.

None of those traversals can report an error (Self, bool, a hash, nothing), so a deep Value cannot be handled safely once it exists. It has to not exist.

Measurements

Max depth surviving, by thread stack size. Linear in stack size, and the ordering is stable at every size, so bytes/level is a property of the code:

Traversal 512 KB 2 MB 8 MB bytes/level Can error?
Display::fmt 837 3,294 13,125 ~625 yes
PartialEq::eq 2,393 9,415 37,502 ~223 no
Clone::clone 2,791 10,983 43,751 ~190 no
Serialize 8,375 32,951 131,255 ~64 yes
drop glue 11,164 43,932 175,004 ~48 no

MAX_VALUE_DEPTH = 512 follows from the worst of these: 512 levels of Display costs ~320 KiB, 6.4x headroom inside the 2 MiB tokio gives Vector's workers (Vector never calls thread_stack_size, so the default applies). It is also 4x every other cap in this crate and 4x serde_json's parser limit, so it cannot plausibly reject real data.

Two claims in the ticket are wrong, and it matters

Drop is not the critical path — it is the most tolerant, and it is unreachable. The ticket says an iterative Drop is "mandatory" and there is "no way for the embedder to defend without" one. Drop tolerates 43,932 levels, 4x more than Clone — and Clone is the ceiling on construction, because Variable::resolve clones the accumulator every iteration. You cannot build deep enough to break drop from VRL.

serde_json has no impl Drop for Value to copy. The ticket cites "the same pattern serde_json::Value uses — see impl Drop for Value". Checked against serde_json-1.0.140: there is no impl Drop anywhere in the crate. Its real defence is the 128-depth limit in its parser (de.rs:38) — it bounds construction, which is what this PR does.

This matters because impl Drop for Value would have been a breaking change: Rust forbids moving out of a type that implements Drop, which breaks into_object(), into_array() and 52 destructuring sites in this crate alone, before counting Vector, which re-exports Value as its event type.

The reachable crash is PartialEq, whose limit (9,415) sits just below Clone's (10,983): build to ~10,000, which Clone survives, then if v == v. Confirmed — at 10,000 iterations build-only lives and eq dies.

What changed

  • MAX_VALUE_DEPTH and depth_exceeds in a new src/value/depth.rs. The check walks an explicit heap worklist rather than recursing, so it cannot overflow the stack it exists to protect, and it stops as soon as the limit is passed — O(limit) for the shape being guarded, not O(size of value).
  • push rejects an item that would put the result over the cap.
  • Drop, Clone, PartialEq, Hash untouched. No new dependency. No public API change.

Test plan

  • cargo test --lib: 1767 passed, 0 failed.
  • 11 new tests: exact boundary in both directions, depth nested in an object, depth hidden behind 1,000 shallow siblings (the check must not be evaded by breadth), and a 100,000-level value asserting the checker itself does not overflow.
  • End-to-end, on a 2 MiB stack: vrl_depth_probe eq 10000, which killed the process before this change, now returns a clean runtime error. 400 iterations still succeed, 600 are rejected — the boundary lands at 512 as designed.
  • examples/depth_probe.rs and examples/vrl_depth_probe.rs ship with this PR and reproduce every number above.

🤖 Generated with Claude Code

…e to

`Value`'s `Clone`, `PartialEq`, `Hash` and drop glue are all structurally
recursive, and a VRL program can build an arbitrarily deep `Value` without a
deeply-nested program: `v = push([], v)` inside `for_each` adds one level per
iteration. Past a few thousand levels the next traversal walks off the end of
the native stack and the process dies of a SIGSEGV that Rust cannot catch,
taking every co-tenant pipeline with it.

None of those traversals can report an error — they return `Self`, `bool`, a
hash, and nothing — so a deep `Value` cannot be handled safely once it exists.
It has to not exist. This adds `MAX_VALUE_DEPTH` and rejects at `push`, the one
operation that grows nesting a level at a time.

Measured overflow depth per traversal on a 2 MiB stack (tokio's default, which
Vector takes since it never calls `thread_stack_size`):

  Display       3,294   ~625 B/level
  PartialEq     9,415   ~223 B/level
  Clone        10,983   ~190 B/level
  Serialize    32,951    ~64 B/level
  drop glue    43,932    ~48 B/level

512 is derived from the worst of these: 512 levels of `Display` costs ~320 KiB,
6.4x headroom inside 2 MiB. It is also 4x every other cap in this crate and 4x
`serde_json`'s parser limit, so it cannot plausibly reject real data.

The measurements correct two claims in the ticket that would have sent this the
wrong way. Drop is the *most* tolerant traversal, not the critical one, and it
is unreachable: `Variable::resolve` clones the accumulator every iteration, so
`Clone` caps construction at ~10,983, four times below drop's limit. And
`serde_json` has no `impl Drop for Value` to copy — checked against 1.0.140,
there is no `impl Drop` in the crate at all. Its actual defence is a depth limit
in its *parser*: it bounds construction, exactly as this does.

That matters because `impl Drop for Value` would have been a breaking change —
Rust forbids moving out of a type that implements `Drop`, which would break
`into_object()`, `into_array()` and 52 destructuring sites in this crate alone,
before counting Vector. `Drop`, `Clone`, `PartialEq` and `Hash` are untouched
here, and there is no new dependency.

`depth_exceeds` walks an explicit heap worklist rather than recursing, so the
check cannot overflow the stack it exists to protect, and it stops as soon as
the limit is passed — O(limit) for the shape being guarded, not O(size).

The probes that produced every number above ship in examples/.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant