fix(value): [OBE-10732] bound the depth a VRL program can nest a Value to - #16
Open
JuanMantica45 wants to merge 1 commit into
Open
fix(value): [OBE-10732] bound the depth a VRL program can nest a Value to#16JuanMantica45 wants to merge 1 commit into
JuanMantica45 wants to merge 1 commit into
Conversation
…e to `Value`'s `Clone`, `PartialEq`, `Hash` and drop glue are all structurally recursive, and a VRL program can build an arbitrarily deep `Value` without a deeply-nested program: `v = push([], v)` inside `for_each` adds one level per iteration. Past a few thousand levels the next traversal walks off the end of the native stack and the process dies of a SIGSEGV that Rust cannot catch, taking every co-tenant pipeline with it. None of those traversals can report an error — they return `Self`, `bool`, a hash, and nothing — so a deep `Value` cannot be handled safely once it exists. It has to not exist. This adds `MAX_VALUE_DEPTH` and rejects at `push`, the one operation that grows nesting a level at a time. Measured overflow depth per traversal on a 2 MiB stack (tokio's default, which Vector takes since it never calls `thread_stack_size`): Display 3,294 ~625 B/level PartialEq 9,415 ~223 B/level Clone 10,983 ~190 B/level Serialize 32,951 ~64 B/level drop glue 43,932 ~48 B/level 512 is derived from the worst of these: 512 levels of `Display` costs ~320 KiB, 6.4x headroom inside 2 MiB. It is also 4x every other cap in this crate and 4x `serde_json`'s parser limit, so it cannot plausibly reject real data. The measurements correct two claims in the ticket that would have sent this the wrong way. Drop is the *most* tolerant traversal, not the critical one, and it is unreachable: `Variable::resolve` clones the accumulator every iteration, so `Clone` caps construction at ~10,983, four times below drop's limit. And `serde_json` has no `impl Drop for Value` to copy — checked against 1.0.140, there is no `impl Drop` in the crate at all. Its actual defence is a depth limit in its *parser*: it bounds construction, exactly as this does. That matters because `impl Drop for Value` would have been a breaking change — Rust forbids moving out of a type that implements `Drop`, which would break `into_object()`, `into_array()` and 52 destructuring sites in this crate alone, before counting Vector. `Drop`, `Clone`, `PartialEq` and `Hash` are untouched here, and there is no new dependency. `depth_exceeds` walks an explicit heap worklist rather than recursing, so the check cannot overflow the stack it exists to protect, and it stops as soon as the limit is passed — O(limit) for the shape being guarded, not O(size). The probes that produced every number above ship in examples/. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OBE-10732 was closed by mistake and reopened — PR #9's description carries the correction that it was never fixed there.
The problem
Value'sClone,PartialEq,Hashand drop glue are all structurally recursive, and a program can build an arbitrarily deepValuewithout a deeply-nested program:v = push([], v)insidefor_eachadds a level per iteration. The next traversal then walks off the native stack — SIGSEGV, not a catchable panic, taking every co-tenant pipeline down.None of those traversals can report an error (
Self,bool, a hash, nothing), so a deepValuecannot be handled safely once it exists. It has to not exist.Measurements
Max depth surviving, by thread stack size. Linear in stack size, and the ordering is stable at every size, so bytes/level is a property of the code:
Display::fmtPartialEq::eqClone::cloneSerializeMAX_VALUE_DEPTH = 512follows from the worst of these: 512 levels ofDisplaycosts ~320 KiB, 6.4x headroom inside the 2 MiB tokio gives Vector's workers (Vector never callsthread_stack_size, so the default applies). It is also 4x every other cap in this crate and 4xserde_json's parser limit, so it cannot plausibly reject real data.Two claims in the ticket are wrong, and it matters
Drop is not the critical path — it is the most tolerant, and it is unreachable. The ticket says an iterative
Dropis "mandatory" and there is "no way for the embedder to defend without" one. Drop tolerates 43,932 levels, 4x more thanClone— andCloneis the ceiling on construction, becauseVariable::resolveclones the accumulator every iteration. You cannot build deep enough to break drop from VRL.serde_jsonhas noimpl Drop for Valueto copy. The ticket cites "the same patternserde_json::Valueuses — seeimpl Drop for Value". Checked againstserde_json-1.0.140: there is noimpl Dropanywhere in the crate. Its real defence is the 128-depth limit in its parser (de.rs:38) — it bounds construction, which is what this PR does.This matters because
impl Drop for Valuewould have been a breaking change: Rust forbids moving out of a type that implementsDrop, which breaksinto_object(),into_array()and 52 destructuring sites in this crate alone, before counting Vector, which re-exportsValueas its event type.The reachable crash is
PartialEq, whose limit (9,415) sits just belowClone's (10,983): build to ~10,000, whichClonesurvives, thenif v == v. Confirmed — at 10,000 iterations build-only lives andeqdies.What changed
MAX_VALUE_DEPTHanddepth_exceedsin a newsrc/value/depth.rs. The check walks an explicit heap worklist rather than recursing, so it cannot overflow the stack it exists to protect, and it stops as soon as the limit is passed — O(limit) for the shape being guarded, not O(size of value).pushrejects an item that would put the result over the cap.Drop,Clone,PartialEq,Hashuntouched. No new dependency. No public API change.Test plan
cargo test --lib: 1767 passed, 0 failed.vrl_depth_probe eq 10000, which killed the process before this change, now returns a clean runtime error. 400 iterations still succeed, 600 are rejected — the boundary lands at 512 as designed.examples/depth_probe.rsandexamples/vrl_depth_probe.rsship with this PR and reproduce every number above.🤖 Generated with Claude Code