Skip to content

feat(fuzz): add native Jev-guided transactions - #16936

Draft
decofe wants to merge 13 commits into
masterfrom
centaur/jev-fuzz-guidance-1789770679
Draft

decofe wants to merge 13 commits into
masterfrom
centaur/jev-fuzz-guidance-1789770679

Conversation

@decofe

@decofe decofe commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Add opt-in native Jev guidance to invariant transaction generation. forge test --invariant-tx-generator jev keeps RNG for most fresh transactions while Jev Choices selects bounded ABI-derived actions and stateful scenarios.

The generator acts as an automatically derived invariant handler. It maps scenarios onto persistent primary/counterparty/third-party roles, calls compatible view functions against current local state, and can use an exact-type result as a state-changing argument. Random and dictionary-backed ABI generation remain local and dominant; Jev never supplies executable code, addresses, calldata, or concrete values.

The scheduler supports large ABIs by keeping the full grammar local while paginating model-visible choices. It interleaves direct and scenario productions, rotates through the candidate frontier, withholds recent choices, and batches up to eight actions or scenarios per request. Missing credentials and provider failures fall back to RNG.

This revision also derives bounded lending/vault lifecycles from common ABI verbs: select actor and asset, enable collateral and supply/deposit in candidate-valid orders, borrow, apply relevant oracle or configuration transitions, then repay, withdraw, redeem, mint fees, liquidate, or check a property. Each lifecycle has ABI-random and dictionary-backed variants. Conventional actors 0x10000, 0x20000, and 0x30000 preserve compatibility with pre-funded fuzz harnesses. In Jev mode, a single ABI bool return of false is treated as an invariant failure, enabling Echidna-style property harnesses without handwritten assertion adapters; RNG mode retains Foundry's existing behavior.

The opt-in discloses eligible signatures, compatible-view signatures/types, local target ordinals, role names, and bounded outcomes to the fixed OpenRouter endpoint. Keys, source, concrete addresses/arguments, calldata, storage, dictionary contents, and view results are not sent.

The no-handler smoke fixture requires one stable actor to deposit, then call withdrawExact(withdrawableBalanceOf(actor)) using a getter on another contract. With the recorded seed, Jev found it in one run and two calls using one request; matched RNG missed after 16 runs and 1,600 calls.

The Recon comparison runs all five pinned SCFuzzBench targets with matched settings. Jev preserves the cataloged Origin Dollar (4/12) and Drips (1/1) bug hits and runs natively on Superform rather than falling back. On four paired 30-second Aave campaigns, the previous scheduler improved median features by 53.6%.

The ABI-lifecycle revision produced the first real Aave win. In a fresh matched seed-0x4, one-worker, 180-second pair, RNG found only the canary in 12,400 calls. Jev found four distinct bugs in 8,300 calls: the canary plus totalBorrowedLessThanSupplied_v0 at 32 seconds, mintFeeShares at 105 seconds, and shouldNotBecomeLiquidatable at 163 seconds. That one smoke campaign exceeds the published plain-Foundry 24-hour median of three broken invariants. It does not yet establish Echidna/Medusa parity: both publish a median of ten, so repeated release-pinned long trials remain the next gate. Full methodology and results are in benches/jev/SCFUZZBENCH.md.

Validation: 13 targeted Jev tests pass; all 198 foundry-evm library tests pass; cargo check -p forge, nightly formatting, and nightly Clippy across the touched crates pass. The benchmark includes model latency.

Configuration documentation is in foundry-rs/book#2099.

Prompted by: @gakonst

Connect a typed Jev decision batch to fixed synthetic Forge input strategies. Record decisions for replay and compare broad sampling, boundary heuristics, and model guidance with isolated failures and clean controls.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
gakonst and others added 2 commits September 18, 2026 22:36
Preserve all 360 paired synthetic trial measurements beside the result interpretation so detection counts and first-failure inputs remain reviewable.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Generate bounded Solidity handlers from typed Choice distributions, preserving runtime reads and sender binding alongside existing corpus-guided invariant execution. Add exact live-response replay, local correctness oracles, and execution tests without claiming a findings improvement.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
@decofe decofe changed the title test(bench): add Jev guidance experiment feat(bench): add Jev-guided sequence grammars Sep 18, 2026
0xalpharush and others added 3 commits September 18, 2026 22:54
Derive bounded typed productions from eligible ABIs and request online Choice batches during fresh transaction generation. Feed actual call outcomes into subsequent decisions while preserving local corpus mutation and replay. Keep RNG as the default and bound remote requests with a visible fallback. Add native unit and configuration tests; full build validation is pending.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Retire the superseded JavaScript experiments from the active tree, retaining their evidence in earlier commits. Add a native CLI fallback regression and make feedback explicitly identify its matching grammar productions.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Log only accepted batch counts, elapsed time, and numeric production IDs so native integration checks can distinguish actual guidance from fallback without exposing provider bodies or credentials.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
@decofe decofe changed the title feat(bench): add Jev-guided sequence grammars feat(fuzz): add native Jev transaction selection Sep 18, 2026
Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
@decofe decofe changed the title feat(fuzz): add native Jev transaction selection feat(fuzz): add native Jev-guided transactions Sep 18, 2026
gakonst and others added 6 commits September 19, 2026 00:40
Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Restore the validated typed-grammar generator so Jev choices can materialize Solidity shortcut handlers with live state reads and deterministic replay. Update its integration test for Forge pulse output and align the missing-credential fallback diagnostic.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants