Add browser initial load optimizer eval - #98919
Draft
wbinnssmith wants to merge 1 commit into
Draft
wbinnssmith wants to merge 1 commit into
wbinnssmith wants to merge 1 commit into
Conversation
Co-authored-by: Will Binns-Smith <755844+wbinnssmith@users.noreply.github.com>
Contributor
Failing CI jobsCommit: 8172ea4 | About building and testing Next.js |
wbinnssmith
added this pull request to stack #98901
September 18, 2026 23:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What?
Add
agent-059-browser-initial-load-optimizer, an eval for the public initial-load optimization skill.Why?
The skill combines analyzer evidence with source inspection and behavior-preserving changes. This eval verifies that agents actually follow that workflow rather than applying familiar bundle advice without measuring it.
Fixture
The
/route deliberately contains four independently scoreable problems:legacy-widgetpins Lodash 4.17.20 while the app uses 4.17.21; both are in the initial client graph.'use client'boundary.The prompt describes only a slow initial route and behavior to preserve. It does not name the skill, analyzer, packages, or fixes.
Scoring
Deterministic checks require a conditional dynamic editor import, no client-side display Markdown stack, a narrower page boundary, and at most one installed Lodash version. LLM criteria verify visible behavior and require a measured analyzer baseline/query/regeneration workflow with the correct scope-vs-byte success signal.
The fixture is configured for baseline, bundled-docs, and local-skill treatments with a 30-minute timeout.
Verification
pnpm eval agent-059-browser-initial-load-optimizer --dry: passed; schedules all three treatments.AI_GATEWAY_API_KEYorVERCEL_OIDC_TOKEN; no agent execution failed.