Skip to content

Add browser initial load optimizer eval - #98919

Draft
wbinnssmith wants to merge 1 commit into
codex/browser-initial-load-skillfrom
codex/browser-initial-load-eval
Draft

wbinnssmith wants to merge 1 commit into
codex/browser-initial-load-skillfrom
codex/browser-initial-load-eval

Conversation

@wbinnssmith

@wbinnssmith wbinnssmith commented Sep 18, 2026

Copy link
Copy Markdown
Member

What?

Add agent-059-browser-initial-load-optimizer, an eval for the public initial-load optimization skill.

Why?

The skill combines analyzer evidence with source inspection and behavior-preserving changes. This eval verifies that agents actually follow that workflow rather than applying familiar bundle advice without measuring it.

Fixture

The / route deliberately contains four independently scoreable problems:

  • CodeMirror is statically imported even though it renders only after a button click.
  • Display-only Markdown and syntax highlighting run inside a broad Client Component.
  • A local legacy-widget pins Lodash 4.17.20 while the app uses 4.17.21; both are in the initial client graph.
  • The route page has an unnecessarily broad 'use client' boundary.

The prompt describes only a slow initial route and behavior to preserve. It does not name the skill, analyzer, packages, or fixes.

Scoring

Deterministic checks require a conditional dynamic editor import, no client-side display Markdown stack, a narrower page boundary, and at most one installed Lodash version. LLM criteria verify visible behavior and require a measured analyzer baseline/query/regeneration workflow with the correct scope-vs-byte success signal.

The fixture is configured for baseline, bundled-docs, and local-skill treatments with a 30-minute timeout.

Verification

  • Starting fixture: RED on every planted static condition.
  • Representative safe solution: GREEN and production-buildable outside the monorepo, matching the eval runner's sandbox layout.
  • pnpm eval agent-059-browser-initial-load-optimizer --dry: passed; schedules all three treatments.
  • Full invocation: all variants skipped because this environment has no AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN; no agent execution failed.
  • Prettier, ESLint, JSON, Next package build, and diff checks pass.

Co-authored-by: Will Binns-Smith <755844+wbinnssmith@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Failing CI jobs

Commit: 8172ea4 | About building and testing Next.js

@wbinnssmith
wbinnssmith added this pull request to stack #98901 September 18, 2026 23:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant