Skip to content

feat(skills): add the PolyCSS agent skill package and evaluation - #84

Open
apresmoi wants to merge 30 commits into
mainfrom
feat/agent-skill-package
Open

feat(skills): add the PolyCSS agent skill package and evaluation#84
apresmoi wants to merge 30 commits into
mainfrom
feat/agent-skill-package

Conversation

@apresmoi

@apresmoi apresmoi commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add @layoutit/polycss-skills, a zero-dependency npx installer that drops the PolyCSS agent skill into .claude/skills or .agents/skills
  • own the skill content in packages/skills/skill/ and mirror it to website/public/skill/ with pnpm sync:skill; pnpm check:skill fails on drift in CI
  • split the single skill.md into an entry point plus 13 per-topic reference docs (authoring invariants, scenes, shapes, loading, lighting, shadows, textures, controls, animation, performance, three parity, troubleshooting, api index)
  • track installs with a content-hash manifest so an upgrade refreshes shipped docs, drops removed ones, and refuses to overwrite files the user edited
  • fix the receiver-shadow lift default: it was 0.001 against a documented 0.05, so castShadow + receiveShadow emitted paths that darkened nothing in all three renderers
  • add eval/skill/, which runs real agent CLIs in isolated workspaces outside the repo and grades what the scene paints in Chromium, across three tracks: polycss (skill installed), polycss-noskill (the control — same library and contract, skill withheld), and three (an external baseline, not a control)

Results

Six tasks, graded on visual criteria only. Tracks are scored independently and never diffed against each other. These numbers predate the polycss-noskill control track, so they compare the skill against an external Three.js baseline rather than isolating the skill's effect.

agent PolyCSS Three.js
Claude Code 29/29 29/29
Codex 29/29 28/29
Grok 28/29 29/29

Grok's remaining miss is a PolyCSS defect, not a skill gap: shadow.lift is in world units but the depth conflict it must win is resolved in device pixels, so below zoom ≈ 1 the shadow collapses into the receiver and disappears. The default camera zoom is 0.65. The limitation is documented in shadows.md and troubleshooting.md with the workaround; a projection-aware fix changes documented semantics across three renderers and is left for a separate decision.

Scores are evidence that the skill is sufficient, not proof it was the only source used — the workspaces sit outside the repo but are not sandboxed. eval/skill/README.md states the limits.

Verification

  • pnpm test
  • pnpm build:packages
  • pnpm build:website
  • pnpm test:scripts
  • pnpm check:readmes
  • pnpm check:skill
  • pnpm eval:skill --agent oracle --track all — 29/29 on both tracks
  • pnpm eval:selftest — 13/13 mutations caught

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant