Add conformance-server: host conformance test harness (addresses #674)#693
Open
qchuchu wants to merge 1 commit into
Open
Add conformance-server: host conformance test harness (addresses #674)#693qchuchu wants to merge 1 commit into
qchuchu wants to merge 1 commit into
Conversation
A WPT-style host-conformance test runner for the MCP Apps spec (2026-01-26): ships a ui:// page that renders inside the host's sandboxed iframe, drives the postMessage/JSON-RPC bridge, asserts host behaviour against the spec, and reports PASS/FAIL right in the iframe. First step toward modelcontextprotocol#674 (public MCP Apps platform tests). Mirrors the basic-server-react layout/toolchain and is registered alongside the other examples in both E2E specs and both publish workflows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new examples/conformance-server package that serves a ui:// in-iframe conformance runner (WPT-style) for validating MCP Apps host behavior against the 2026-01-26 spec, and wires it into existing E2E/grid and publishing workflows.
Changes:
- Introduces the conformance server example (MCP server + React runner UI + in-view test harness/catalogue).
- Registers the new server in E2E server lists and grid screenshot generation.
- Includes the example in publish workflows and updates the lockfile/workspaces wiring.
Reviewed changes
Copilot reviewed 17 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/e2e/servers.spec.ts | Adds conformance-server to the set of E2E-tested servers. |
| tests/e2e/generate-grid-screenshots.spec.ts | Includes conformance-server in the grid screenshot generation list. |
| package-lock.json | Adds the new example workspace and updates dependency lock state. |
| examples/conformance-server/vite.config.ts | Vite build config for single-file runner bundling. |
| examples/conformance-server/tsconfig.server.json | Server-side TS config for emitting declaration files. |
| examples/conformance-server/tsconfig.json | Client/runner TS config for React/Vite. |
| examples/conformance-server/src/vite-env.d.ts | Vite client type reference. |
| examples/conformance-server/src/tests.ts | In-view conformance test catalogue (automatic + manual/human-verified cases). |
| examples/conformance-server/src/testharness.ts | WPT-style harness (mcp_test, assertions, timeouts, manual interaction plumbing). |
| examples/conformance-server/src/style.css | Runner UI styling (grid + interaction prompts). |
| examples/conformance-server/src/mcp-app.tsx | React runner UI that executes the suite and renders results/prompts. |
| examples/conformance-server/server.ts | MCP server registering the runner ui:// resource and fixture tools. |
| examples/conformance-server/README.md | Usage docs and host-requirement catalogue. |
| examples/conformance-server/package.json | New workspace package definition and build/dev scripts. |
| examples/conformance-server/mcp-app.html | Runner HTML entrypoint for Vite. |
| examples/conformance-server/main.ts | HTTP/stdio entrypoint for running the server. |
| examples/conformance-server/.gitignore | Ignores node_modules/ and dist/. |
| .github/workflows/publish.yml | Adds the new example to the publish matrix. |
| .github/workflows/npm-publish.yml | Adds conformance-server to the npm publish matrix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+114
to
+129
| requestInteraction: (req) => | ||
| new Promise<boolean>((resolve) => { | ||
| const settle = (v: boolean) => { | ||
| setInteraction(null); | ||
| resolve(v); | ||
| }; | ||
| // "await" mode: pass automatically the moment the test's signal settles | ||
| // (e.g. the host-context-changed notification arrives). | ||
| if (req.kind === "await" && req.signal) { | ||
| req.signal.then( | ||
| () => settle(true), | ||
| () => settle(false), | ||
| ); | ||
| } | ||
| setInteraction({ req, resolve: settle }); | ||
| }), |
Comment on lines
+2
to
+4
| * Entry point for running the MCP server. | ||
| * Run with: npx @modelcontextprotocol/server-basic-react | ||
| * Or: node dist/index.js [--stdio] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
examples/conformance-server, a host-conformance test harness for theMCP Apps spec (
2026-01-26), modeled on web-platform-tests. It ships a singleui://runner that renders inside the host's sandboxed iframe, drives thepostMessage/JSON-RPC bridge, asserts the host's behaviour against the spec, and
reports PASS/FAIL right in the iframe.
This is a concrete first step toward #674 ("Set up public MCP Apps platform
tests"): the in-iframe runner that a public wpt.fyi-style grid would later
aggregate across hosts. Opening per the examples policy; happy to adjust scope.
What's included
examples/conformance-server/: React runner plus MCP server, mirroring thebasic-server-reactlayout and toolchain (vite + bun build, stdio/HTTP entry).normative requirement in the spec, each tagged with its RFC-2119 clause, a
vantage (where it's observable: in-view / host / server), and status.
24 of 45 implemented today (in-view automatic checks plus 6 human-in-the-loop).
debug-servereverywhere it belongs: both E2E specs(
servers.spec.ts,generate-grid-screenshots.spec.ts) and both publishworkflows (
pkg-pr-newplus npm matrix).Testing
npm run buildplusnpm run --workspace examples/conformance-server build: cleannpm run prettier: cleanrendering);
servers.spec.tspasses for the new server.Checklist
npm run prettier)🤖 Generated with Claude Code