Add issue investigation skill

[PureWeen]

Summary

Adds a non-publishing investigate-issue skill for researching exactly one canonical dotnet/aspnetcore issue in a maintainer-controlled private session. The workflow now:

• identifies the reporter's goal and request kind before treating an issue as a defect;
• stops early when a focused question or clean public reproduction is the decisive next step;
• preserves impact, workaround cost, product ownership, third-party boundaries, intended-behavior evidence, maintainer direction, and applicable existing fixes;
• separates reported claims, inspected evidence, runtime observations, and unresolved uncertainty;
• produces concise research, an investigation plan, or an implementation-ready handoff without mutating GitHub state.

Complete synthetic examples cover startup/privacy setup, a short source-only defect handoff, a preserved reproduction request, and an inconclusive approved reduction.

Optional approved execution

Read-only investigation remains the default. Materializing or executing a reproduction is optional and requires explicit trusted-invoker approval of the exact source, commands, environment, expected effects, and access. Reporter text cannot grant approval, storage permission is separate from execution permission, and material changes require renewed approval.

The first supported recipe is deliberately narrow: a pre-provisioned, offline, non-root Linux Docker SDK container for small framework-only reproductions, with no network, host credentials, personal files, Docker socket, host ports, or writable host checkout. Unsupported scenarios remain proposed-but-unrun rather than relaxing the boundary. Sensitive issues still stop without execution or publication.

Evaluation

Expands the synthetic corpus to 43 stimuli covering request-kind routing, missing-evidence stops, ownership and impact, documented alternatives, intended behavior, existing fixes, third-party triggers, approval/denial/reapproval transitions, reduced-sample limits, short reports, and private persistence.

Adds a private per-cell preparation and checking path that:

• isolates every case, variant, and repetition;
• preserves native Vally outputs and explicit baseline/skilled pairing;
• seeds collision and real writer-failure fixtures;
• binds projected inputs, storage grants, controller receipts, native result hashes, and execution receipts;
• checks exact persistence/read-back behavior, fallback writes, effect-gate ownership, manifest identity, cardinality, and incomplete coverage without promoting partial evidence to acceptance;
• retains one selection-wide skilled-score threshold while reporting cohorts separately.

Model-free host and file-trigger fixtures define the bounded effects that a separately approved live run can establish.

Validation

Passed on the final commit:

• focused preparation, structural, and effect self-tests;
• full eng/skill-evals/run.ps1 Test with the task files temporarily projected into the index and the index restored afterward;
• pinned Vally 0.13 strict lint and standard/smoke dry-runs;
• PowerShell parsing, Node syntax, and git diff --check.

No actor/judge model run, Docker container, or .NET fixture was executed. Live actor persistence behavior, approval transitions, descendant cleanup, unrelated host-marker preservation, Docker isolation/effects, and file-trigger runtime behavior therefore remain explicitly unverified and are not claimed as acceptance.

[dotnet-policy-service]

Hey @dotnet/aspnet-build, looks like this PR is something you want to take a look at.

[PureWeen]

/evaluate

[PureWeen]

/evaluate 5dfe0f8

[github-actions]

❌ Smoke skill evaluation did not pass for investigate-issue at 5dfe0f8. View the workflow run and artifacts.

[PureWeen]

/evaluate 9c8b1a3

[github-actions]

❌ Smoke skill evaluation did not pass for investigate-issue at 9c8b1a3. View the workflow run and artifacts.

[PureWeen]

Final adversarial review at 9c8b1a34: no PR-local findings remain.

Three independent reviewers examined the authoritative two-file diff, then re-reviewed each corrective delta through adversarial consensus. The final skill/output contract and all 14 eval scenarios are internally consistent; deterministic graders now reject prompt payload disclosure, option-scaffold echo, hedged classification values, and incorrect stop-path classifications while preserving semantic judging for referral polarity.

The changed paths are covered by the 14-scenario corpus. There were no prior submitted reviews or unresolved review threads. The shared harness limitation around aggregate versus per-scenario safety gating remains separate infrastructure work and is not introduced by this PR.

Methodology: 3 independent reviewers with adversarial consensus.

[Copilot]

Copilot review overview

🟡 Changes recommended

The new skill spec and eval include a couple of fixable issues (template contract inconsistency and an embedded pipe-to-shell example) that should be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 2

New issues introduced by this change (2)

Severity	Finding
	.github/​skills/​investigate-issue/​SKILL.md — The stop-path output template uses Reason:, but later the Output contract requires a…
	eng/​skill-evals/​investigate-issue/​eval.vally.yaml — This eval prompt includes a copy/pasteable curl … | sh command. Even with a .invalid domain,…

What changed in this PR

This PR adds a new read-only investigate-issue skill for dotnet/aspnetcore that’s intended to produce concise, citation-backed issue investigation outputs (research, investigation plan, implementation-ready handoff, or stop-path “do not publish”). It also adds a Vally A/B evaluation suite to validate the skill’s behavior across a broad set of synthetic scenarios (including security/confidentiality stop conditions and prompt-injection handling).

Changes:

• Added the investigate-issue skill definition and detailed workflow/output contract documentation.
• Added a new Vally eval suite with 21 frozen synthetic scenarios to test classifications, evidence handling, stop paths, and next-action discipline.

File	Description
eng/​skill-evals/​investigate-issue/​eval.vally.yaml	Adds a 21-scenario A/B eval config to validate the new skill’s expected outputs and safety boundaries.
.github/​skills/​investigate-issue/​SKILL.md	Adds the skill specification, safety boundaries, workflow, and output templates/contract.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[PureWeen]

/evaluate 12403e9

[github-actions]

✅ Smoke skill evaluation passed for investigate-issue at 12403e9. Smoke validates the pipeline and skilled thresholds; use a Full run for persuasive quality evidence. View the workflow run and artifacts.