Skip to content

DOC Document attack outcomes and scenario resiliency - #2296

Open
romanlutz wants to merge 18 commits into
microsoft:mainfrom
romanlutz:romanlutz-document-attack-outcomes
Open

DOC Document attack outcomes and scenario resiliency#2296
romanlutz wants to merge 18 commits into
microsoft:mainfrom
romanlutz:romanlutz-document-attack-outcomes

Conversation

@romanlutz

@romanlutz romanlutz commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • document the two independent axes of attack execution health and objective outcome in the Scenario resiliency guide
  • clarify that handled model refusals are persistable, scorable completed responses rather than execution failures
  • add a Mermaid flow covering target responses, retries, scoring, completed and incomplete objectives, partial failures, and Scenario run states
  • expand undersized Mermaid diagrams into the right margin with MyST col-page-right and refine layouts to prevent clipping, crossings, and overlaps

Before and after

Screenshots use the same 1366px browser viewport. Before is main at 630061a80; after is this PR at 24dea2a5b.

Page Before After
Framework architecture Framework architecture before Framework architecture after
Multi-turn attack flow Multi-turn flow before Multi-turn flow after
Scoring architecture blog Scoring architecture before Scoring architecture after
Memory schema Memory schema before Memory schema after
Scorer hierarchy Scorer hierarchy before Scorer hierarchy after
Combining scorers Combining scorers before Combining scorers after
Attack technique selection Attack techniques before Attack techniques after
Scenario outcomes and resiliency Scenario resiliency before Scenario outcomes and resiliency after

Validation

  • uv run jupyter-book build --html
  • Jupytext round-trip checks for synchronized notebooks
  • file-scoped pre-commit hooks
  • rendered Mermaid clipping and geometry audit

Copilot AI added 18 commits July 27, 2026 18:42
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a008124c-220e-491c-9ea1-2144867ba53d
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a008124c-220e-491c-9ea1-2144867ba53d
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a008124c-220e-491c-9ea1-2144867ba53d
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a008124c-220e-491c-9ea1-2144867ba53d
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a008124c-220e-491c-9ea1-2144867ba53d
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a008124c-220e-491c-9ea1-2144867ba53d
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a008124c-220e-491c-9ea1-2144867ba53d
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 41ffe12e-e54d-42b1-8409-15b357eac74a
@romanlutz romanlutz changed the title DOC Document attack and scenario outcomes DOC Document attack outcomes and scenario resiliency Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants