Skip to content

feat: add --with-ai options for generate-arazzo command - #3052

Merged
DmitryAnansky merged 21 commits into
mainfrom
feat/generate-arazzo-with-ai
Sep 2, 2026
Merged

feat: add --with-ai options for generate-arazzo command#3052
DmitryAnansky merged 21 commits into
mainfrom
feat/generate-arazzo-with-ai

Conversation

@DmitryAnansky

@DmitryAnansky DmitryAnansky commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

What/Why/How?

  • Added --with-ai and --ai-provider options for generate-arazzo command .
  • Fixed respect and the x-security-scheme-required-values rule rejecting x-security HTTP schemes written with non-lowercase casing (such as Basic, Bearer, or Digest) — RFC 7235 scheme names are case-insensitive.

Notes:

  • Use x-security when possible.
  • Provide hint with placeholders how to execute generated workflow.
  • Complex security flows are not supported due to lack of initial information to reconstruct it.
  • Two-pass design generation for big descriptions like GITHUB API.

Reference

https://github.com/Redocly/redocly/issues/25890

Testing

Screenshots (optional)

Check yourself

  • This PR follows the contributing guide
  • All new/updated code is covered by tests
  • Core code changed? - Tested with other Redocly products (internal contributions only)
  • New package installed? - Tested in different environments (browser/node)
  • Documentation update has been considered

Security

  • The security impact of the change has been considered
  • Code follows company security practices and guidelines

Note

Medium Risk
Changes touch authentication handling in generated Arazzo and Respect execution, and --with-ai sends resolved OpenAPI to a local AI CLI—review credential input naming and data-sharing before use.

Overview
generate-arazzo is refactored into a dedicated module and extended with --with-ai (plus --ai-provider, --ai-model, --ai-concurrency, --max-workflows). After the usual mechanical baseline, an optional pass calls a local AI CLI (claude, codex, or cursor) to merge operations into multi-step workflows; answers are gated (operation references, workflow caps, spec lint) and failures fall back to the baseline with a warning. Large OpenAPI documents use a two-phase flow (scenario selection from an operation index, then per-scenario design with parallel workers).

On success, the written file is prefixed with an AI-inferred comment. The command also prints a respect hint with --input placeholders for every workflow input (including resolved component refs).

Arazzo auto-generation now targets Arazzo 1.1.0, emits x-security on steps (replacing workflow-level Authorization parameters), adds OAuth2/OpenID inputs, and uses scheme-scoped basic/digest username/password fields. respect-core generate() returns a structured TestDescription instead of a string.

Shared CLI AI plumbing (providers, lint/fence helpers, runWorkerPool) is centralized under utils/ai and reused by generate-spec.

Case-insensitive HTTP auth schemes (Basic, Bearer, Digest, etc.) are honored in respect execution, the x-security-scheme-required-values rule, and related digest handling (RFC 7235).

Reviewed by Cursor Bugbot for commit 6505e10. Bugbot is set up for automated code reviews on this repo. Configure here.

@changeset-bot

changeset-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 6505e10

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@redocly/cli Minor
@redocly/respect-core Minor
@redocly/openapi-core Minor
@redocly/client-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@DmitryAnansky DmitryAnansky self-assigned this Aug 25, 2026
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Performance Benchmark (Lower is Faster)

CLI Version Bundle Lint Check Config
cli-latest ▓ 1.00x (Fastest) ▓ 1.00x ± 0.01 ▓ 1.00x (Fastest)
cli-next ▓ 1.01x ± 0.01 ▓ 1.00x (Fastest) ▓ 1.01x ± 0.02

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 78.78% (🎯 77%) 14222 / 18052
🔵 Statements 78.54% (🎯 77%) 15318 / 19501
🔵 Functions 82.44% (🎯 81%) 2922 / 3544
🔵 Branches 71.79% (🎯 71%) 10568 / 14720
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/cli/src/types.ts 100% 100% 100% 100%
packages/cli/src/commands/generate-arazzo/index.ts 93.33% 89.28% 80% 93.33% 48, 91-92
packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts 100% 100% 100% 100%
packages/cli/src/commands/generate-arazzo/ai/prompt.ts 96.42% 94.73% 100% 96.29% 197-201
packages/cli/src/commands/generate-arazzo/ai/provider-answer.ts 88.09% 74.07% 100% 88.09% 60, 64, 70, 74, 80
packages/cli/src/commands/generate-arazzo/ai/two-pass.ts 87.5% 72.88% 87.5% 88.63% 55, 65, 75, 78, 109-113, 116, 119-121, 134, 137, 171, 183, 185, 196, 289, 296
packages/cli/src/commands/generate-spec/index.ts 0% 0% 0% 0% 31-113
packages/cli/src/commands/generate-spec/ai/refine.ts 97.08% 86.2% 100% 97% 104, 135, 139
packages/cli/src/utils/spinner.ts 93.1% 78.57% 100% 93.1% 26, 57
packages/cli/src/utils/ai/output.ts 100% 100% 100% 100%
packages/cli/src/utils/ai/worker-pool.ts 100% 87.5% 100% 100%
packages/core/src/rules/respect/x-security-scheme-required-values.ts 90.9% 93.75% 100% 90.9% 40-43
packages/respect-core/src/generate.ts 0% 100% 0% 0% 17
packages/respect-core/src/modules/arazzo-description-generator/generate-arazzo-description.ts 100% 85.71% 100% 100%
packages/respect-core/src/modules/arazzo-description-generator/generate-inputs-arazzo-components.ts 100% 100% 100% 100%
packages/respect-core/src/modules/arazzo-description-generator/generate-step-security.ts 94.73% 90.9% 100% 94.73% 32
packages/respect-core/src/modules/arazzo-description-generator/generate-workflows-from-description.ts 100% 76.66% 100% 100%
packages/respect-core/src/modules/context-parser/get-security-parameters.ts 100% 100% 100% 100%
packages/respect-core/src/modules/flow-runner/validate-x-security-parameters.ts 100% 100% 100% 100%
packages/respect-core/src/utils/api-fetcher.ts 55.1% 45.37% 73.33% 54.79% 88-94, 124, 151-167, 178-179, 189-192, 197-200, 205, 208-229, 242-255, 263, 310-316, 323-401, 409, 417, 438, 453, 460, 463, 467
Generated in workflow #11607 for commit 6505e10 by the Vitest Coverage Report Action

@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai and --ai-provider options for generate-arazzo command [POC] feat: add the experimental --with-ai options for generate-arazzo command [POC] Aug 25, 2026
@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai options for generate-arazzo command [POC] feat: add the experimental --with-ai options for generate-arazzo command (POC) Aug 25, 2026
Comment thread .changeset/generate-arazzo-with-ai.md Outdated
@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai options for generate-arazzo command (POC) feat: add the experimental --with-ai options for generate-arazzo command Aug 26, 2026
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch 3 times, most recently from c3b2dec to 37a0234 Compare August 26, 2026 14:05
@DmitryAnansky
DmitryAnansky marked this pull request as ready for review August 27, 2026 10:01
@DmitryAnansky
DmitryAnansky requested review from a team as code owners August 27, 2026 10:01
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai options for generate-arazzo command feat: add --with-ai options for generate-arazzo command Aug 27, 2026
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch from 2709a88 to c39b251 Compare August 27, 2026 14:30
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch from 50c3833 to 24e1f7a Compare August 27, 2026 15:38
Comment thread .changeset/generate-arazzo-respect-hint.md Outdated
Comment thread .changeset/generate-arazzo-with-ai.md Outdated
Comment thread .changeset/respect-scheme-case.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
@DmitryAnansky
DmitryAnansky requested a review from JLekawa August 28, 2026 08:07
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch from bce13b9 to d4eb89e Compare August 31, 2026 08:39
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread .changeset/generate-arazzo-with-ai.md Outdated
Comment thread .changeset/generate-arazzo-with-ai.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
@DmitryAnansky
DmitryAnansky requested a review from JLekawa August 31, 2026 13:59
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md
Comment thread packages/cli/src/commands/generate-arazzo/ai/prompt.ts
Comment thread packages/cli/src/commands/generate-arazzo/ai/prompt.ts
Comment thread packages/cli/src/commands/generate-arazzo/ai/prompt.ts
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
Comment thread packages/cli/src/commands/generate-arazzo/ai/prompt.ts
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
Comment thread packages/cli/src/commands/generate-arazzo/ai/prompt.ts

@sobanieca-redocly sobanieca-redocly left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cyberpunks ownership looks good. Left one comment about merging and sharing AI utilities between generate-spec and generate-arazzo

Comment thread packages/cli/src/commands/generate-arazzo/index.ts
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch 2 times, most recently from aaa1b5e to 336c8a3 Compare September 1, 2026 14:58
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch from a3784e9 to 6c97708 Compare September 2, 2026 06:47

@vadyvas vadyvas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Comment thread packages/cli/src/__tests__/commands/generate-arazzo.test.ts Outdated
@DmitryAnansky
DmitryAnansky merged commit 1645ad2 into main Sep 2, 2026
52 of 53 checks passed
@DmitryAnansky
DmitryAnansky deleted the feat/generate-arazzo-with-ai branch September 2, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants