Scale fleet queries and harden sideloading in AzLocal.UpdateManagement v0.9.22

[NeilBird]

Summary

Release AzLocal.UpdateManagement v0.9.22 with two complementary hardening tracks:

• Scale Azure Resource Graph queries and pipeline reporting for large estates, including environments approaching or exceeding 1,000 subscriptions.
• Make the opt-in disconnected-cluster sideload workflow safe, scalable, and diagnosable for customer operation.

Existing deployments remain backward compatible. Fleet management-group scope is optional, and the generated config/sideload-settings.yml starter is disabled by default. Copy/Update creates settings only when absent and never overwrites or migrates operator-owned files.

Fleet-scale improvements

• Recover from ARG ResponsePayloadTooLarge by reducing page size and retrying the same logical page while preserving continuation completeness.
• Add optional management-group scope through inert, schema-versioned config/fleet-settings.yml; explicit subscription scope still takes precedence.
• Reduce Monitor: 1 and Monitor: 2 wire payloads and use stable ordering for paged queries.
• Batch explicit values, resolve subscription names in one query, and fetch failed-run health evidence once per effective fleet scope.
• Bound high-cardinality Markdown tables and enforce the GitHub step-summary byte budget while keeping CSV, JSON, JUnit, and HTML artifacts complete.
• Bound ServiceNow incident fan-out and stop after repeated API failures.

Sideload hardening

• Add authoritative schema-1 config/sideload-settings.yml plus exported Get-AzLocalSideloadSettings validation.
• Remove active SIDELOAD_* configuration fallbacks; enabled: false is the default and YAML is the sole runtime configuration source.
• Preserve existing settings byte-for-byte; unsupported schemas fail clearly and no speculative migration command is included.
• Persist state before Scheduled Task launch, assign unique operation IDs, and prevent superseded workers from overwriting current state.
• Separate copy, discovery, and import retries; persist NeedsSbe and ImportFailed truthfully; add heartbeat and no-progress controls.
• Add independent copy ceilings: maxConcurrentCopies limits the fleet/shared network and optional maxConcurrentCopiesPerRunner limits the current host. Existing schema-1 files that omit the host cap retain their prior behavior by defaulting it to the fleet cap.
• Count host capacity from shared-state OwningMachine values, so a serialized reconciliation run can enforce both ceilings without runner-to-runner administrative trust. A tested/throttled pool can use, for example, maxConcurrentCopies: 30 and maxConcurrentCopiesPerRunner: 10.
• Keep Scheduled Tasks runner-local. Shared state/log/cache paths support re-drive from another available runner, but automatic peer WinRM administration is intentionally excluded; peer remoting remains a documented break-glass option.
• Reject S4U/Interactive task identities for UNC-backed copies and require a network-capable principal. Password mode reads the task password from Key Vault as SecureString.
• Ensure self-hosted jobs install Az.Accounts, Az.KeyVault, powershell-yaml, and AzLocal.UpdateManagement with bounded PSGallery retries.
• Expand Markdown/JUnit diagnostics with unreadable-state errors, operation/process IDs, log paths, and state-specific remediation.
• Continue using per-cluster UpdateSideloaded and UpdateVersionInProgress Azure tags to gate Apply Updates and reconcile successful runs.

Pipelines and documentation

• Update GitHub Actions and Azure DevOps sideload/preflight flows to consume typed settings and forward both copy ceilings.
• Add create-only starter deployment to Copy-AzLocalPipelineExample and Update-AzLocalPipelineExample.
• Update the module README, CI/CD runbook, sideload operations guide, pipeline appendix, and changelog.
• Document anonymized typed Robocopy profiles and effective allow-listed switches, with a link to the official Microsoft Learn Robocopy syntax reference.
• Document the serialized active/passive runner-pool model, runner-local task affinity, central UNC state/logging, HA boundaries, scaling considerations, and 100-cluster throughput sizing.
• Add an explicit PowerShell Gallery package allow-list and stage-only validation path.

Validation

• Full Pester suite: 1,687 passed, 0 failed, 1 skipped.
• Sideload-focused suite: 81 passed, 0 failed.
• Sideload workflow/documentation contracts: 5 passed, 0 failed.
• Retry and self-hosted prerequisite contracts: 41 passed, 0 failed.
• Publish-Module.ps1 -StageOnly: passed for v0.9.22 with 71 exports, 242 packaged files, zero unexpected artifacts, and all required runtime/sideload assets present.
• PowerShell, Markdown, GitHub Actions, and Azure DevOps diagnostics: clean.
• git diff --check: clean; all newly added lines are ASCII-compatible.
• Live Azure integration suite against the approved AdaptiveCloudLab subscription: 43 passed, 0 failed, 0 skipped.
• Read-only v0.9.22 live release certification: passed with every exercised report-rendering, ARG transport, ring-readiness, and pipeline-template assertion green. The optional -SchedulePath check was not exercised because no schedule path was supplied.