fix: cloud control hotswap removes reserved aws: tags

[dgandhi62]

Two cli-integ tests on the shared cc-hotswap stack failed on every PR (tests 5 & 6), blocking the merge queue:

• hotswap deployment supports CloudControl-based resources with attribute resolution
• hotswap deployment caches template and uses it for subsequent hotswaps

Both died on the --hotswap deploy with:

❌  cdktest-...-cc-hotswap failed to deploy
‣ HotswapFailed: [cloudcontrol] Failed to update <queue-url> (AWS::SQS::Queue)
‣ ValidationException: aws: prefixed tag key names are not allowed for external use.

Only the SQS Queue failed. The CloudWatch Dashboard and Events Rule in the same stack hotswapped successfully.

Nothing in this repo changed across the boundary: the last run with both shards green was 2026-08-31 21:12 UTC, the first red run 2026-09-01 15:24 UTC, and no commits merged to main in that window. The same PR passed both shards on Aug 31 and failed both on Sep 1. The hotswap code had been untouched since Aug 17 and the cc-hotswap fixture since Jul 31, so this seems to be a service-side change in what the tag update is allowed to do.

Why this happened (most likely)

The Queue's only changed property is Tags (the fixture's DynamicTag value moves between deploys), so the Cloud Control hotswap path emitted a single wholesale operation:

[{ "op": "replace", "path": "/Tags", "value": [
    { "Key": "DynamicTag",     "Value": "new value" },
    { "Key": "DynamoTableArn", "Value": "arn:aws:dynamodb:..." }
]}]

/Tags addresses the whole list, so this declares the resource's complete desired tag set. A resource created by CloudFormation also carries the reserved aws:cloudformation:stack-name / stack-id / logical-id tags, which are never present in the template. Therefore, reconciling to that set now seems to imply deleting them. Per the CloudFormation resource tagging docs the aws: prefix is reserved and such tags cannot be updated or deleted externally, and the SQS quotas say the same for delete.

Fix

Address individual tags by their index in the resource's current Tags list, so reserved tags are never named by the patch and the service sees no change to them:

[{ "op": "replace", "path": "/Tags/4", "value": { "Key": "DynamicTag", "Value": "new value" } }]

The live ordering is not the template ordering, so the indices cannot be derived from the template. They come from reading current state with Cloud Control GetResource.

Situation	Operation
Tag exists on the resource, value changed	replace /Tags/<liveIndex>
Tag in the template, absent from the resource	add /Tags/-
Tag on the resource, no longer in the template	remove /Tags/<liveIndex>, unless the key is aws:-prefixed
Tag unchanged	no operation

Ordering matters because JSON Patch operations apply in sequence: replacements go first (their indices refer to the unmodified list), removals follow in descending index order so each one leaves the others valid, and appends go last since /Tags/- only touches the end.

If the current tags cannot be read, this raises HotswapTagReadFailed instead of falling back to a wholesale replace. On a CloudFormation-created resource that fallback is precisely the request the service rejects, so degrading to it would resurface the original failure with a misleading cause. The error names the resource and points at cloudcontrolapi:GetResource permissions, and keeps the underlying cause attached.

Not covered

A few resource types model Tags as a { key: value } map rather than a list. There is no index to address there, so that shape keeps the previous behaviour. It is called out in a comment as untested against a service that also carries reserved tags on a map-shaped property.

Integ fixture

Re-enables the DynamicTag line in the cc-hotswap fixture, which was temporarily disabled to unblock CI. That restores Queue/Tags hotswap coverage — without it, the only tag-carrying resource in that stack never exercises the Cloud Control tag path and nothing would catch a regression here.

Testing

• Unit tests for the index-addressed patch (built on the live tag layout observed on a real queue), appends, index-descending removals that spare reserved keys, resources with no tags yet, and the loud read failure — which also asserts that no UpdateResource is sent in that case. The main test asserts the serialized patch contains neither a wholesale "path":"/Tags" nor the string aws: anywhere. 55/55 pass in the suite.
• Integ test cdk-hotswap-deployment-supports-cloudcontrol-based-resources.integtest reproduced the failure against a real account in us-east-1 before this change, and passes with it.

Known assumption

The approach assumes the live tag ordering is stable between the GetResource read and when Cloud Control applies the patch. The passing integ test shows this holds in practice for SQS; it is not proven to be deterministic across calls in general.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.33%. Comparing base (5b77563) to head (00a8756).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1940   +/-   ##
=======================================
  Coverage   91.33%   91.33%           
=======================================
  Files          79       79           
  Lines       12164    12164           
  Branches     1721     1721           
=======================================
  Hits        11110    11110           
  Misses       1019     1019           
  Partials       35       35           

Flag	Coverage Δ
suite.unit	91.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.