diff --git a/.github/workflows/terraform.yml b/.github/workflows/terraform.yml index d9b5dda3d0..67db3ea837 100644 --- a/.github/workflows/terraform.yml +++ b/.github/workflows/terraform.yml @@ -64,7 +64,7 @@ jobs: key: tflint-${{ hashFiles('.tflint.hcl') }} - if: contains(matrix.terraform, '1.5.') name: Setup TFLint - uses: terraform-linters/setup-tflint@6e1e0642c0289bd619021bf6b34e3c08ed1e005a # v6.3.0 + uses: terraform-linters/setup-tflint@b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93 # v6.2.2 with: github_token: ${{ secrets.GITHUB_TOKEN }} - if: contains(matrix.terraform, '1.5.') @@ -138,7 +138,7 @@ jobs: key: tflint-${{ hashFiles('.tflint.hcl') }} - if: contains(matrix.terraform, '1.3.') name: Setup TFLint - uses: terraform-linters/setup-tflint@6e1e0642c0289bd619021bf6b34e3c08ed1e005a # v6.3.0 + uses: terraform-linters/setup-tflint@b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93 # v6.2.2 with: github_token: ${{ secrets.GITHUB_TOKEN }} - if: contains(matrix.terraform, '1.3.') @@ -203,7 +203,7 @@ jobs: key: tflint-${{ hashFiles('.tflint.hcl') }} - if: contains(matrix.terraform, '1.5.') name: Setup TFLint - uses: terraform-linters/setup-tflint@6e1e0642c0289bd619021bf6b34e3c08ed1e005a # v6.3.0 + uses: terraform-linters/setup-tflint@b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93 # v6.2.2 with: github_token: ${{ secrets.GITHUB_TOKEN }} - if: contains(matrix.terraform, '1.5.') diff --git a/docs/adr/002-runner-orchestration-provider-boundary.md b/docs/adr/002-runner-orchestration-provider-boundary.md index 3583f389e3..5f7e5344c6 100644 --- a/docs/adr/002-runner-orchestration-provider-boundary.md +++ b/docs/adr/002-runner-orchestration-provider-boundary.md @@ -14,14 +14,14 @@ Before this change, the multi-runner module received workflow-job demand through That layout assumes every runner config uses the same demand-control model. It also makes the runner-config module responsible for webhook-specific resources. Adding another model would require provider conditionals throughout the module or a second copy of the common runner and compute-provider wiring. -GitHub Actions Runner Scale Sets require a different control model. A future implementation is expected to use the runner scale-set and agent APIs, including: +GitHub Actions Runner Scale Sets require a different control model. The scale-set implementation uses the runner scale-set and agent APIs, including: - `_apis/runtime/runnerscalesets` - `_apis/distributedtask/pools/0/agents` -Unlike the current event and schedule driven Lambda components, a scale-set controller maintains reconciliation state and long-lived coordination with GitHub. It may therefore need a containerized service, with ECS as a candidate deployment target, rather than another independent Lambda handler. +Unlike the event- and schedule-driven Lambda components, a scale-set controller maintains reconciliation state and long-lived coordination with GitHub. It therefore runs as a containerized ECS/Fargate service rather than another independent Lambda handler. -The Terraform contract should make that future addition possible without moving webhook fields a second time. This ADR defines that boundary. It does not implement the scale-set API client, controller, container image, or ECS resources. +The Terraform contract must support both models without moving webhook fields a second time or creating one controller service per runner config. This ADR defines the boundary and the initial scale-set integration: the API client, one controller process with multiple reconcilers, container packaging, ECS resources, and EC2 compute capability. Automatic scale-set discovery or creation, multi-task leader election, and horizontal controller scaling remain outside this decision. ## Terminology @@ -29,7 +29,7 @@ The Terraform contract should make that future addition possible without moving - **Orchestration provider**: The implementation that receives or reconciles runner demand and owns the control components needed to turn that demand into capacity actions. - **Compute provider**: The implementation that creates and manages runner capacity, such as EC2. It supplies capabilities to the selected orchestration provider. - **Webhook orchestration**: The existing webhook, queue, scale-up, scale-down, pool, and job-retry implementation. -- **Scale-set orchestration**: A future stateful controller built on GitHub's runner scale-set APIs. +- **Scale-set orchestration**: The stateful controller built on GitHub's runner scale-set APIs and deployed in controller groups. This ADR uses “runner config” for the concept and `runner-config` for the module. @@ -39,7 +39,7 @@ We will use typed orchestration-provider and compute-provider boundaries in the ### Provider selection is per runner config -Every experimental runner config must contain an `orchestration_provider` object with exactly one non-null typed provider block. In this phase the only supported block is `webhook`: +Every experimental runner config must contain an `orchestration_provider` object with exactly one non-null typed provider block. The supported blocks are `webhook` and `scale_set`: ```hcl experimental = { @@ -78,11 +78,13 @@ experimental = { Selection is based on the populated provider block, not on a string discriminator. The wrapper's nullness and any value that controls resource shape must be known during planning. Other values inside the selected provider may remain unknown until apply. -Validation counts non-null provider blocks rather than naming one special case. A future provider can therefore be added as a sibling without changing the selection rule. Different runner configs may select different providers once more than one exists, but one runner config cannot combine providers. +Validation counts non-null provider blocks rather than naming one special case. Another provider can therefore be added as a sibling without changing the selection rule. Different runner configs may select webhook and scale-set orchestration in the same deployment, but one runner config cannot combine providers. + +Scale-set selection owns GitHub scope and installation identity, existing scale-set identity, desired capacity, boot timeout, and optional session/work-folder settings. The [runner scale-set controller guide](../scale-set.md) documents the experimental input contract and operating model. ### Global orchestration blocks provide defaults; they do not select providers -`experimental.orchestration_provider.webhook` is the global defaults and shared-component namespace for webhook orchestration. Its presence does not select webhook orchestration for every runner config. Selection remains under `experimental.multi_runner_config..orchestration_provider`. +`experimental.orchestration_provider.webhook` is the global defaults and shared-component namespace for webhook orchestration. `experimental.orchestration_provider.scale_set` is the global shared-controller namespace for scale-set orchestration. Neither block selects a provider for a runner config. Selection remains under `experimental.multi_runner_config..orchestration_provider`. The webhook global namespace owns: @@ -115,18 +117,23 @@ runner-config override > experimental.orchestration_provider.webhook default Tag maps merge from broad to narrow. A runner-config override affects only that runner config; it does not configure a shared singleton. +The global scale-set namespace owns controller grouping, container settings, non-secret manifest storage, ECS cluster/task/service/IAM settings, private task networking, logging, and controller resource tags. Per-runner scale-set settings are not global defaults: each scope, installation ID, scale-set identity, capacity range, and boot timeout belongs to exactly one runner config. Scale-set orchestration uses only the primary App ID and private-key Parameter Store references derived from `experimental.github.app`; it does not select an entry from `experimental.github.additional_apps`. Each runner config supplies its own installation-ID reference because the primary App can have installations in multiple GitHub scopes. + +`modules/multi-runner` filters scale-set selections, gathers their exact-keyed compute-provider capabilities, and calls `modules/orchestration-providers/scale-set` once. The provider resolves controller groups using `compute_provider`, `runner_config`, or explicit `custom` grouping. Each group owns one ECS service, one task definition, one normally running task, one controller process, and multiple reconcilers. Runner-config does not instantiate a scale-set controller per runner config. + ### Module ownership follows the provider boundary The Terraform implementation is split as follows: | Layer | Responsibility | | --- | --- | -| `modules/multi-runner` | Selects stable or experimental mode, resolves global and runner-config values, owns shared webhook ingress and build queues, and routes typed provider objects. | +| `modules/multi-runner` | Selects stable or experimental mode, resolves global and runner-config values, owns shared webhook ingress and build queues, routes typed provider objects, and makes the one aggregated scale-set provider call. | | `modules/runner-config` | Composes provider-neutral runner resources, selects exactly one orchestration provider and one compute provider, creates or selects the runner role, and connects provider capabilities. | | `modules/orchestration-providers/webhook` | Owns webhook orchestration composition, provider defaults, tag layering, and the scale, pool, and retry leaf modules. | | `modules/orchestration-providers/webhook/scale-runners` | Owns the scale-up and scale-down Lambdas, schedules, queue integration, IAM, and outputs. | | `modules/orchestration-providers/webhook/pool` | Owns optional scheduled pool resources and IAM. | | `modules/orchestration-providers/webhook/job-retry` | Owns optional queued-job retry resources and IAM. | +| `modules/orchestration-providers/scale-set` | Aggregates selected scale-set runner configs and compute capabilities, resolves controller groups, stores non-secret reconciler manifests, and owns ECS/Fargate, task IAM, networking, health checks, and logging. | | `modules/runner-config/ssm-housekeeper` | Owns provider-neutral cleanup of runner token and configuration parameters, including its component-specific Lambda artifact. | | `modules/compute-providers///trust-policy` | Produces the provider-specific runner-role trust policy before the common runner role is resolved. | | `modules/compute-providers//` | Owns capacity resources and returns policy, environment-variable, managed-policy, and resource capabilities. | @@ -138,15 +145,18 @@ flowchart TD Multi["multi-runner: normalize and route"] --> Config["runner-config: compose one runner config"] Config --> Selector{"exactly one orchestration provider"} Selector --> Webhook["orchestration-providers/webhook"] - Selector -. future .-> ScaleSet["orchestration-providers/scale-set"] + Selector --> Marker["scale-set selection and fixed JIT lifecycle"] Config --> ComputeSelector{"exactly one compute provider"} ComputeSelector --> Trust["compute-providers/aws/ec2/trust-policy"] Trust --> Role["runner role"] Role --> EC2["compute-providers/aws/ec2"] EC2 --> Capabilities["compute capabilities"] - Capabilities --> Adapter["runner-config capability adapter"] - Adapter --> Webhook - Adapter -. future .-> ScaleSet + Capabilities --> Webhook + Capabilities --> Aggregate["multi-runner: exact-keyed scale-set aggregate"] + Marker --> Aggregate + Aggregate --> ScaleSet["one orchestration-providers/scale-set call"] + ScaleSet --> Groups["N controller groups"] + Groups --> Reconcilers["one service/task/controller and N reconcilers per group"] Webhook --> Scale["scale-runners"] Webhook --> Pool["pool"] Webhook --> Retry["job-retry"] @@ -158,7 +168,7 @@ Provider leaf modules live below `modules/orchestration-providers/webhook`, not The selected compute provider remains independent from the selected orchestration provider. Its trust-policy submodule supplies the runner-role trust document before the common role is resolved. The full compute provider then returns policy documents, environment variables, managed-policy references, and resources needed by orchestration components. -`runner-config` adapts that provider output into the scale-up, scale-down, and pool capabilities consumed by webhook orchestration. The webhook provider owns its Lambda roles and attaches the capability fragments it needs. The compute provider does not create the common runner role or webhook resources. +`runner-config` adapts the legacy policy and environment fragments into the scale-up, scale-down, and pool capabilities consumed by webhook orchestration. It also exposes the selected provider's typed orchestration capabilities to multi-runner. For scale-set orchestration, EC2 returns provider-owned, non-secret runtime JSON plus environment and IAM fragments; GitHub scope, credentials, desired capacity, and boot timeout remain orchestration-owned. The webhook provider owns its Lambda roles, and the scale-set provider owns controller task roles. The compute provider creates neither orchestration resource. This direction keeps the dependency graph one-way: @@ -166,17 +176,17 @@ This direction keeps the dependency graph one-way: runner-config -> compute provider -> capability contract -> runner-config adapter -> orchestration provider ``` -A future scale-set controller may require a different subset or extension of the capability contract. That extension belongs at the provider boundary; it must not add scale-set conditionals to the webhook leaves. +Each additional orchestration capability belongs at the compute-provider boundary; it must not add provider conditionals to unrelated orchestration leaves. ### Compatibility and state are explicit Stable inputs are translated into the same internal canonical representation so defaults and shared singleton values have one resolution path. Stable runner configs continue to call the existing `modules/runners` implementation at their existing addresses. Opting into experimental v2 is module-wide: a non-empty `experimental.multi_runner_config` replaces, rather than merges with, the stable map. -The runner config uses one explicitly named, count-addressed module per concrete provider. Compute modules follow `module.compute__[0]`, while orchestration modules follow `module.orchestration_[0]`. The AWS EC2 modules therefore use `module.compute_aws_ec2_trust_policy[0]` and `module.compute_aws_ec2[0]`. +Concrete provider modules use explicitly named, count-addressed labels. Compute modules follow `module.compute__[0]`, while orchestration modules follow `module.orchestration_[0]`. The AWS EC2 modules therefore use `module.compute_aws_ec2_trust_policy[0]` and `module.compute_aws_ec2[0]` inside runner-config; multi-runner uses `module.orchestration_scale_set[0]` for the cross-runner aggregate. Declarative moved blocks preserve existing experimental state for the AWS namespace rename: `module.compute_ec2_trust_policy[0]` moves to `module.compute_aws_ec2_trust_policy[0]`, and `module.compute_ec2[0]` moves to `module.compute_aws_ec2[0]`. These moves are scoped to those v2 child modules; unrelated earlier experimental addresses remain outside the stable contract and require explicit migration when affected. -The canonical v2 output groups demand-control resources under `orchestration_provider.webhook` and compute resources under the selected namespace and provider, currently `provider.aws.ec2`. Direct `scale_up`, `scale_down`, and `pool` outputs remain compatibility aliases during the experimental transition. Moved blocks preserve resource state addresses but cannot rewrite configuration expressions, so consumers must update references from the former experimental `provider.ec2` path. +The canonical v2 output groups per-runner webhook resources under `orchestration_provider.webhook`, represents scale-set selection under `orchestration_provider.scale_set`, and keeps compute resources under the selected namespace and provider, currently `provider.aws.ec2`. Direct `scale_up`, `scale_down`, and `pool` outputs remain compatibility aliases and are null for scale-set runner configs. Grouped controller resources are exposed separately through the top-level `scale_set` output. No moved blocks are added for the new experimental scale-set path. Moved blocks preserve earlier compute state addresses but cannot rewrite configuration expressions, so consumers must update references from the former experimental `provider.ec2` path. This ADR does not define an automatic stable-v1-to-v2 state migration. Existing deployments remain on the stable path until that migration is separately designed and documented. @@ -194,28 +204,21 @@ The shared webhook remains at its existing unconditional module address. The sha Any later proposal to make those shared modules conditional is a separate compatibility and state decision. -### Scale-set implementation is deferred +### Scale-set integration and deferred capabilities -No `scale_set` field is added to the Terraform type in this phase. The typed object and module layout reserve the extension point without publishing an incomplete contract. +The v2 type publishes `scale_set` as a sibling of `webhook`. The runtime opens GitHub message sessions, reconciles one or more existing scale sets, stores no credential values in manifests, and delegates runner lifecycle operations through typed compute-provider adapters. Terraform deploys one service per resolved controller group with desired count one. -A follow-up design must decide at least: +The provider adopts existing GitHub scale sets and deliberately does not discover, create, or delete them. Callers supply the scale-set name and ID plus a Parameter Store reference for the installation ID. The canonical GitHub scope and numeric scale-set ID form the ownership key. Terraform rejects duplicate keys within one module instance, and operators must keep the same key unique across deployments because separate Terraform states cannot detect competing controllers. -1. the public TypeScript SDK surface for runner scale-set and agent operations; -2. authentication, API-version negotiation, error mapping, retries, and idempotency; -3. the reconciliation and persistence model for desired, acquired, busy, and removed runners; -4. the controller's shutdown, recovery, concurrency, and high-availability behavior; -5. the container build and release contract for the TypeScript service; -6. whether ECS/Fargate is the default deployment and how networking, scaling, logging, health checks, and upgrades work; -7. the capabilities required from each compute provider; and -8. Terraform migration and coexistence behavior when the new provider is enabled. +Scale-set-created EC2 instances carry `ghr:created_by=scale-set-service`, and their reconciler owns GitHub deregistration and compute termination. The unchanged shared termination watcher can match the same environment, so a module instance containing scale-set runner configs rejects watcher-based runner deregistration. The watcher may remain enabled as a logging and metrics observer. Horizontal task scaling, leader election, automatic failover between controller tasks, and provider-independent persistent reconciliation state require a later decision. Until then, ECS restarts the single desired task and each controller uses its stable session-owner contract to reconnect. -The intended end state permits webhook and scale-set orchestration in the same multi-runner module instance when different runner configs select them. It does not permit both controllers to own the same runner config. +Webhook and scale-set orchestration can coexist in one multi-runner module instance when different runner configs select them. They cannot both own the same runner config, and scale-set selections do not receive webhook build queues or matcher entries. ## Consequences ### Positive -- A future orchestration provider becomes a sibling module instead of a cross-cutting conditional. +- Additional orchestration providers become sibling modules instead of cross-cutting conditionals. - Common runner settings and compute-provider configuration remain reusable across demand-control models. - Provider-owned queue, Lambda, artifact, IAM, and output settings have one discoverable namespace. - Exact-one validation prevents ambiguous ownership of a runner config. @@ -228,7 +231,7 @@ The intended end state permits webhook and scale-set orchestration in the same m - Global webhook defaults and per-runner webhook selection use similarly named blocks with different purposes. - Internal modules have explicit adapter objects and capability contracts that require maintenance. - Module-level validation resources add state-only objects and are not evaluated by a targeted plan that excludes them. -- Adding a stateful provider will still require new runtime, deployment, observability, and failure-recovery design; the Terraform boundary alone does not solve those concerns. +- The initial scale-set provider runs one task per group; horizontal high availability still requires leader election or an external ownership protocol. - Compatibility aliases temporarily expose both canonical and historical v2 output paths. ## Alternatives Considered @@ -251,11 +254,11 @@ Scale, pool, and retry are all webhook orchestration behavior. Leaving them unde **Decision**: Move the leaves under the webhook provider root. Earlier experimental addresses are not retained as an in-module state-migration contract. -### Add the scale-set schema and ECS service now +### Create one ECS service per runner config -Publishing placeholders would lock in names and types before the API client, reconciliation semantics, and runtime model have been validated. +This gives every reconciler independent deployment and failure isolation, but creates excessive services, task definitions, roles, and idle tasks for large installations. -**Decision**: Publish only the provider-neutral extension point now. Add the scale-set provider in a follow-up ADR and implementation. +**Decision**: Create one service per resolved controller group. Default grouping packs runner configs by compute-provider type; callers can choose per-runner or explicit custom grouping. ### Make the shared webhook and webhook secret conditional @@ -269,20 +272,25 @@ Implementation and review must verify the boundary at several levels. ### Terraform contract tests -- A runner config with exactly one webhook provider plans successfully. -- Zero selected orchestration providers fail with a focused validation message. Once a second typed provider is introduced, selecting multiple providers must fail the same rule. +- Runner configs with exactly one webhook or scale-set provider plan successfully. +- Zero or multiple selected orchestration providers fail the same focused validation rule. - Provider-wrapper nullness and other graph-shaping values must be plan-known; non-shaping values inside the selected provider may remain unknown until apply. - Per-runner values override global webhook defaults, and omitted nullable values inherit them. - Shared singleton resources consume global values rather than arbitrary per-runner overrides. - Stable inputs preserve stable resource addresses and output shape. - Fresh plans use the canonical `module.compute_aws_ec2_trust_policy[0]` and `module.compute_aws_ec2[0]` runner-config addresses, with declarative moved blocks mapping the prior v2 child labels. - Canonical nested outputs and compatibility aliases reference the same resources. +- Webhook and scale-set runner configs coexist, scale-set-only deployments retain the unconditional shared ingress, and only webhook selections create build queues and matcher entries. +- Default and custom grouping create one aggregated scale-set provider call with exact-keyed runner and compute-capability maps. +- Duplicate canonical GitHub scope and scale-set ID tuples fail within one module instance. ### Provider integration tests - `runner-config` routes only the selected orchestration provider. - The webhook root composes scale, pool, and retry leaves with the resolved values supplied by `multi-runner`; it does not invent fallback ARNs or empty resource objects. - Compute-provider capability fragments reach the correct webhook component. +- Scale-set compute capabilities reach the matching reconciler manifest and task IAM policy without GitHub credentials, scope, desired capacity, or boot timeout entering compute-provider JSON. +- A module instance containing scale-set runner configs rejects termination-watcher runner deregistration while permitting metrics-only watcher operation. - Existing shared queue-policy and compute-provider IAM behavior remains unchanged by this refactor. ### Compatibility checks @@ -297,4 +305,5 @@ Before an existing experimental deployment adopts the module rename, operators m ## References - [Experimental orchestration- and compute-provider refactor](../modules/internal/compute-provider-refactor.md) +- [Runner scale-set controller](../scale-set.md) - [GitHub Actions Runner Scale Set client](https://github.com/actions/scaleset) diff --git a/docs/index.md b/docs/index.md index cdfe9ed130..d583b2694b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,21 +1,21 @@ # GitHub Self-Hosted on AWS on Spot Instances -This [Terraform](https://www.terraform.io/) module creates the required infrastructure needed to host [GitHub Actions](https://github.com/features/actions) self-hosted, auto-scaling runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). It provides the required logic to handle the lifecycle for scaling up and down using a set of AWS Lambda functions. Runners are scaled down to zero to avoid costs when no workflows are active. +This [Terraform](https://www.terraform.io/) module creates the infrastructure needed to host [GitHub Actions](https://github.com/features/actions) self-hosted, auto-scaling runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). The stable webhook control plane manages runner lifecycle with AWS Lambda functions. Experimental multi-runner v2 can instead adopt existing GitHub runner scale sets and reconcile them through a long-running ECS service. Runners can scale down to zero when no workflows are active. ![Architecture](assets/runners.light.png#only-light) ![Architecture](assets/runners.dark.png#only-dark) ## Motivation -GitHub Actions `self-hosted` runners provide a flexible option to run CI workloads on the infrastructure of your choice. However, currently GitHub does not provide tooling to automate the creation and scaling of action runners. This module creates the AWS infrastructure to host action runners on spot instances. It also provides lambda modules to orchestrate the lifecycle of the action runners. +GitHub Actions `self-hosted` runners provide a flexible option to run CI workloads on infrastructure you control. This module creates the AWS infrastructure to host those runners on Spot instances and provides webhook-driven Lambda or experimental scale-set ECS orchestration for their lifecycle. -Lambda was selected as the preferred runtime for two primary reasons. Firstly, it enables the development of compact components with limited access to AWS and GitHub. Secondly, it offers a scalable configuration with minimal expenses, applicable at both the repository and organizational levels. The Lambda functions will be responsible for provisioning Linux-based EC2 instances equipped with Docker to handle CI workloads compatible with Linux and/or Docker. The primary objective is to facilitate Docker-based workloads. +Lambda was selected as the preferred runtime for the webhook model for two primary reasons. Firstly, it enables the development of compact components with limited access to AWS and GitHub. Secondly, it offers a scalable configuration with minimal expenses, applicable at both the repository and organizational levels. GitHub runner scale sets require a persistent message session, so experimental v2 runs that orchestration provider as an ECS/Fargate service instead. Both models can provision Linux-based EC2 instances equipped with Docker for Linux and container workloads. A pertinent question may arise: why not opt for Kubernetes? The current strategy aligns closely with the implementation of GitHub's action runners. The chosen approach involves installing the runner on a host where the necessary software is readily available, maintaining proximity to GitHub's existing practices. Another viable option could be AWS Auto Scaling groups. However, this alternative usually demands broader permissions at the instance level from GitHub. Additionally, managing the scaling process, both up and down, becomes a non-trivial task in this scenario. ## Overview -The module is designed to be used in a GitHub organization. It can also be used in a GitHub repository, but this does not support all features. The module is receiving GitHub webhook events for the `workflow_job` event. The module will create a new runner if the event is for a workflow that requires a runner, and no runner is available. Alternatively the module can be configured as ephemeral runners. In this case the module will create a new runner for each workflow job event. +The module is designed to be used in a GitHub organization. It can also be used in a GitHub repository, but this does not support all features. In the stable webhook model, the module receives the `workflow_job` event and creates a runner when a matching workflow requires capacity. Alternatively, experimental v2 can reconcile demand from an existing GitHub runner scale set. Webhook orchestration supports persistent or ephemeral runners; scale-set orchestration uses fixed ephemeral JIT runners. For ephemeral runners a pool can be configured. The pool maintains a minimum number of runners based on a schedule. The pool works only for org level runners. @@ -70,7 +70,7 @@ The AMI cleaner is a lambda that will clean up AMIs that are older than a config This feature is Beta, changes will not trigger a major release as long in beta. -The Instance Termination Watcher is creating log and optional metrics for termination of instances. Currently only spot termination warnings are watched. See [configuration](configuration/) for more details. +The Instance Termination Watcher creates logs and optional metrics for instance termination. Currently only Spot termination warnings are watched. In a module instance that also contains scale-set runner configs, runner deregistration must be disabled on the watcher; it can remain enabled as a logging and metrics observer while the scale-set reconciler owns GitHub runner deregistration. See [configuration](configuration/) for more details. ### Job Retry @@ -96,20 +96,23 @@ Permissions are managed in several places. Below are the most important ones. Fo - The GitHub App requires access to actions and to publish `workflow_job` events to the AWS webhook (API gateway). - The scale up lambda should have access to EC2 for creating and tagging instances. - The scale down lambda should have access to EC2 to terminate instances. +- Each scale-set controller-group task role is limited to that group's manifest path, exact GitHub App parameters and KMS keys, and the selected compute-provider capability statements. -Besides these permissions, the lambdas also need permission to CloudWatch (for logging and scheduling), SSM and S3. For more details about the required permissions see the [documentation](modules/public/setup-iam-permissions.md) of the IAM module which uses permission boundaries. +Besides these permissions, the Lambdas also need access to CloudWatch for logging and scheduling, Parameter Store, and S3. The scale-set controller uses CloudWatch Logs, exact Parameter Store and optional KMS references, and the provider-owned capacity actions for its group. For more details about the required permissions see the [documentation](modules/public/setup-iam-permissions.md) of the IAM module which uses permission boundaries. ## Terraform main modules Currently we support two main modules. The existing `runners` module remains the stable EC2 implementation, and the `multi-runner` module creates multiple runner configurations in one deployment. Stable top-level `multi_runner_config` entries continue to use the unchanged `runners` module when `experimental.multi_runner_config` is empty. A non-empty experimental map takes priority over the stable map; the maps are not combined. Experimental entries use the new provider-oriented `runner-config`. -Multi-runner centralizes mode selection and canonical configuration in `config.experimental.translation.tf`. A non-empty experimental runner-configuration map selects v2; otherwise the file projects the flat globals and stable runner configurations into the same schema. That selection produces `local.raw_translated_experimental`, from which the same file derives `local.translated_experimental_base` by applying schema defaults, global/runner-configuration precedence, tag merges, IAM ownership, paths, observability, webhook queues, and provider defaults. Provider selection and the shared runner-binary syncer and discovery use this plan-known base. After discovery, the translation file derives the final `local.translated_experimental`, including labels, runner-config GitHub client settings, webhook queue event mapping, Lambda artifact and principals, the webhook pool Lambda wrapper, SSM KMS, and the discovered EC2 binaries object. The remaining shared components, webhook queues, and runner implementations consume that final canonical representation. Stable runner configurations are adapted back into the existing `module.runners["configuration"]` call, preserving their Terraform addresses while removing a second configuration path. The `module.runner_configs` call directly iterates the gated final runner configurations, inlines the environment tag and live GitHub App and build-queue references into `orchestration_provider.webhook`, and forwards the remaining canonical objects, including the typed orchestration and compute-provider wrappers. +Multi-runner centralizes mode selection and canonical configuration in `config.experimental.translation.tf`. A non-empty experimental runner-configuration map selects v2; otherwise the file projects the flat globals and stable runner configurations into the same schema. That selection produces `local.raw_translated_experimental`, from which the same file derives `local.translated_experimental_base` by applying schema defaults, global/runner-configuration precedence, tag merges, IAM ownership, paths, observability, webhook queues, and provider defaults. Provider selection and the shared runner-binary syncer and discovery use this plan-known base. After discovery, the translation file derives the final `local.translated_experimental`, including labels, runner-config GitHub client settings, webhook queue event mapping, Lambda artifact and principals, the webhook pool Lambda wrapper, SSM KMS, and the discovered EC2 binaries object. The remaining shared components, webhook queues, and runner implementations consume that final canonical representation. Stable runner configurations are adapted back into the existing `module.runners["configuration"]` call, preserving their Terraform addresses while removing a second configuration path. The `module.runner_configs` call iterates v2 entries and forwards the typed orchestration and compute-provider wrappers. Webhook selections receive their live App and build-queue references in the runner-config child; scale-set selections expose their compute capability to one aggregate provider call at `module.orchestration_scale_set[0]`. -The `experimental` object provides sibling global defaults through `tags`, `roles`, `runner`, `github`, `lambda`, `orchestration_provider`, `ssm`, `observability`, and `compute_provider`. Root `experimental.lambda` is provider-neutral shared Lambda substrate: the artifact bucket, runtime, architecture, principals, networking, tags, and role defaults used across orchestration and non-orchestration consumers. Webhook-specific global defaults live together under `experimental.orchestration_provider.webhook`, including the maximum runner count, the shared webhook's routing and matcher storage, build-queue defaults and encryption, control-plane artifact selectors, and webhook, scale-up, scale-down, and pool Lambda settings. This global block supplies defaults; it does not select an orchestration provider. Each runner configuration separately makes that selection through its own `orchestration_provider` wrapper. The only supported orchestration provider today is `orchestration_provider.webhook`, which owns that runner configuration's maximum runner count, registration scope, matcher, build-queue overrides, scale-up and scale-down settings, scheduled pool, and job retry. Keeping those fields behind a typed provider wrapper allows future orchestration providers to be introduced as mutually exclusive siblings without moving the common runner, Lambda substrate, SSM, observability, or compute-provider contracts again. The shared GitHub App Parameter Store module, webhook, runner-binary syncer, termination watcher, and AMI housekeeper consume the translated global values. Migrated v2 consumers do not fall back to matching flat inputs; those flat values seed stable-mode translation only. Per-configuration overrides remain configuration-specific. A nullable runner-configuration field with a corresponding experimental global inherits that global value when omitted or null. A runner configuration that selects an external runner IAM role intentionally suppresses inherited managed policies and additional trust policy JSON because the module does not manage that role. Tag maps merge from broad to narrow. Only module naming (`prefix`), `aws_partition`, and `aws_region` remain active flat-only inputs; legacy `iam_overrides` remains in the input schema but has no active consumer. +The `experimental` object provides sibling global defaults through `tags`, `roles`, `runner`, `github`, `lambda`, `orchestration_provider`, `ssm`, `observability`, and `compute_provider`. Root `experimental.lambda` is provider-neutral shared Lambda substrate. `experimental.orchestration_provider.webhook` owns webhook defaults and shared routing, matcher, queue, artifact, and Lambda settings. `experimental.orchestration_provider.scale_set` owns shared controller grouping, container, manifest storage, ECS, networking, logging, and tags. Neither global block selects a provider; every runner config selects exactly one typed `webhook` or `scale_set` sibling. Webhook selections own lifecycle, capacity, registration scope, matching, queue overrides, scale, pool, and retry. Scale-set selections own GitHub scope, the installation-ID reference, existing scale-set identity, desired capacity, and boot timeout. + +Multi-runner makes one aggregated scale-set provider call for every selected scale-set runner config. The default grouping creates one ECS service per compute-provider type; `runner_config` grouping creates one per runner config, and `custom` grouping maps selected runner configs into explicit groups. Each group normally runs one task, one controller, and one reconciler per scale set. See [Runner scale-set controller](scale-set.md) for the deployment and ownership contract. Migrated v2 consumers do not fall back to matching flat inputs; flat values seed stable-mode translation only. A runner configuration that selects an external runner IAM role suppresses inherited managed policies and additional trust policy JSON because the module does not manage that role. Tag maps merge from broad to narrow. Only module naming (`prefix`), `aws_partition`, and `aws_region` remain active flat-only inputs; legacy `iam_overrides` remains in the input schema but has no active consumer. Global `experimental.orchestration_provider.webhook.queue` owns v2 defaults for build-queue delay (`30` seconds), retention (`86400` seconds), visibility (`180` seconds), redrive, tags, and encryption. Runner-configuration `experimental.multi_runner_config[].orchestration_provider.webhook.queue` fields override the global delay, retention, visibility, redrive, and tag values; encryption remains global-only. Omitting the whole encryption block selects SQS-managed encryption and null KMS attributes. If the block is supplied explicitly, all three leaf keys are required: use a non-null `sqs_managed_sse_enabled` with null KMS fields for the non-KMS mode, or set that field to null and provide `kms_master_key_id` for KMS mode. This encryption configures the multi-runner build queues and their dead-letter queues, not the webhook provider's separate job-retry queue, and its CMK is independent from `experimental.ssm.kms_key_id`. Runner-config forwards the distinct build-queue key to the webhook orchestration provider: scale-up receives `kms:Decrypt`, while job-retry receives `kms:Decrypt` and `kms:GenerateDataKey` for publishing. The existing shared `modules/webhook` contract remains unchanged and still requires caller-supplied key access when that publisher targets customer-managed encrypted queues. For v2, `experimental.multi_runner_config[].orchestration_provider.webhook.queue.visibility_timeout_seconds` must be at least six times the resolved `experimental.multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.timeout`; the Lambda timeout does not itself configure queue visibility. The v1 translation continues to use `runners_scale_up_lambda_timeout` and flat `queue_encryption`. -V2 requires `experimental.github.app`, and `experimental.github.additional_apps` defaults to an empty list. These nested values are authoritative end-to-end: the shared Parameter Store module persists or selects their credentials, and v2 runner configurations consume the resulting references. Flat `github_app` and `additional_github_apps` seed stable-mode translation only. `experimental.orchestration_provider.webhook.github.repository_white_list` defaults to `[]` and filters the shared webhook when populated. `experimental.github.enterprise_server.url` defaults to `null` and configures v2 runner-config GitHub clients and the termination watcher. `experimental.github.enterprise_server.ssl_verify` and `experimental.github.user_agent` remain runner-config client settings and default to `true` and `github-aws-runners`. Both client settings belong inside `experimental.github`; they are not root `experimental` fields or per-configuration orchestration settings. +V2 requires `experimental.github.app`, and `experimental.github.additional_apps` defaults to an empty list. These nested values are authoritative end-to-end: the shared Parameter Store module persists or selects their credentials, and v2 runner configurations consume the resulting references. Scale-set orchestration always uses the primary App ID and private-key references from `experimental.github.app`; it does not select from `additional_apps`. Every scale-set runner config provides its own `orchestration_provider.scale_set.github.installation_id_ssm` name and ARN, plus an optional KMS key ARN. Manifests contain only Parameter Store references, never credential values. Flat `github_app` and `additional_github_apps` seed stable-mode translation only. `experimental.orchestration_provider.webhook.github.repository_white_list` defaults to `[]` and filters the shared webhook when populated. `experimental.github.enterprise_server.url` defaults to `null` and configures v2 runner-config GitHub clients and the termination watcher. `experimental.github.enterprise_server.ssl_verify` and `experimental.github.user_agent` remain runner-config client settings and default to `true` and `github-aws-runners`. The scale-set service scopes disabled TLS verification to each reconciler and uses the configured user agent as the `system` identity inside GitHub's structured scale-set protocol header. Both settings belong inside `experimental.github`; they are not root `experimental` fields or per-configuration orchestration settings. The shared webhook, runner configurations, SSM housekeepers, runner-binary syncer, termination watcher, and AMI housekeeper consume the provider-neutral runtime, architecture, networking, role, and tag defaults under `experimental.lambda`; `lambda.principals` configures runner-config, runner-binary-syncer, termination-watcher, and AMI-housekeeper roles, but not the webhook role. Global observability settings configure logging and tracing, and global metrics also configure the termination watcher. The runner-control artifact shared by webhook scale, pool, and job-retry is selected globally under `experimental.orchestration_provider.webhook.lambda.artifact`: `artifact.zip` selects a local archive, while `artifact.s3.{key,object_version}` selects an object from the shared `experimental.lambda.artifact.s3.bucket`. Leaving both artifact sources null uses the packaged runner archive. The module validates that zip and S3 are not selected together and that an S3 wrapper has a non-null shared bucket and key. Stable-mode translation preserves the legacy precedence in which a configured flat S3 bucket wins over the flat runner zip. The shared bucket alone selects no component. Each artifact-capable singleton—including the webhook—uses it only when that component's separate nested `artifact.s3` wrapper supplies its key and optional object version. Runner-config's common SSM housekeeper independently resolves `multi_runner_config[].ssm.housekeeper.lambda.artifact` over the global `experimental.ssm.housekeeper.lambda.artifact`; S3 combines the component key and version with the shared artifact bucket, zip uses the selected local path, and no selection uses the packaged control-plane archive. The runner-binary syncer follows the same parallel selector at `experimental.compute_provider.aws.ec2.runner_binaries.syncer.artifact.{zip,s3}`, with its S3 key and optional object version also resolved against `experimental.lambda.artifact.s3.bucket`. `experimental.orchestration_provider.webhook` owns queue selection, EventBridge routing, accepted event types, and matcher-parameter tier, while `experimental.orchestration_provider.webhook.lambda.webhook` owns the separate ingress webhook artifact, API Gateway access logs, sizing, and component tags. `compute_provider.aws.ec2.instance_termination_watcher`, `compute_provider.aws.ec2.ami.housekeeper`, and `compute_provider.aws.ec2.runner_binaries` own their singleton-specific features, artifacts, sizing, schedules, and related settings. @@ -117,7 +120,11 @@ The shared webhook, runner configurations, SSM housekeepers, runner-binary synce Global `ssm.paths.root` is the base for shared and runner-configuration-owned parameters. The shared GitHub App and webhook paths append `ssm.paths.app` (default `app`) and `ssm.paths.webhook` (default `webhook`), while normalization appends the runner-configuration key only for configuration-owned paths. The default derived base is `/github-action-runners/${prefix}`, and runner token/config segments default to `runners/tokens` and `runners/config`. Global `ssm.kms_key_id` is an optional ARN-valued scalar that encrypts the shared GitHub App parameters, configures the webhook, and adds matching decrypt permissions to every runner configuration; it does not select encryption for runtime-created runner parameters. Webhook-provider leaves conditionally omit their KMS statements when this value is null, while apply-time-unknown key ARNs remain valid during planning. Nested metrics retain the established defaults: disabled, using the `GitHub Runners` namespace, with the rate-limit, job-retry, Spot-termination, and Spot-warning switches enabled. Spot metrics are global termination-watcher settings rather than per-configuration overrides. -Each runner configuration selects two independent typed providers: one `orchestration_provider` provider for demand control and one namespaced `compute_provider` for runner capacity. `orchestration_provider.webhook` is the sole supported orchestration provider today. The global `experimental.compute_provider` block owns shared v2 provider defaults plus the runner-binary, termination-watcher, and AMI-housekeeper singleton configuration, but it does not select a provider. Today the only selectable compute leaf is `compute_provider.aws.ec2`. The wrapped provider objects reach runner-config, which validates each exact-one selection and dispatches the matching root-level provider module. `orchestration-providers/webhook` owns scale-up, scale-down, scheduled pool, job retry, and their webhook-specific defaults and tag layering; runner-config retains common SSM housekeeping plus the common runner role and attachments. The EC2 implementation lives under `compute-providers/aws/ec2`, supplies EC2-specific policy requirements, and owns the instance profile, launch template, bootstrap resources, and runner log groups. Runner-config dispatches it at `module.compute_aws_ec2[0]` and exposes its resources under the matching nested output path `provider.aws.ec2` (for multi-runner, `runners_map_v2[""].provider.aws.ec2`). Declarative moved blocks preserve state created at the earlier experimental `module.compute_ec2[0]` and `module.compute_ec2_trust_policy[0]` child addresses when upgrading to the namespaced labels. They do not migrate stable-v1 `module.runners` state to v2, and they cannot rewrite configuration references from `provider.ec2` to `provider.aws.ec2`. These modules are internal experimental implementation boundaries, not standalone public entry points. Later releases can add mutually exclusive namespace and provider siblings without changing the common contract. See the [experimental orchestration- and compute-provider refactor](modules/internal/compute-provider-refactor.md) and [multi-runner v2 migration roadmap](modules/public/multi-runner.md#multi-runner-v2-migration-roadmap). EC2 is the only active Terraform-managed provider; microVM, CodeBuild, and other provider modules are future work. +Each runner configuration selects two independent typed providers: one `orchestration_provider` for demand control and one namespaced `compute_provider` for runner capacity. V2 supports `orchestration_provider.webhook` and `orchestration_provider.scale_set`; the only selectable compute leaf is currently `compute_provider.aws.ec2`. The wrapped objects reach runner-config, which validates each exact-one selection and dispatches the selected compute provider plus per-runner webhook orchestration when applicable. Scale-set orchestration is aggregated at multi-runner scope so grouping can span runner configs. + +The scale-set provider adopts existing GitHub scale sets and never discovers, creates, or deletes them. A canonical GitHub scope plus numeric scale-set ID must be unique across all deployments that can reach that scope; Terraform can reject duplicates only inside its own module instance. Scale-set-created EC2 instances are tagged `ghr:created_by=scale-set-service` and owned by their scale-set reconciler. The shared termination watcher may coexist only as a logging and metrics observer with runner deregistration disabled. + +The EC2 implementation lives under `compute-providers/aws/ec2`, supplies provider-specific policy and scale-set capability requirements, and owns the instance profile, launch template, bootstrap resources, and runner log groups. Runner-config dispatches it at `module.compute_aws_ec2[0]` and exposes resources under `provider.aws.ec2` (for multi-runner, `runners_map_v2[""].provider.aws.ec2`). Declarative moved blocks preserve state created at the earlier experimental `module.compute_ec2[0]` and `module.compute_ec2_trust_policy[0]` child addresses. They do not migrate stable-v1 `module.runners` state to v2 or rewrite consumer references from `provider.ec2` to `provider.aws.ec2`. These modules remain internal experimental boundaries. See the [experimental orchestration- and compute-provider refactor](modules/internal/compute-provider-refactor.md), [runner scale-set controller](scale-set.md), and [multi-runner v2 migration roadmap](modules/public/multi-runner.md#multi-runner-v2-migration-roadmap). EC2 is the only active Terraform-managed compute provider; MicroVM, CodeBuild, and other compute providers remain future work. Both modules are built on top of the same base modules. When using the multi-runner module you can deploy different runners with only one deployment. diff --git a/docs/modules/internal/compute-provider-refactor.md b/docs/modules/internal/compute-provider-refactor.md index 4577ffc5ba..351cbe7e6a 100644 --- a/docs/modules/internal/compute-provider-refactor.md +++ b/docs/modules/internal/compute-provider-refactor.md @@ -8,7 +8,7 @@ The scale-up, scale-down, pool, job-retry, queue, SSM housekeeping, and GitHub registration workflows are not inherently EC2-specific. The legacy `runners` module combines webhook demand orchestration with EC2 launch templates, instance profiles, bootstrap parameters, log groups, IAM permissions, and Lambda environment variables. Adding another orchestration or compute provider in that structure would require copying common behavior or adding provider conditionals throughout the module. -The refactor introduces two typed boundaries. An orchestration provider owns the demand-control model and its components; a compute provider owns runner capacity and exports capabilities consumed by that orchestration. A future scale-set controller or MicroVM backend can therefore be added without moving public provider-owned settings or scattering provider conditionals through leaf modules; the central typed schema, normalization, routing, and dispatch still require extension. +The refactor introduces two typed boundaries. An orchestration provider owns the demand-control model and its components; a compute provider owns runner capacity and exports capabilities consumed by that orchestration. The scale-set controller now uses this boundary alongside webhook orchestration, and a future MicroVM backend can be added without moving public provider-owned settings or scattering provider conditionals through leaf modules; the central typed schema, normalization, routing, and dispatch still require extension. ## Ownership model @@ -16,12 +16,13 @@ The implementation is split into common runner-config composition, orchestration | Layer | Owns | | --- | --- | -| `multi-runner` | Module-level v1/v2 mode selection, flat/nested input projection, canonical global/runner-config resolution, typed orchestration- and compute-provider routing, config keys, webhook build queues and matching, and runner-binary discovery. | -| `runner-config` | Typed orchestration- and compute-provider dispatch, shared runner bootstrap config in SSM, the SSM housekeeper, and either a module-managed common runner role with policy attachments or selection of an external runner role. | +| `multi-runner` | Module-level v1/v2 mode selection, flat/nested input projection, canonical global/runner-config resolution, typed orchestration- and compute-provider routing, config keys, webhook build queues and matching, runner-binary discovery, and the one aggregated scale-set provider call. | +| `runner-config` | Typed orchestration- and compute-provider dispatch, fixed scale-set JIT lifecycle selection, provider-neutral capability output, shared runner bootstrap config in SSM, the SSM housekeeper, and either a module-managed common runner role with policy attachments or selection of an external runner role. | | `orchestration-providers/webhook` | Webhook defaults and tag layering, plus composition of the provider-owned control-plane leaves. | | `orchestration-providers/webhook/scale-runners` | Compute-provider-neutral scale-up and scale-down Lambdas, schedules and queue integration, and their execution roles and policies. | | `orchestration-providers/webhook/pool` | Optional scheduled runner-pool resources and their Lambda and IAM wiring. | | `orchestration-providers/webhook/job-retry` | Optional queued-job retry resources and their Lambda and IAM wiring. | +| `orchestration-providers/scale-set` | Cross-runner controller grouping, non-secret reconciler manifests, ECS/Fargate services and task definitions, task IAM, private networking, health checks, and logging. | | `runner-config/ssm-housekeeper` | Parameter Store cleanup Lambda, schedule, logging, and IAM resources. | | `compute-providers///trust-policy` | Provider-specific default runner-role trust, merged with the optional caller-provided trust document before the common role is created. | | `compute-providers//` | Provider-specific resources, permission requirements, and the IAM and environment-variable fragments consumed by the common control plane after the runner role is resolved. | @@ -30,15 +31,17 @@ The EC2 provider owns the instance profile, launch template, security group, AMI Runner-config, the root orchestration and compute providers, and their leaf modules are internal implementation boundaries rather than standalone public modules. Callers opt into the experimental interface through `experimental.multi_runner_config`; `multi-runner` calls `runner-config`, which selects the provider modules. Their direct input and output contracts may change while v2 remains experimental. -Each external v2 runner config selects demand orchestration separately from its compute provider. The required `orchestration_provider` wrapper has one supported provider today: `experimental.multi_runner_config..orchestration_provider.webhook`. It owns the runner config's lifecycle and maximum runner count, registration scope, matcher, build-queue overrides, scale-up, scale-down, pool, and job-retry settings. The wrapper is intentionally typed as a provider boundary so later orchestration implementations can be added as mutually exclusive siblings without moving common settings again. +Each external v2 runner config selects demand orchestration separately from its compute provider. The required `orchestration_provider` wrapper supports mutually exclusive `webhook` and `scale_set` siblings. Webhook owns lifecycle and maximum runner count, registration scope, matcher, build-queue overrides, scale-up, scale-down, pool, and job retry. Scale-set owns its GitHub scope and installation-ID reference, existing scale-set name and ID, desired capacity, boot timeout, and optional session owner and work folder. Exactly one sibling must be non-null. + +Scale-set orchestration is adoption-only: the controller does not discover, create, or delete GitHub scale sets. Its ownership key is the canonical GitHub scope plus numeric scale-set ID. Validation rejects duplicates inside one multi-runner deployment, while operators must keep the tuple unique across deployments because independent Terraform states cannot detect one another. The controller uses only the primary App ID and private-key references derived from `experimental.github.app`; each selected runner config supplies the installation-ID Parameter Store reference for its own scope. The runner config also populates exactly one typed compute-provider leaf, such as `experimental.multi_runner_config..compute_provider.aws.ec2`; that leaf's presence must be known during planning because it determines capacity routing. Multi-runner resource preconditions enforce both selections and the public contract's cross-scope and plan-shaping rules, while each provider implementation validates its resolved internal contract. -After resolving global `experimental.compute_provider.aws.ec2` values with the selected runner config's `compute_provider.aws.ec2` overrides, `multi-runner` preserves the namespaced typed wrapper expected by `runner-config`. The direct contract is `compute_provider = { aws = { ec2 = { ... } } }`, not a flat EC2 object. Runner-config flattens each populated namespace and provider leaf into an internal dispatch key such as `aws_ec2`, validates that exactly one leaf is non-null, and passes `compute_provider.aws.ec2` to `module.compute_aws_ec2[0]` as its nested `config` object. The webhook runtime registry still receives the provider type `ec2`; the namespace is part of Terraform dispatch so different clouds can expose similarly named services without colliding. Runner-config independently validates the exact-one `orchestration_provider = { webhook = { ... } }` wrapper and invokes the selected root orchestration provider with provider-neutral common objects such as `runner`, the Lambda substrate, SSM, observability, and the selected compute-provider capabilities. +After resolving global `experimental.compute_provider.aws.ec2` values with the selected runner config's `compute_provider.aws.ec2` overrides, `multi-runner` preserves the namespaced typed wrapper expected by `runner-config`. The direct contract is `compute_provider = { aws = { ec2 = { ... } } }`, not a flat EC2 object. Runner-config flattens each populated namespace and provider leaf into an internal dispatch key such as `aws_ec2`, validates that exactly one leaf is non-null, and passes `compute_provider.aws.ec2` to `module.compute_aws_ec2[0]` as its nested `config` object. The webhook runtime registry still receives the provider type `ec2`; the namespace is part of Terraform dispatch so different clouds can expose similarly named services without colliding. Runner-config independently validates exact-one orchestration selection. It calls `module.orchestration_webhook[0]` for webhook selections; for scale-set selections it applies fixed ephemeral JIT lifecycle settings and exposes `{ type, capabilities }` through `compute_provider_contract` without creating a controller child per runner config. Binary discovery is completed before the runner-config call. `config.experimental.translation.tf` enriches the final canonical runner config at `compute_provider.aws.ec2.binaries_syncer.s3`, leaving `s3` null when synchronization is disabled. The `module.runner_configs` call then passes that runner config's wrapped `compute_provider` object unchanged. Runner-config and the EC2 provider therefore receive the typed provider-owned shape; neither expects a bare `{ arn, id, key }` object directly at `compute_provider.aws.ec2.binaries_syncer`. -Runner-config creates or selects the runner IAM role, but the current EC2 provider owns the role's default trust-policy document. Each provider implementation supplies a small `trust-policy` submodule that accepts `additional_trust_policy_json` and returns the final `assume_role_policy`. The full provider separately returns its nested `provider` contract containing `policies.runner`, `policies.scale_up`, `policies.scale_down`, and `policies.pool`, component environment variables, and provider resources. When runner-config creates the runner role, it uses the isolated trust-policy output and attaches the returned runner policies. An external role bypasses both operations, so its caller owns trust and permissions. Runner-config passes the scale-up, scale-down, and pool capabilities to the selected orchestration provider in either case. A provider never creates or attaches the common runner IAM role. +Runner-config creates or selects the runner IAM role, but the current EC2 provider owns the role's default trust-policy document. Each provider implementation supplies a small `trust-policy` submodule that accepts `additional_trust_policy_json` and returns the final `assume_role_policy`. The full provider separately returns its nested `provider` contract containing the provider `type`, typed `capabilities`, legacy webhook policy/environment fragments, and provider resources. EC2's additive `capabilities.scale_set` contains provider-owned non-secret runtime JSON, environment variables, and structured IAM statements. It excludes GitHub scope and credentials, desired capacity, and boot timeout. When runner-config creates the runner role, it uses the isolated trust-policy output and attaches the returned runner policies. An external role bypasses both operations, so its caller owns trust and permissions. A provider never creates or attaches the common runner IAM role or an orchestration role. The trust relationship is deliberately rendered by an isolated provider submodule: @@ -47,8 +50,8 @@ The trust relationship is deliberately rendered by an isolated provider submodul 3. `compute-providers///trust-policy` combines the provider default with `runner.iam.additional_trust_policy_json` without referencing the runner-role input. 4. `runner-config` creates the common runner role from the returned `assume_role_policy`, or selects an external role without applying that trust policy. 5. The full compute provider receives the resolved role so it can create resources such as the EC2 instance profile and render `iam:PassRole` statements. -6. The provider returns its nested policy, environment-variable, and resource contract. -7. Runner-config attaches runner policies only to a module-managed runner role, while `orchestration-providers/webhook` attaches scale-up, scale-down, and pool policy fragments to the roles it owns through its leaves. +6. The provider returns its nested type, capability, policy, environment-variable, and resource contract. +7. Runner-config attaches runner policies only to a module-managed runner role. `orchestration-providers/webhook` attaches scale-up, scale-down, and pool policy fragments to the roles it owns; multi-runner aggregates typed scale-set capabilities for the single scale-set provider call, whose controller groups own their task roles. The trust-policy output depends only on its input documents, not on the full provider resources that consume the runner role. This preserves provider ownership of the trust relationship while keeping the dependency graph one-way. @@ -57,10 +60,10 @@ The trust-policy output depends only on its input documents, not on the full pro Multi-runner produces one canonical consumer representation for both input modes: 1. `config.experimental.translation.tf` selects the module mode and builds `local.raw_translated_experimental`. A non-empty experimental runner-config map selects the nested `var.experimental` input for v2; otherwise the file projects flat module globals and stable `multi_runner_config` entries into the same schema for v1. -2. The same translation file then derives `local.translated_experimental_base`. It applies schema defaults and global/runner-config precedence, merges tags, resolves IAM ownership and paths, and normalizes observability, `orchestration_provider.webhook`, and compute-provider values. Provider selection, plan-shaping validation, and the shared runner-binary syncer and discovery consume this fully resolved base. +2. The same translation file then derives `local.translated_experimental_base`. It applies schema defaults and global/runner-config precedence, merges tags, resolves IAM ownership and paths, and normalizes observability, both orchestration-provider siblings, and compute-provider values. Provider selection, plan-shaping validation, and the shared runner-binary syncer and discovery consume this fully resolved base. 3. After runner-binary discovery, the translation file derives the final `local.translated_experimental`. It completes runner labels, GitHub enterprise and User-Agent settings, shared Lambda artifacts and principals, the internal build-queue KMS projection and runner-control artifact, SSM KMS, and each enabled EC2 runner config's `compute_provider.aws.ec2.binaries_syncer.s3`. Webhook event-source mapping and pool resolution are already complete in the base object. The remaining shared components, webhook queues, and runner implementations consume the final canonical object. -Stable translation always emits `orchestration_provider.webhook`, but stable runner configs remain on `module.runners[""]`: `runners.tf` adapts each final canonical runner config back to the existing `modules/runners` input contract, preserving Terraform addresses without maintaining a separate config source. This is not the phase-2 implementation migration to `runner-config`. For v2, `module.runner_configs` directly iterates the gated final runner-config map. Its adapter passes environment-augmented tags, GitHub settings with live App references, and the resolved Lambda, SSM, and observability inputs at the runner-config top level. It injects the live build queue into the webhook orchestration input, forwards the provider-owned fields accepted by runner-config, and omits `matcherConfig` because the shared webhook consumes it. The wrapped compute-provider object is forwarded unchanged. Binary output enrichment and all other derived config shaping are already complete in canonical translation. +Stable translation always emits `orchestration_provider.webhook` and an explicitly null `scale_set`, but stable runner configs remain on `module.runners[""]`: `runners.tf` adapts each final canonical runner config back to the existing `modules/runners` input contract, preserving Terraform addresses without maintaining a separate config source. For v2, `module.runner_configs` directly iterates the gated final runner-config map. Its adapter injects a live build queue only for webhook selections and passes only a plan-known marker for scale-set selection. The wrapped compute-provider object is forwarded unchanged. Multi-runner separately filters scale-set selections, gathers exact-keyed `compute_provider_contract` outputs, and calls `module.orchestration_scale_set[0]` once so grouping can span runner configs. ## Phase 1 dispatch and compatibility @@ -77,10 +80,10 @@ flowchart TD Base --> Discovery["Provider selection, runner-binary syncer, and discovery"] Discovery --> Final["translated_experimental: enrich aws.ec2 binaries_syncer.s3"] Final --> Singleton["Shared SSM, webhook, termination watcher, and AMI housekeeper"] - Final --> Shared["Webhook build queues and matching"] + Final --> Shared["Webhook-only build queues and matching"] Final -->|v1 legacy-argument adapter| Legacy["module.runners[key]"] Final -->|v2 direct module input adaptation| RunnerConfig["module.runner_configs[key]"] - RunnerConfig --> Orchestration["orchestration-providers/webhook"] + RunnerConfig --> Orchestration["orchestration-providers/webhook when selected"] Orchestration --> Scaling["orchestration-providers/webhook/scale-runners"] Orchestration --> Pool["orchestration-providers/webhook/pool"] Orchestration --> Retry["orchestration-providers/webhook/job-retry"] @@ -90,8 +93,11 @@ flowchart TD Role --> Provider RunnerConfig --> Provider["compute-providers//"] Provider --> Contract["compute-provider capability contract"] - Contract --> Adapter["runner-config capability adapter"] - Adapter --> Orchestration + Contract --> Orchestration + Contract --> Aggregate["multi-runner exact-keyed scale-set aggregate"] + Final --> Aggregate + Aggregate --> ScaleSet["one orchestration-providers/scale-set call"] + ScaleSet --> Groups["controller groups: one ECS service/task/controller and N reconcilers"] ``` The canonical object gives shared singleton resources one global representation and each webhook orchestration and runner implementation one fully resolved runner-config representation: @@ -103,6 +109,7 @@ The canonical object gives shared singleton resources one global representation - Stable queue tagging and the flat `runners_map` output remain unchanged. - When `experimental.multi_runner_config` is non-empty, every key in the experimental map calls `modules/runner-config` at `module.runner_configs[""]`; stable-map entries are not dispatched. - Experimental v2 uses `module.runner_configs[""]`; within each entry, the canonical provider child addresses are `module.runner_configs[""].module.compute_aws_ec2_trust_policy[0]`, `module.runner_configs[""].module.compute_aws_ec2[0]`, and `module.runner_configs[""].module.orchestration_webhook[0]`. Moved blocks inside `runner-config` preserve existing experimental state from the earlier `module.compute_ec2_trust_policy[0]` and `module.compute_ec2[0]` child labels when upgrading to these namespaced labels. +- Scale-set orchestration is aggregated outside the per-runner child at `module.orchestration_scale_set[0]`; no moved block is added for this new experimental path. - Experimental resources are exposed separately through the nested `runners_map_v2` output. - The maps are not combined. A non-empty v2 map has explicit priority over the stable map. @@ -110,7 +117,7 @@ The provider-label moves are limited to the existing v2 child modules. No v1-to- ## Opting in -Nested global settings are the source of defaults for v2 runner configs and the applicable singleton shared components. The GitHub App Parameter Store module, webhook, runner-binary syncer, termination watcher, and AMI housekeeper consume translated globals. Every v2 runner config must currently select `orchestration_provider.webhook`; no other orchestration provider is implemented. The wrapper is the durable provider boundary for future mutually exclusive siblings. Migrated v2 consumers do not fall back to matching flat inputs; those values seed the stable-mode translation only. The singleton-specific webhook, binary-syncer, termination-watcher, and AMI-housekeeper settings all have nested owners. Only module naming (`prefix`), `aws_partition`, and `aws_region` remain active flat-only inputs; legacy `iam_overrides` remains in the input schema but has no active consumer. Per-runner-config values override globals only inside that runner config and do not replace singleton-owned global settings. Each webhook runner config nevertheless contributes matcher, build-queue, and compute-provider routing data to the shared webhook, while its resolved binary-syncer enablement and OS/architecture determine shared syncer membership. Nested defaults mirror established v1 behavior, while nullable runner-config fields inherit the corresponding global when omitted or set to null. Set a global override only when the value is genuinely shared by every runner config, and put runner-specific differences in that config. An external `runner.iam.role` is the exception to normal inheritance: inherited managed policies and additional trust policy JSON are suppressed because the module does not manage that role. +Nested global settings are the source of defaults for v2 runner configs and the applicable singleton shared components. The GitHub App Parameter Store module, webhook, scale-set controller, runner-binary syncer, termination watcher, and AMI housekeeper consume translated globals. Every v2 runner config selects exactly one of `orchestration_provider.webhook` and `orchestration_provider.scale_set`. Migrated v2 consumers do not fall back to matching flat inputs; those values seed the stable-mode translation only. The singleton-specific webhook, binary-syncer, termination-watcher, and AMI-housekeeper settings all have nested owners. Global scale-set settings own controller grouping, container, manifest storage, ECS, networking, logging, and tags; per-runner scale-set settings own scope, installation, identity, capacity, and boot time. Only module naming (`prefix`), `aws_partition`, and `aws_region` remain active flat-only inputs; legacy `iam_overrides` remains in the input schema but has no active consumer. Webhook runner configs contribute matcher, build-queue, and compute-provider routing data to the shared webhook. Scale-set runner configs contribute exact-keyed compute capabilities to the shared controller provider and create no build queue or matcher entry. An external `runner.iam.role` suppresses inherited IAM-management inputs because the module does not manage that role. ```hcl module "multi_runner" { @@ -135,8 +142,8 @@ module "multi_runner" { } github = { - # Required in v2. These nested values are authoritative for shared SSM - # and every v2 runner config. + # Required in v2. Scale-set orchestration uses the primary app below; + # additional_apps does not select a scale-set controller identity. app = var.github_app additional_apps = var.additional_github_apps @@ -258,6 +265,22 @@ module "multi_runner" { } } } + + # Shared controller topology. This block does not select scale-set + # orchestration for a runner config. + scale_set = { + grouping = { + strategy = "compute_provider" + } + container = { + # Use an immutable image digest in production. + image = var.scale_set_controller_image + } + network = { + vpc_id = var.vpc_id + subnet_ids = var.controller_subnet_ids + } + } } # Shared resources append app/webhook. Runner configs append their key. @@ -270,8 +293,8 @@ module "multi_runner" { # This ARN-valued scalar may be unknown until apply. It encrypts shared # app parameters created by this module, configures the webhook, and - # grants runner-config decrypt access. Existing *_ssm references retain - # their external encryption. + # grants the selected runner-config or scale-set controller consumers + # decrypt access. Existing *_ssm references retain external encryption. kms_key_id = aws_kms_key.github_app_parameters.arn parameters = { @@ -404,10 +427,8 @@ module "multi_runner" { Environment = "arm-runners" } - # Demand-control settings are selected through a typed orchestration - # provider. Webhook is the only supported provider today; future - # providers can be added as mutually exclusive siblings without moving - # these fields again. + # Demand-control settings are selected through one typed webhook or + # scale_set provider block. orchestration_provider = { webhook = { # This runner config overrides the webhook provider's global cap. @@ -465,6 +486,37 @@ module "multi_runner" { } } } + + scale_set_linux = { + runner = { + architecture = "x64" + } + + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example" + installation_id_ssm = { + name = var.github_installation_id_parameter_name + arn = var.github_installation_id_parameter_arn + kms_key_arn = var.github_installation_id_kms_key_arn + } + } + name = "linux-scale-set" + id = 101 + min_runners = 0 + max_runners = 20 + } + } + + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + } + } + } + } } } } @@ -472,17 +524,17 @@ module "multi_runner" { ## Inputs, tags, and outputs -The `experimental` object has global siblings for `tags`, `roles`, `runner`, `github`, `lambda`, `orchestration_provider`, `ssm`, `observability`, and `compute_provider`, in addition to its runner-config map at `multi_runner_config`. Root `experimental.lambda` contains only provider-neutral shared Lambda substrate: the artifact bucket, runtime, architecture, principals, networking, role, and tag values. These settings configure v2 runner configs and shared consumers beyond webhook orchestration, including the runner-binary syncer, termination watcher, AMI housekeeper, and per-runner-config SSM housekeepers. `lambda.principals` configures v2 runner-config, runner-binary-syncer, termination-watcher, and AMI-housekeeper roles, but not the shared webhook role. Global webhook-specific defaults are grouped under `experimental.orchestration_provider.webhook`: runner lifecycle, boot time, and maximum count; repository filtering; shared routing and matcher storage; queue defaults and encryption; the runner-control artifact shared by scale, pool, and job-retry; and the ingress webhook, scale, and pool Lambda component settings. The global orchestration block is a defaults namespace, while each runner config's separate `orchestration_provider` wrapper is the exact-one provider selector. The termination watcher, AMI housekeeper, and runner-binary syncer retain their nested component owners under `compute_provider.aws.ec2`. The active flat-only settings are `prefix`, `aws_partition`, and `aws_region`; legacy `iam_overrides` remains in the schema without an active consumer. +The `experimental` object has global siblings for `tags`, `roles`, `runner`, `github`, `lambda`, `orchestration_provider`, `ssm`, `observability`, and `compute_provider`, in addition to its runner-config map at `multi_runner_config`. Root `experimental.lambda` contains only provider-neutral shared Lambda substrate: the artifact bucket, runtime, architecture, principals, networking, role, and tag values. Global webhook-specific defaults are grouped under `experimental.orchestration_provider.webhook`: runner lifecycle, boot time, and maximum count; repository filtering; shared routing and matcher storage; queue defaults and encryption; the runner-control artifact shared by scale, pool, and job retry; and the ingress webhook, scale, and pool Lambda component settings. Global scale-set topology is grouped under `experimental.orchestration_provider.scale_set`: grouping, container runtime, config store, ECS, network, logging, and tags. These global orchestration blocks configure shared components but do not select a provider; each runner config's separate wrapper is the exact-one selector. The termination watcher, AMI housekeeper, and runner-binary syncer retain their nested component owners under `compute_provider.aws.ec2`. The active flat-only settings are `prefix`, `aws_partition`, and `aws_region`; legacy `iam_overrides` remains in the schema without an active consumer. -Each v2 runner config groups common provider-neutral settings by owner under `runner`, `lambda`, `ssm`, and `observability`; backend settings live under `compute_provider..`. Demand-control settings live under a separate `orchestration_provider` wrapper. Its sole supported block today is `orchestration_provider.webhook`, containing provider-owned runner lifecycle, boot-time, and capacity settings, `github.organization_runners`, `matcherConfig`, `queue`, `lambda.scale.up`, `lambda.scale.down`, `lambda.pool`, and `job_retry`. A nullable per-runner-config field inherits its corresponding experimental global when omitted or null, except that an external runner role suppresses inherited IAM management inputs. Precedence within a runner config is therefore a non-null runner-config override followed by the global nested value, including that field's nested schema default. Per-runner-config precedence does not replace singleton-owned global settings: the shared GitHub App Parameter Store module, webhook implementation and routing, runner-binary syncer settings, termination watcher, and AMI housekeeper use translated globals. The shared webhook nevertheless aggregates each webhook runner config's matcher, build queue, and compute-provider route. A runner config's resolved binary-syncer enablement and OS/architecture determine whether its pair participates in the shared syncer set; all syncer and distribution-bucket settings remain global. +Each v2 runner config groups common provider-neutral settings by owner under `runner`, `lambda`, `ssm`, and `observability`; backend settings live under `compute_provider..`. Demand-control settings live under an exact-one `orchestration_provider` wrapper. `webhook` contains provider-owned lifecycle, capacity, matcher, queue, scale, pool, and retry settings. `scale_set` contains the GitHub config URL and installation-ID SSM reference, existing scale-set name and ID, capacity range, boot timeout, and optional session/work-folder settings. The fixed scale-set runner lifecycle is ephemeral with JIT enabled. Per-runner webhook precedence remains runner-config override followed by global nested value. Scale-set identity and capacity do not inherit global values; the global scale-set block owns only the shared controller topology. The shared webhook aggregates only webhook matchers, build queues, and compute routes. Multi-runner aggregates only scale-set entries and their exact-keyed compute capability contracts into the single scale-set provider call. Global `experimental.orchestration_provider.webhook.queue` owns v2 build-queue defaults. `delay_webhook_event` defaults to `30`, `job_queue_retention_in_seconds` to `86400`, `visibility_timeout_seconds` to `180`, and `tags` to `{}`. `redrive_build_queue.enabled` defaults to `false`, while `redrive_build_queue.maxReceiveCount` defaults to null. A null per-runner-config redrive wrapper or leaf inherits its corresponding global value, and an enabled result requires a resolved `maxReceiveCount` greater than zero. Fields under `experimental.multi_runner_config[].orchestration_provider.webhook.queue` override those global defaults, and runner-config queue tags merge over global queue tags. Build-queue visibility is independent from Lambda config: `experimental.multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.timeout` controls the function only, while `experimental.multi_runner_config[].orchestration_provider.webhook.queue.visibility_timeout_seconds` controls SQS and must be at least six times the resolved scale-up timeout. Queue encryption is global-only. Omitting the entire `experimental.orchestration_provider.webhook.queue.encryption` block defaults `sqs_managed_sse_enabled` to `true` and the KMS fields to null, matching flat `queue_encryption`. If callers supply an explicit block, all three leaf keys are required: use explicit nulls for inactive fields, with a non-null SQS-managed switch for the non-KMS mode or a non-null `kms_master_key_id` for KMS mode. It configures the multi-runner build queues and their dead-letter queues, not the webhook provider's separate job-retry queue. Runner configs cannot override encryption. The queue CMK and `experimental.ssm.kms_key_id` are independent and are forwarded separately to `orchestration-providers/webhook`: scale-up receives queue-key `kms:Decrypt`, job-retry receives queue-key `kms:Decrypt` and `kms:GenerateDataKey`, and both retain separate Parameter Store decrypt statements. The existing shared `modules/webhook` contract remains unchanged and still requires caller-supplied key access when it publishes to customer-managed encrypted queues. Build queues and dead-letter queues continue to reuse the existing singleton `DenyInsecureTransport` queue policy; this refactor does not change its wildcard `Resource`. The v1 translation retains the flat contract: per-runner-config delay, retention, redrive, and tags keep their stable sources, build-queue visibility comes from `runners_scale_up_lambda_timeout`, and encryption comes from `queue_encryption`. -Global `experimental.github` owns the GitHub App credentials persisted or selected by shared SSM and used by v2 runner configs: `app` is required and `additional_apps` defaults to `[]`. `experimental.orchestration_provider.webhook.github.repository_white_list` defaults to `[]` and filters the shared webhook when populated. `experimental.github.enterprise_server.url` defaults to `null` and configures both runner-config GitHub clients and the shared termination watcher. `experimental.github.enterprise_server.ssl_verify` defaults to `true`, and `experimental.github.user_agent` defaults to `github-aws-runners`; both remain runner-config client settings. Neither field is a root `experimental` sibling. Per-runner-config `orchestration_provider.webhook.github.organization_runners` is the separate registration-scope setting; runner configs do not override credentials, repository filtering, the enterprise endpoint, or the User-Agent. +Global `experimental.github` owns the GitHub App credentials persisted or selected by shared SSM and used by v2 runner configs: `app` is required and `additional_apps` defaults to `[]`. Scale-set orchestration uses the primary App ID and private key from `app`; it does not select an entry from `additional_apps`. `experimental.orchestration_provider.webhook.github.repository_white_list` defaults to `[]` and filters the shared webhook when populated. `experimental.github.enterprise_server.url` defaults to `null`, `experimental.github.enterprise_server.ssl_verify` defaults to `true`, and `experimental.github.user_agent` defaults to `github-aws-runners`. The scale-set service scopes disabled TLS verification to each reconciler and places the configured user agent in the required structured protocol header's `system` field. Per-runner webhook `github.organization_runners` remains the registration-scope setting. Each scale-set runner config instead owns an HTTPS `github.config_url` and an existing `github.installation_id_ssm` reference because the primary App can use different installations for different organizations or repositories. Scale-set manifests contain only Parameter Store names, never credential values. The shared App ID and private key use global `ssm.kms_key_id`; an external installation-ID parameter declares its own optional `kms_key_arn`. -Shared SSM creates or selects Parameter Store credentials from the authoritative `experimental.github` object, and the webhook and every v2 runner config consume the resulting references. Flat `github_app` and `additional_github_apps` seed only the stable-mode translation and impose no equality requirement in v2. The webhook does not make GitHub API requests. +Shared SSM creates or selects Parameter Store credentials from the authoritative `experimental.github` object. Webhook runner configs consume the resulting references, and the aggregated scale-set provider forwards the primary App references to its selected reconcilers. Flat `github_app` and `additional_github_apps` seed only the stable-mode translation and impose no equality requirement in v2. The webhook does not make GitHub API requests. Global `experimental.orchestration_provider.webhook` configures the shared webhook's queue-selection strategy, EventBridge implementation and accepted events, and matcher-configuration Parameter Store tier in addition to its queue and Lambda component defaults. `orchestration_provider.webhook.eventbridge.enable` and the matcher tier must be known during planning because they select module or parameter-chunk shape. `first` deterministically chooses the first equally matched queue by priority, `random` spreads jobs among equals, and `all` sends a job to every match at the cost of multiple runner launches and registrations. @@ -496,15 +548,15 @@ The global `experimental.compute_provider.aws.ec2` block owns v2 defaults for EC Webhook-orchestration runner-control artifacts are selected globally through `experimental.orchestration_provider.webhook.lambda.artifact.zip` or `experimental.orchestration_provider.webhook.lambda.artifact.s3.{key,object_version}` and shared by scale, pool, and job-retry. The S3 wrapper selects an object from the shared `experimental.lambda.artifact.s3.bucket`; null zip and S3 wrappers use the packaged runner archive. V2 validation rejects simultaneous zip and S3 selection and requires a non-null shared bucket and key when the S3 wrapper is present. Stable-mode translation preserves the old S3-wins rule by clearing the translated zip and creating the runner artifact's S3 wrapper whenever the flat `lambda_s3_bucket` is set. The shared bucket alone selects no component. Every artifact-capable singleton uses its own `artifact.s3` wrapper to supply that component's key and optional object version. Runner-config's common SSM housekeeper independently resolves `experimental.multi_runner_config[].ssm.housekeeper.lambda.artifact` over the global `experimental.ssm.housekeeper.lambda.artifact`; S3 combines the component key and version with the shared artifact bucket, zip uses the selected local path, and no selection uses the packaged control-plane archive. Stable translation maps the existing runner artifact into this separate canonical component contract. The ingress webhook artifact remains separate under `experimental.orchestration_provider.webhook.lambda.webhook.artifact`; the runner-binary syncer uses the parallel `experimental.compute_provider.aws.ec2.runner_binaries.syncer.artifact.{zip,s3}` selector, whose S3 key and optional object version resolve against the same shared bucket. `experimental.compute_provider.aws.ec2.instance_termination_watcher` owns watcher enablement, feature flags, runner deregistration, environment, artifact, and sizing. `experimental.compute_provider.aws.ec2.ami.housekeeper` owns enablement, cleanup behavior, artifact, sizing, and schedule. Watcher enablement and feature flags, runner-deregistration enablement, and AMI-housekeeper enablement must be known during planning because they control child resource shape. -Tags follow the same ownership model but merge rather than replace. Within v2 webhook queue and runner-config scopes, experimental global tags merge with runner-config tags and then with orchestration component or subcomponent tags from broad to narrow; a narrower value wins for a duplicate key. Singleton shared resources use only global scopes. Shared SSM merges `experimental.tags`, `ssm.tags`, and the forced `ghr:environment` tag. The webhook base resources merge `experimental.tags` with that environment tag; its Lambda additionally merges `experimental.lambda.tags` with `experimental.orchestration_provider.webhook.lambda.webhook.tags`. The runner-binary syncer, termination watcher, and AMI housekeeper receive global `tags`, the environment tag, and `lambda.tags`. Distribution buckets additionally merge `compute_provider.aws.ec2.runner_binaries.s3.tags`. EC2 global provider tags merge with per-runner-config `compute_provider.aws.ec2.tags`. EC2 runtime tags belong under `compute_provider.aws.ec2.tags`; bootstrap tags required by the runner are reserved inside the provider and are not propagated to common resources. +Tags follow the same ownership model but merge rather than replace. Within v2 webhook queue and runner-config scopes, experimental global tags merge with runner-config tags and then with orchestration component or subcomponent tags from broad to narrow; a narrower value wins for a duplicate key. Singleton shared resources use only global scopes. Shared SSM merges `experimental.tags`, `ssm.tags`, and the forced `ghr:environment` tag. The webhook base resources merge `experimental.tags` with that environment tag. Scale-set resources merge experimental global tags, `orchestration_provider.scale_set.tags`, and the forced environment tag; config-store and log tags apply to their narrower resources. EC2 global provider tags merge with per-runner `compute_provider.aws.ec2.tags`. When scale-set orchestration is selected, runner-config rejects caller values for EC2 scale-set ownership and lifecycle tag keys so controller IAM conditions cannot be bypassed; webhook selections retain their existing tag contract. Provider-created instances carry `ghr:created_by=scale-set-service`. Because the unchanged termination watcher can match the same environment, multi-runner rejects watcher-based GitHub deregistration whenever a runner config selects scale-set orchestration; metrics-only watcher operation remains allowed. Application logging settings stay together under `observability.logs`, including `level`, retention, encryption, class, and runner-config log-group tags. Tracing stays under `observability.tracing`, and metrics enablement, namespace, and individual metric switches stay under `observability.metrics`. -In v1 mode, entries remain exclusively in `runners_map` and retain their flat output fields; `runners_map_v2` is empty. In v2 mode, entries are exposed exclusively through `runners_map_v2` and `runners_map` is empty. Common resources are grouped under `runner`, demand-control resources under `orchestration_provider.webhook`, and provider-specific resources under `provider..`. The provider namespace and type are derived from the selected typed compute-provider leaf. For example, a module-managed common runner role is available at `runners_map_v2[""].runner.role`; the value is null when the caller selects an external role. Scale-up resources are available at `runners_map_v2[""].orchestration_provider.webhook.scale_up`, and launch-template and runner-log artifacts are under `runners_map_v2[""].provider.aws.ec2`. The top-level `scale_up`, `scale_down`, and `pool` fields remain compatibility aliases for their webhook-provider counterparts. The webhook `pool` value is null when no pool config is supplied. Output references are configuration expressions rather than state addresses, so moved blocks cannot rewrite the former experimental `provider.ec2` path for consumers. +In v1 mode, entries remain exclusively in `runners_map` and retain their flat output fields; `runners_map_v2` is empty. In v2 mode, entries are exposed exclusively through `runners_map_v2` and `runners_map` is empty. Common resources are grouped under `runner`, orchestration selection or per-runner resources under `orchestration_provider.`, and provider-specific resources under `provider..`. The top-level per-runner `scale_up`, `scale_down`, and `pool` compatibility aliases are null in scale-set mode. Cross-runner controller resources are exposed once through the module's top-level `scale_set` output, with `controller_groups` keyed by the resolved grouping. No state move is added for this new experimental path. ## Plan-time provider selection and IAM shape -Terraform must know resource and dynamic-block shape during planning, even when an ARN is produced by another resource and remains unknown until apply. Provider ownership inputs continue to use caller-known wrapper objects as discriminators. The webhook orchestration leaves conditionally emit KMS statements from the nullable Parameter Store and build-queue key scalars; a null value omits the statement, while an apply-time-unknown ARN remains valid during planning. These provider-owned statements do not render placeholder or sentinel ARNs. At the internal runner-config boundary, the relevant config fragments are: +Terraform must know resource and dynamic-block shape during planning, even when an ARN is produced by another resource and remains unknown until apply. Provider ownership inputs continue to use caller-known wrapper objects as discriminators. The webhook orchestration leaves conditionally emit KMS statements from nullable Parameter Store and build-queue key scalars. Scale-set credential references use nullable scalar `kms_key_arn` fields; the provider composes their optional decrypt fragment into a static policy-document input, so an unknown ARN defers policy content without an unknown dynamic-block count or a sentinel ARN. At the internal runner-config boundary, the relevant config fragments are: ```hcl ssm = { @@ -529,7 +581,7 @@ compute_provider = { The populated `aws.ec2` leaf tells both multi-runner routing and runner-config dispatch which provider implementation exists and must therefore be known during planning. Canonical translation preserves both namespace levels, and the `module.runner_configs` input forwards the wrapper unchanged at the runner-config boundary. The `orchestration_provider` wrapper follows the same exact-one rule. Within the compute block, each ownership-wrapper object tells Terraform that the corresponding policy exists; its `arn` may safely be computed. At this internal boundary, `ssm.kms_key_id`, the derived `orchestration_provider.webhook.queue.kms_key_id`, and values such as `observability.logs.kms_key_id` remain nullable scalar inputs even when their ARNs are unknown until apply. The public source of the derived queue key is `experimental.orchestration_provider.webhook.queue.encryption.kms_master_key_id`. -For experimental multi-runner v2, global `experimental.ssm.kms_key_id` encrypts shared GitHub App parameters created by the module, configures the webhook with the same key, and adds matching decrypt permissions to every runner config so its control-plane functions can read those credentials. Parameters selected through existing `*_ssm` references retain their external encryption and access requirements. The global key's value may be unknown until apply. It does not select encryption for runtime-created runner parameters. Queue encryption is a separate global contract, may use a different CMK, and reaches only the scale-up consumer and job-retry publisher policies inside the webhook orchestration provider. +For experimental multi-runner v2, global `experimental.ssm.kms_key_id` encrypts shared GitHub App parameters created by the module, configures the webhook, and authorizes selected controller consumers to decrypt the shared App ID and private key. External installation-ID references retain their own optional KMS declaration. The global key's value may be unknown until apply. It does not select encryption for runtime-created runner parameters or build queues. Queue encryption remains a separate webhook-only contract. ## Migration phases @@ -540,4 +592,4 @@ For experimental multi-runner v2, global `experimental.ssm.kms_key_id` encrypts A future compute provider must add a typed external namespace and provider leaf, multi-runner normalization and routing, a provider-specific `trust-policy` submodule, runner-config dispatch, and an integration that returns the same nested environment-variable, policy, and resource contract before it can be selected in Terraform. Today the typed schema exposes only `aws.ec2`, so unsupported namespace or provider attributes fail input-schema validation. When another implemented leaf is added, the exact-one selection preconditions will reject runner configs that populate more than one supported compute provider. -A future orchestration provider must add a typed global-default block where shared settings are needed, a typed per-runner-config selector block, runner-config dispatch, a capability adapter for each supported compute provider, provider-grouped outputs, and focused routing and coexistence tests. Once a second typed orchestration provider exists, validation must also reject a runner config that selects more than one provider. +An additional orchestration provider must add a typed global block where shared settings are needed, a typed per-runner selector block, runner-config selection behavior, capability support from each compatible compute provider, grouped outputs, and focused routing and coexistence tests. The existing exact-one validation automatically rejects any runner config that selects more than one supported orchestration provider. diff --git a/docs/scale-set.md b/docs/scale-set.md new file mode 100644 index 0000000000..2969fda990 --- /dev/null +++ b/docs/scale-set.md @@ -0,0 +1,193 @@ +# Runner scale-set controller + +!!! warning "Experimental v2" + + The scale-set controller is wired into the experimental multi-runner v2 interface through `experimental.multi_runner_config`. It is not available through the stable top-level `multi_runner_config`, and its API and state addresses may change while v2 remains experimental. + +The controller is a long-running service because a GitHub runner scale set uses a message session and continuous reconciliation, rather than one Lambda invocation for each webhook event. + +## Prerequisites and ownership + +The provider adopts an existing GitHub runner scale set; it does not discover, create, or delete one. Before deployment, another provisioning step must create the scale set and supply its numeric ID, expected name, and optional runner-group ID. The controller verifies that identity before opening a message session and fails closed on any mismatch. Destroying the ECS controller does not delete the GitHub scale set or its registered runners. + +Controller ownership is identified by the canonical GitHub scope and numeric scale-set ID. Canonicalization ignores URL case, one trailing slash, and the default HTTPS port. Terraform rejects a duplicate tuple within one multi-runner deployment. Operators must also keep that tuple unique across every deployment that can reach the same GitHub scope; separate Terraform states cannot detect one another and competing controllers would contend for the same message session. + +The EC2 compute-provider module emits its non-secret runtime configuration and scoped task-role capability through `provider.capabilities.scale_set`. Runner-config exposes the selected compute capability, and multi-runner aggregates all scale-set selections into one orchestration-provider call before resolving controller groups. These contracts remain experimental rather than stable standalone module interfaces. + +Before publishing a release that relies on the default GHCR image, verify that an unauthenticated client can pull the package. Repository-linked package visibility can inherit organization settings, so this must be checked from the published package rather than assumed from the workflow result. + +## Experimental v2 contract + +The global `experimental.orchestration_provider.scale_set` block configures the shared controller topology. Each runner config selects `orchestration_provider.scale_set` separately and supplies the existing GitHub scale-set identity and its installation-ID Parameter Store reference: + +```hcl +module "multi_runner" { + source = "github-aws-runners/github-runner/aws//modules/multi-runner" + + prefix = "example" + + experimental = { + github = { + app = var.github_app + } + + orchestration_provider = { + scale_set = { + network = { + vpc_id = var.vpc_id + subnet_ids = var.controller_subnet_ids + } + } + } + + multi_runner_config = { + linux = { + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example" + installation_id_ssm = { + name = var.github_installation_id_parameter_name + arn = var.github_installation_id_parameter_arn + kms_key_arn = var.github_installation_id_kms_key_arn + } + } + name = "linux-runners" + id = 101 + min_runners = 0 + max_runners = 20 + } + } + + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7g.large"] + } + } + } + } + } + } +} +``` + +The controller always uses the primary App ID and private-key references derived from `experimental.github.app`; it does not select an entry from `experimental.github.additional_apps`. Each runner config supplies its own `github.installation_id_ssm` reference because the primary App can have a different installation in each organization or repository. Terraform and the reconciler manifests carry only Parameter Store names and ARNs, never App credentials or installation-ID values. + +`experimental.github.enterprise_server.ssl_verify` is serialized into every scale-set reconciler. A false value applies only to that reconciler's GitHub App and scale-set HTTP clients, so verified and self-signed GHES configurations can share one grouped task without changing process-global TLS behavior. `experimental.github.user_agent` is retained as the `system` identity inside GitHub's structured scale-set protocol User-Agent; it does not replace the protocol header. + +## Deployment model + +Multi-runner invokes `module.orchestration_scale_set[0]` once when at least one v2 runner config selects scale-set orchestration. That aggregate provider call packs the selected reconcilers into one or more controller groups. Every **controller group** has the same deployment shape: + +```mermaid +flowchart TD + Service["ECS service
desired count: 1"] --> Task["One Fargate task"] + Task --> Container["One application container"] + Container --> Controller["One ScaleSetController"] + Controller --> A["Reconciler A
scale set A / session A"] + Controller --> B["Reconciler B
scale set B / session B"] + Controller --> N["Reconciler N
scale set N / session N"] + A --> ComputeA["Selected compute-provider instance"] + B --> ComputeB["Selected compute-provider instance"] + N --> ComputeN["Selected compute-provider instance"] +``` + +The terms have distinct meanings: + +- An **ECS service** keeps the group's desired task count at one and rolls task-definition revisions. +- A **task definition** is the immutable template for the container, IAM roles, health check, logs, and runtime settings. +- A **running task** is one deployment of that template. +- The task contains one **controller process**. +- The controller runs one independent **reconciler** and GitHub message session for every scale set assigned to the group. + +A controller group is a packing, IAM, deployment, and failure boundary. It is not a GitHub runner group. Scaling an ECS service above one would create competing sessions for the same scale sets, so the module fixes the desired count at one. + +Deployments are stop-first (`minimumHealthyPercent = 0`, `maximumPercent = 100`). This avoids overlapping the old and new message-session owners. It creates a short control-plane interruption during rollout; unacknowledged messages remain available for the replacement controller. + +## Grouping strategies + +The same runtime supports three grouping strategies. Every runner config must belong to exactly one group. + +| Strategy | Behavior | Advantages | Costs | +| --- | --- | --- | --- | +| `compute_provider` | One group for each compute-provider type. This is the default. | Fewer ECS services and tasks; a natural default for provider-specific IAM and dependencies. | Configs of the same provider share rollout and process blast radius; the task role contains the union of their permissions. | +| `runner_config` | One group for every runner config. | Maximum isolation for IAM, health, logs, and deployment. | One ECS service, task definition, ENI, log group, and running-task cost per config. | +| `custom` | Explicit groups map to explicit runner-config sets. | Isolates sensitive or high-volume configs while packing smaller configs together. | The caller owns the grouping design and must account for the union of permissions and aggregate load in each group. | + +Example custom assignment: + +```hcl +grouping = { + strategy = "custom" + custom = { + groups = { + general = { + runner_configs = ["linux-small", "linux-medium"] + } + isolated = { + runner_configs = ["privileged-builds"] + } + microvm = { + runner_configs = ["microvm-small", "microvm-large"] + } + } + } +} +``` + +Future grouping algorithms should only produce the same normalized `group -> runner configs` mapping. The controller and provider contracts do not depend on how that mapping was selected. + +## Runtime configuration and secrets + +The Terraform module writes one non-secret `String` parameter per reconciler below the controller group's Parameter Store path. The ECS task receives only the group name, path, and revision. At startup the service loads the direct children of that path, validates their bounded versioned JSON, and creates the reconcilers. + +The group manifest contains references to the GitHub App parameters, never their values. The task reads the exact referenced parameters and decrypts only their configured KMS keys. GitHub App credentials are refreshed without placing private keys, installation tokens, message-session tokens, messages, or JIT configurations in environment variables or logs. + +Each group gets a separate task role. Its policy is the union of: + +- the group's configuration path; +- the GitHub App parameter and optional KMS ARNs used by its members; and +- the selected scale-set capability fragments supplied by those members' compute providers. + +Grouping therefore changes both the runtime blast radius and IAM scope. Large custom groups can also reach AWS inline-policy or API-rate quotas sooner. + +## Compute-provider contract + +Scale-set support is additive. It uses a separate `ScaleSetComputeProvider` registry instead of adding mandatory methods to the existing Lambda control-plane provider interface. + +For every reconciliation, the controller supplies: + +- the desired runner count; +- the orchestration-owned runner boot timeout; +- exact GitHub runner observations known to the controller; +- an explicit signal distinguishing lifecycle-only observations from a complete joined Actions and public GitHub inventory; +- a callback that creates a JIT configuration for an expected runner name; and +- a callback that removes only an exact runner ID, name, and scale-set match. + +The selected compute provider owns capacity discovery, creation, provider tags/state, JIT publication, and safe termination. It must retain busy, ambiguous, or unknown capacity. This makes a restart fail safe: missing in-memory lifecycle state can leak capacity, but it must not cause a running job to be terminated. + +A provider plugin may also declare bounded, non-secret process environment variables needed by its SDK or runtime adapter. Registration rejects reserved AWS, ECS, GitHub, Node.js, and controller namespaces, invalid names, control characters, oversized values, and conflicting grouped values. Per-runner configuration and all credentials remain in Parameter Store; the EC2 provider currently declares no additional process environment variables. + +The initial EC2 implementation derives ownership from exact provider-created tags, including a SHA-256 hash of the canonical GitHub configuration scope, and never trusts a mutable GitHub runner ID alone. It only scales down an exact, currently idle runner. + +A `config-published` instance counts as serving while it is inside the orchestration-owned boot window (`bootTimeoutMinutes`, default `10`) or after the controller observes an exact online or `JobStarted` identity. Once that window expires, the provider requests one complete inventory pass. Old offline, missing, ambiguous, or otherwise unknown capacity is retained, but no longer suppresses a replacement. The existing runner bootstrap does not provide a transactional "JIT configuration claimed" handshake, so an interrupted `provisioning` or `publishing` instance is also retained for operator recovery rather than terminated optimistically. Reconciliation may use at most one physical instance above desired capacity to restore serving capacity; it stops adding replacements at that limit so persistent ambiguity cannot create unbounded EC2 cost. + +## EC2 lifecycle ownership + +Scale-set-created EC2 instances carry `ghr:created_by=scale-set-service`, and their scale-set reconciler owns discovery, GitHub runner removal, and termination. The unchanged shared termination watcher can match the same environment, so a module instance containing scale-set runner configs rejects `instance_termination_watcher.enable_runner_deregistration = true`. The watcher may remain enabled with deregistration disabled for logging and metrics. Mixed deployments that require classic watcher deregistration need a separate deployment or filtering boundary that cannot match scale-set capacity. + +## Health and operations + +The container exposes two loopback endpoints: + +- `/healthz` is liveness. Transient GitHub or AWS failures leave the process live while reconcilers back off, avoiding ECS restart loops. +- `/readyz` is readiness. It is successful only when every reconciler has an active, progressing session. + +Terraform accepts only `/healthz` for `container.health_path`; `/readyz` cannot be selected as the ECS liveness probe. One reconciler can be degraded without terminating healthy reconcilers in the same group. Structured logs are the externally available signal for that partial failure today. `/readyz` is bound to loopback for a custom same-task observer; it is not currently exported through ECS Exec, a load balancer, or a CloudWatch metric. Graceful shutdown cancels polling and closes every message session within the ECS stop timeout. + +## Container image + +The repository builds `lambdas/services/scale-set/Dockerfile` for `linux/amd64` and `linux/arm64`. Releases publish the official image with an SBOM, provenance, and an attestation. + +The Terraform module has a convenience `:latest` default and enables ECS image-version consistency. Production deployments should set `container.image` to the immutable digest printed in the release notes. The official GHCR package must be public for anonymous Fargate pulls; private ECR overrides must also provide the repository ARN so the execution role can receive the required pull permissions. diff --git a/mkdocs.yaml b/mkdocs.yaml index d974019b1b..acfc3b03bf 100644 --- a/mkdocs.yaml +++ b/mkdocs.yaml @@ -57,6 +57,7 @@ nav: - Configuration: configuration.md - Getting started: getting-started.md - Security: security.md + - Runner scale sets: scale-set.md - Architecture decisions: - ADR-002 Runner orchestration provider boundary: adr/002-runner-orchestration-provider-boundary.md - Modules: diff --git a/modules/compute-providers/aws/ec2/README.md b/modules/compute-providers/aws/ec2/README.md index 435623f6be..7ebcc2e192 100644 --- a/modules/compute-providers/aws/ec2/README.md +++ b/modules/compute-providers/aws/ec2/README.md @@ -4,6 +4,8 @@ This internal module owns the EC2 compute implementation used by the common runn The module returns one nested `provider` contract. It groups EC2-specific Lambda settings under `environment_variables`, permission requirements under `policies.runner`, `policies.scale_up`, `policies.scale_down`, and `policies.pool`, and EC2 artifacts under `resources`. The parent runner configuration owns the shared runner role, provider-policy attachments, Lambda functions, execution roles, schedules, queues, retry flow, and SSM housekeeper. +Scale-set support is additive under `provider.capabilities.scale_set`. `configuration_json` contains only resolved, non-secret EC2 launch and JIT Parameter Store settings; GitHub scope, credentials, desired capacity, and boot timeout remain orchestration-owned. `environment_variables` declares provider-owned non-secret process settings and is empty for EC2 today. `iam_statements` grants the controller EC2 discovery and requires the protected application, creator, and environment request tags when launching capacity. It also scopes runner-role pass-through, JIT publication to the runner token path, and termination or retagging of existing instances. The existing runner-role policy separately limits JIT reads and deletion to the calling instance identity. + EC2 is the only active compute provider. The parent runner configuration selects it when `aws.ec2` is the one populated typed leaf under `compute_provider`; no separate namespace or type input is required. Runner-config dispatches this module at `module.compute_aws_ec2[0]` from the `modules/compute-providers/aws/ec2` source and publishes its resources under `provider.aws.ec2`. A future provider must add its own typed namespace and provider leaf and implement the same contracts before it can be selected. @@ -61,7 +63,7 @@ No modules. |------|-------------|------|---------|:--------:| | [aws\_partition](#input\_aws\_partition) | AWS partition used to construct IAM ARNs. | `string` | `"aws"` | no | | [aws\_region](#input\_aws\_region) | AWS region used by compute-provider resources and policy documents. | `string` | n/a | yes | -| [config](#input\_config) | EC2 compute-provider configuration. Paths match `compute_provider.aws.ec2` in the runner configuration.

- `ami`: Optional AMI discovery and encryption configuration. Null selects defaults for `runner.os` and `runner.architecture`.
- `ami.filter`: AMI filter names mapped to accepted values and merged over the provider defaults.
- `ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Its object presence is the plan-time ownership discriminator.
- `ami.id_ssm_parameter.arn`: ARN of the external AMI-ID parameter. The ARN may remain unknown until apply.
- `ami.kms_key`: Optional customer-managed KMS key required for encrypted AMIs or snapshots. Its object presence is the plan-time policy discriminator.
- `ami.kms_key.arn`: ARN of the AMI KMS key. The ARN may remain unknown until apply.
- `vpc_id`: VPC in which runner networking resources are created.
- `subnet_ids`: Subnets from which the control plane may launch runners.
- `overrides.name_runner`: Optional Name tag override for runner compute resources.
- `overrides.name_sg`: Optional Name tag override for the managed security group.
- `instance_profile`: Optional externally managed instance profile. Its object presence is the plan-time ownership discriminator.
- `instance_profile.name`: Name of the external instance profile. The name may remain unknown until apply.
- `instance_profile_path`: IAM path for the provider-managed instance profile. Null derives the path from `prefix`.
- `binaries_syncer.enabled`: Uses the synchronized runner distribution from S3 during bootstrap.
- `binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `binaries_syncer.s3.arn`: Runner-distribution bucket ARN used by IAM policies.
- `binaries_syncer.s3.id`: Runner-distribution bucket name used in the bootstrap URI.
- `binaries_syncer.s3.key`: Runner-distribution object key.
- `block_device_mappings`: EBS mappings added to the launch template.
- `block_device_mappings[].delete_on_termination`: Deletes the volume when its runner terminates.
- `block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `block_device_mappings[].encrypted`: Enables EBS encryption.
- `block_device_mappings[].iops`: Provisioned IOPS for volume types that support configurable IOPS.
- `block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `block_device_mappings[].volume_initialization_rate`: Fixed initialization rate for supported snapshot-backed volumes.
- `block_device_mappings[].volume_size`: EBS volume size in GiB.
- `block_device_mappings[].volume_type`: EBS volume type.
- `ebs_optimized`: Requests EBS-optimized instances.
- `instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `instance_allocation_strategy`: EC2 Fleet allocation strategy.
- `instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `instance_max_spot_price`: Optional maximum hourly Spot price.
- `instance_types`: EC2 instance types available to the control plane.
- `user_data`: Runner bootstrap user-data configuration.
- `user_data.enabled`: Enables launch-template user data.
- `user_data.template`: Optional path to a custom user-data template.
- `user_data.content`: Optional complete user-data content used instead of a template.
- `user_data.pre_install`: Script inserted before runner installation.
- `user_data.post_install`: Script inserted after runner installation.
- `user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets.
- `ssm_enabled`: Includes Session Manager permissions in the provider's runner policy group.
- `create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `cloudwatch_agent.enabled`: Enables CloudWatch agent configuration for runner instances.
- `cloudwatch_agent.config`: Optional complete CloudWatch agent configuration.
- `managed_security_group_enabled`: Creates and attaches the provider-managed security group.
- `log_files`: Optional files collected by the CloudWatch agent. Null uses provider defaults.
- `log_files[].log_group_name`: CloudWatch log-group name before optional prefixing.
- `log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path.
- `log_files[].file_path`: File or glob read by the CloudWatch agent.
- `log_files[].log_stream_name`: CloudWatch log-stream name template.
- `log_files[].log_class`: CloudWatch log-group class for the collected file.
- `key_name`: Optional EC2 key-pair name.
- `additional_security_group_ids`: Existing security groups attached to runners.
- `detailed_monitoring_enabled`: Enables detailed EC2 monitoring.
- `egress_rules`: Rules created on the managed security group.
- `egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `egress_rules[].from_port`: First destination port in the permitted range.
- `egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `egress_rules[].security_groups`: Destination security-group IDs.
- `egress_rules[].self`: Allows traffic to the managed security group itself.
- `egress_rules[].to_port`: Last destination port in the permitted range.
- `egress_rules[].description`: Optional rule description.
- `tags`: Runner instance, volume, network-interface, and eligible Spot-request tags. Provider-required bootstrap tags take final precedence.
- `metadata_options`: Instance Metadata Service configuration.
- `metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when enabled.
- `metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `credit_specification`: CPU credit mode for burstable instance types.
- `cpu_options`: CPU topology and processor-feature configuration.
- `cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `placement`: EC2 placement configuration.
- `placement.affinity`: Dedicated Host affinity setting.
- `placement.availability_zone`: Availability Zone in which runner instances are placed.
- `placement.group_id`: Placement-group ID.
- `placement.group_name`: Placement-group name.
- `placement.host_id`: Dedicated Host ID.
- `placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `placement.spread_domain`: Spread-domain placement value.
- `placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `placement.partition_number`: Placement-group partition number.
- `license_specifications`: License Manager configurations added to the launch template.
- `license_specifications[].license_configuration_arn`: ARN of an AWS License Manager license configuration.
- `associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `enable_on_demand_failover_for_errors`: EC2 errors that trigger on-demand fallback after a Spot failure.
- `scale_errors`: EC2 errors treated as retryable scale-up failures.
- `use_dedicated_host`: Enables the dedicated-host launch path. | 
object({
    ami = optional(object({
      filter = optional(map(list(string)), { state = ["available"] })
      owners = optional(list(string), ["amazon"])
      id_ssm_parameter = optional(object({
        arn = string
      }), null)
      kms_key = optional(object({
        arn = string
      }), null)
    }), null)
    vpc_id     = string
    subnet_ids = list(string)
    overrides = optional(object({
      name_runner = optional(string, "")
      name_sg     = optional(string, "")
    }), {})
    instance_profile = optional(object({
      name = string
    }), null)
    instance_profile_path = optional(string, null)
    binaries_syncer = optional(object({
      enabled = optional(bool, true)
      s3 = optional(object({
        arn = string
        id  = string
        key = string
      }), null)
    }), {})
    block_device_mappings = optional(list(object({
      delete_on_termination      = optional(bool, true)
      device_name                = optional(string, "/dev/xvda")
      encrypted                  = optional(bool, true)
      iops                       = optional(number)
      kms_key_id                 = optional(string)
      snapshot_id                = optional(string)
      throughput                 = optional(number)
      volume_initialization_rate = optional(number)
      volume_size                = number
      volume_type                = optional(string, "gp3")
    })), [{ volume_size = 30 }])
    ebs_optimized                 = optional(bool, false)
    instance_target_capacity_type = optional(string, "spot")
    instance_allocation_strategy  = optional(string, "lowest-price")
    instance_type_priorities      = optional(map(number), null)
    instance_max_spot_price       = optional(string, null)
    instance_types                = list(string)
    user_data = optional(object({
      enabled               = optional(bool, true)
      template              = optional(string, null)
      content               = optional(string, null)
      pre_install           = optional(string, "")
      post_install          = optional(string, "")
      debug_logging_enabled = optional(bool, false)
    }), {})
    ssm_enabled                     = optional(bool, false)
    create_service_linked_role_spot = optional(bool, false)
    cloudwatch_agent = optional(object({
      enabled = optional(bool, true)
      config  = optional(string, null)
    }), {})
    managed_security_group_enabled = optional(bool, true)
    log_files = optional(list(object({
      log_group_name   = string
      prefix_log_group = bool
      file_path        = string
      log_stream_name  = string
      log_class        = optional(string, "STANDARD")
    })), null)
    key_name                      = optional(string, null)
    additional_security_group_ids = optional(list(string), [])
    detailed_monitoring_enabled   = optional(bool, false)
    egress_rules = optional(list(object({
      cidr_blocks      = list(string)
      ipv6_cidr_blocks = list(string)
      prefix_list_ids  = list(string)
      from_port        = number
      protocol         = string
      security_groups  = list(string)
      self             = bool
      to_port          = number
      description      = string
      })), [{
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = ["::/0"]
      prefix_list_ids  = null
      from_port        = 0
      protocol         = "-1"
      security_groups  = null
      self             = null
      to_port          = 0
      description      = null
    }])
    tags = optional(map(string), {})
    metadata_options = optional(object({
      instance_metadata_tags      = optional(string, "enabled")
      http_endpoint               = optional(string, "enabled")
      http_tokens                 = optional(string, "required")
      http_put_response_hop_limit = optional(number, 1)
    }), {})
    credit_specification = optional(string, null)
    cpu_options = optional(object({
      core_count            = optional(number)
      threads_per_core      = optional(number)
      amd_sev_snp           = optional(string)
      nested_virtualization = optional(string)
    }), null)
    placement = optional(object({
      affinity                = optional(string)
      availability_zone       = optional(string)
      group_id                = optional(string)
      group_name              = optional(string)
      host_id                 = optional(string)
      host_resource_group_arn = optional(string)
      spread_domain           = optional(string)
      tenancy                 = optional(string)
      partition_number        = optional(number)
    }), null)
    license_specifications = optional(list(object({
      license_configuration_arn = string
    })), [])
    associate_public_ipv4_address        = optional(bool, false)
    enable_on_demand_failover_for_errors = optional(list(string), [])
    scale_errors = optional(list(string), [
      "UnfulfillableCapacity",
      "MaxSpotInstanceCountExceeded",
      "TargetCapacityLimitExceededException",
      "RequestLimitExceeded",
      "ResourceLimitExceeded",
      "MaxSpotInstanceCountExceeded",
      "MaxSpotFleetRequestCountExceeded",
      "InsufficientInstanceCapacity",
      "InsufficientCapacityOnHost",
    ])
    use_dedicated_host = optional(bool, false)
  })
| n/a | yes | +| [config](#input\_config) | EC2 compute-provider configuration. Paths match `compute_provider.aws.ec2` in the runner configuration.

- `ami`: Optional AMI discovery and encryption configuration. Null selects defaults for `runner.os` and `runner.architecture`.
- `ami.filter`: AMI filter names mapped to accepted values and merged over the provider defaults.
- `ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Its object presence is the plan-time ownership discriminator.
- `ami.id_ssm_parameter.arn`: ARN of the external AMI-ID parameter. The ARN may remain unknown until apply.
- `ami.kms_key`: Optional customer-managed KMS key required for encrypted AMIs or snapshots. Its object presence is the plan-time policy discriminator.
- `ami.kms_key.arn`: ARN of the AMI KMS key. The ARN may remain unknown until apply.
- `vpc_id`: VPC in which runner networking resources are created.
- `subnet_ids`: Subnets from which the control plane may launch runners.
- `overrides.name_runner`: Optional Name tag override for runner compute resources.
- `overrides.name_sg`: Optional Name tag override for the managed security group.
- `instance_profile`: Optional externally managed instance profile. Its object presence is the plan-time ownership discriminator.
- `instance_profile.name`: Name of the external instance profile. The name may remain unknown until apply.
- `instance_profile_path`: IAM path for the provider-managed instance profile. Null derives the path from `prefix`.
- `binaries_syncer.enabled`: Uses the synchronized runner distribution from S3 during bootstrap.
- `binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `binaries_syncer.s3.arn`: Runner-distribution bucket ARN used by IAM policies.
- `binaries_syncer.s3.id`: Runner-distribution bucket name used in the bootstrap URI.
- `binaries_syncer.s3.key`: Runner-distribution object key.
- `block_device_mappings`: EBS mappings added to the launch template.
- `block_device_mappings[].delete_on_termination`: Deletes the volume when its runner terminates.
- `block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `block_device_mappings[].encrypted`: Enables EBS encryption.
- `block_device_mappings[].iops`: Provisioned IOPS for volume types that support configurable IOPS.
- `block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `block_device_mappings[].volume_initialization_rate`: Fixed initialization rate for supported snapshot-backed volumes.
- `block_device_mappings[].volume_size`: EBS volume size in GiB.
- `block_device_mappings[].volume_type`: EBS volume type.
- `ebs_optimized`: Requests EBS-optimized instances.
- `instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `instance_allocation_strategy`: EC2 Fleet allocation strategy.
- `instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `instance_max_spot_price`: Optional maximum hourly Spot price.
- `instance_types`: EC2 instance types available to the control plane.
- `user_data`: Runner bootstrap user-data configuration.
- `user_data.enabled`: Enables launch-template user data.
- `user_data.template`: Optional path to a custom user-data template.
- `user_data.content`: Optional complete user-data content used instead of a template.
- `user_data.pre_install`: Script inserted before runner installation.
- `user_data.post_install`: Script inserted after runner installation.
- `user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets.
- `ssm_enabled`: Includes Session Manager permissions in the provider's runner policy group.
- `create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `cloudwatch_agent.enabled`: Enables CloudWatch agent configuration for runner instances.
- `cloudwatch_agent.config`: Optional complete CloudWatch agent configuration.
- `managed_security_group_enabled`: Creates and attaches the provider-managed security group.
- `log_files`: Optional files collected by the CloudWatch agent. Null uses provider defaults.
- `log_files[].log_group_name`: CloudWatch log-group name before optional prefixing.
- `log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path.
- `log_files[].file_path`: File or glob read by the CloudWatch agent.
- `log_files[].log_stream_name`: CloudWatch log-stream name template.
- `log_files[].log_class`: CloudWatch log-group class for the collected file.
- `key_name`: Optional EC2 key-pair name.
- `additional_security_group_ids`: Existing security groups attached to runners.
- `detailed_monitoring_enabled`: Enables detailed EC2 monitoring.
- `egress_rules`: Rules created on the managed security group.
- `egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `egress_rules[].from_port`: First destination port in the permitted range.
- `egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `egress_rules[].security_groups`: Destination security-group IDs.
- `egress_rules[].self`: Allows traffic to the managed security group itself.
- `egress_rules[].to_port`: Last destination port in the permitted range.
- `egress_rules[].description`: Optional rule description.
- `tags`: Runner instance, volume, network-interface, and eligible Spot-request tags. Provider-required bootstrap tags take final precedence.
- `metadata_options`: Instance Metadata Service configuration.
- `metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when enabled.
- `metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `credit_specification`: CPU credit mode for burstable instance types.
- `cpu_options`: CPU topology and processor-feature configuration.
- `cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `placement`: EC2 placement configuration.
- `placement.affinity`: Dedicated Host affinity setting.
- `placement.availability_zone`: Availability Zone in which runner instances are placed.
- `placement.group_id`: Placement-group ID.
- `placement.group_name`: Placement-group name.
- `placement.host_id`: Dedicated Host ID.
- `placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `placement.spread_domain`: Spread-domain placement value.
- `placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `placement.partition_number`: Placement-group partition number.
- `license_specifications`: License Manager configurations added to the launch template.
- `license_specifications[].license_configuration_arn`: ARN of an AWS License Manager license configuration.
- `associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `enable_on_demand_failover_for_errors`: EC2 errors that trigger on-demand fallback after a Spot failure.
- `scale_errors`: EC2 errors treated as retryable scale-up failures.
- `use_dedicated_host`: Enables the dedicated-host launch path. | 
object({
    ami = optional(object({
      filter = optional(map(list(string)), { state = ["available"] })
      owners = optional(list(string), ["amazon"])
      id_ssm_parameter = optional(object({
        arn = string
      }), null)
      kms_key = optional(object({
        arn = string
      }), null)
    }), null)
    vpc_id     = string
    subnet_ids = list(string)
    overrides = optional(object({
      name_runner = optional(string, "")
      name_sg     = optional(string, "")
    }), {})
    instance_profile = optional(object({
      name = string
    }), null)
    instance_profile_path = optional(string, null)
    binaries_syncer = optional(object({
      enabled = optional(bool, true)
      s3 = optional(object({
        arn = string
        id  = string
        key = string
      }), null)
    }), {})
    block_device_mappings = optional(list(object({
      delete_on_termination      = optional(bool, true)
      device_name                = optional(string, "/dev/xvda")
      encrypted                  = optional(bool, true)
      iops                       = optional(number)
      kms_key_id                 = optional(string)
      snapshot_id                = optional(string)
      throughput                 = optional(number)
      volume_initialization_rate = optional(number)
      volume_size                = number
      volume_type                = optional(string, "gp3")
    })), [{ volume_size = 30 }])
    ebs_optimized                 = optional(bool, false)
    instance_target_capacity_type = optional(string, "spot")
    instance_allocation_strategy  = optional(string, "lowest-price")
    instance_type_priorities      = optional(map(number), null)
    instance_max_spot_price       = optional(string, null)
    instance_types                = list(string)
    user_data = optional(object({
      enabled               = optional(bool, true)
      template              = optional(string, null)
      content               = optional(string, null)
      pre_install           = optional(string, "")
      post_install          = optional(string, "")
      debug_logging_enabled = optional(bool, false)
    }), {})
    ssm_enabled                     = optional(bool, false)
    create_service_linked_role_spot = optional(bool, false)
    cloudwatch_agent = optional(object({
      enabled = optional(bool, true)
      config  = optional(string, null)
    }), {})
    managed_security_group_enabled = optional(bool, true)
    log_files = optional(list(object({
      log_group_name   = string
      prefix_log_group = bool
      file_path        = string
      log_stream_name  = string
      log_class        = optional(string, "STANDARD")
    })), null)
    key_name                      = optional(string, null)
    additional_security_group_ids = optional(list(string), [])
    detailed_monitoring_enabled   = optional(bool, false)
    egress_rules = optional(list(object({
      cidr_blocks      = list(string)
      ipv6_cidr_blocks = list(string)
      prefix_list_ids  = list(string)
      from_port        = number
      protocol         = string
      security_groups  = list(string)
      self             = bool
      to_port          = number
      description      = string
      })), [{
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = ["::/0"]
      prefix_list_ids  = null
      from_port        = 0
      protocol         = "-1"
      security_groups  = null
      self             = null
      to_port          = 0
      description      = null
    }])
    tags = optional(map(string), {})
    metadata_options = optional(object({
      instance_metadata_tags      = optional(string, "enabled")
      http_endpoint               = optional(string, "enabled")
      http_tokens                 = optional(string, "required")
      http_put_response_hop_limit = optional(number, 1)
    }), {})
    credit_specification = optional(string, null)
    cpu_options = optional(object({
      core_count            = optional(number)
      threads_per_core      = optional(number)
      amd_sev_snp           = optional(string)
      nested_virtualization = optional(string)
    }), null)
    placement = optional(object({
      affinity                = optional(string)
      availability_zone       = optional(string)
      group_id                = optional(string)
      group_name              = optional(string)
      host_id                 = optional(string)
      host_resource_group_arn = optional(string)
      spread_domain           = optional(string)
      tenancy                 = optional(string)
      partition_number        = optional(number)
    }), null)
    license_specifications = optional(list(object({
      license_configuration_arn = string
    })), [])
    associate_public_ipv4_address        = optional(bool, false)
    enable_on_demand_failover_for_errors = optional(list(string), [])
    scale_errors = optional(list(string), [
      "UnfulfillableCapacity",
      "MaxSpotInstanceCountExceeded",
      "TargetCapacityLimitExceededException",
      "RequestLimitExceeded",
      "ResourceLimitExceeded",
      "MaxSpotFleetRequestCountExceeded",
      "InsufficientInstanceCapacity",
      "InsufficientCapacityOnHost",
    ])
    use_dedicated_host = optional(bool, false)
  })
| n/a | yes | | [github](#input\_github) | GitHub Enterprise Server settings available to compute-provider bootstrap data.

- `enterprise_server.url`: Optional GitHub Enterprise Server base URL. Null selects GitHub.com.
- `enterprise_server.ssl_verify`: Enables TLS certificate verification for GitHub Enterprise Server. | 
object({
    enterprise_server = optional(object({
      url        = optional(string, null)
      ssl_verify = optional(bool, true)
    }), {})
  })
| `{}` | no | | [observability](#input\_observability) | CloudWatch Logs settings available to compute-provider runner log groups.

- `logs.retention_in_days`: Retention period for provider-owned runner log groups.
- `logs.kms_key_id`: Optional KMS key ID or ARN used to encrypt runner log groups.
- `logs.tags`: Shared log-group tags that override module-level `tags`. | 
object({
    logs = optional(object({
      retention_in_days = optional(number, 180)
      kms_key_id        = optional(string, null)
      tags              = optional(map(string), {})
    }), {})
  })
| `{}` | no | | [prefix](#input\_prefix) | Prefix used to identify resources created for the runner configuration. | `string` | `"github-actions"` | no | diff --git a/modules/compute-providers/aws/ec2/outputs.tf b/modules/compute-providers/aws/ec2/outputs.tf index 422383df0f..83b2647fca 100644 --- a/modules/compute-providers/aws/ec2/outputs.tf +++ b/modules/compute-providers/aws/ec2/outputs.tf @@ -16,6 +16,8 @@ output "resources" { output "provider" { description = "Nested EC2 compute-provider contract consumed by runner-config." value = { + type = "ec2" + capabilities = { scale_set = local.scale_set_capability } environment_variables = local.provider_environment_variables policies = local.provider_policies resources = local.provider_resources diff --git a/modules/compute-providers/aws/ec2/scale-set.tf b/modules/compute-providers/aws/ec2/scale-set.tf new file mode 100644 index 0000000000..06cf84bb8a --- /dev/null +++ b/modules/compute-providers/aws/ec2/scale-set.tf @@ -0,0 +1,255 @@ +# Provider-owned runtime and IAM fragments for the additive scale-set +# orchestration capability. GitHub credentials, GitHub scope, desired capacity, +# and boot timeout remain orchestration-owned and are not serialized here. +locals { + scale_set_ec2_instance_criteria = merge( + { + instanceTypes = var.config.instance_types + targetCapacityType = var.config.instance_target_capacity_type + instanceAllocationStrategy = var.config.instance_allocation_strategy + }, + var.config.instance_type_priorities == null ? {} : { + instanceTypePriorities = var.config.instance_type_priorities + }, + var.config.instance_max_spot_price == null ? {} : { + maxSpotPrice = var.config.instance_max_spot_price + }, + ) + + scale_set_runtime_configuration = merge( + { + region = var.aws_region + environment = var.prefix + runnerNamePrefix = var.runner.name_prefix + jitConfigParameterPath = "${var.ssm.paths.root}/${var.ssm.paths.tokens}" + subnets = var.config.subnet_ids + launchTemplateName = aws_launch_template.runner.name + ec2instanceCriteria = local.scale_set_ec2_instance_criteria + onDemandFailoverOnError = var.config.enable_on_demand_failover_for_errors + scaleErrors = var.config.scale_errors + useDedicatedHost = var.config.use_dedicated_host + ssmParameterTags = [ + for key in sort(keys(local.ssm_parameter_tags)) : { + Key = key + Value = local.ssm_parameter_tags[key] + } + ] + }, + local.ami_id_ssm_external ? { + amiIdSsmParameterName = local.ami_id_ssm_parameter_name + } : {}, + ) + + scale_set_owned_instance_conditions = [ + { + test = "StringEquals" + variable = "ec2:ResourceTag/ghr:Application" + values = toset(["github-action-runner"]) + }, + { + test = "StringEquals" + variable = "ec2:ResourceTag/ghr:created_by" + values = toset(["scale-set-service"]) + }, + { + test = "StringEquals" + variable = "ec2:ResourceTag/ghr:environment" + values = toset([var.prefix]) + }, + ] + + scale_set_owned_request_conditions = [ + { + test = "StringEquals" + variable = "aws:RequestTag/ghr:Application" + values = toset(["github-action-runner"]) + }, + { + test = "StringEquals" + variable = "aws:RequestTag/ghr:created_by" + values = toset(["scale-set-service"]) + }, + { + test = "StringEquals" + variable = "aws:RequestTag/ghr:environment" + values = toset([var.prefix]) + }, + ] + + scale_set_launch_dependency_resources = toset(concat( + [ + "arn:${var.aws_partition}:ec2:${var.aws_region}::image/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:*:snapshot/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:dedicated-host/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:network-interface/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:placement-group/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:security-group/*", + aws_launch_template.runner.arn, + ], + [ + for subnet_id in var.config.subnet_ids : + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:subnet/${subnet_id}" + ], + var.config.key_name == null ? [] : [ + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:key-pair/${var.config.key_name}", + ], + )) + + scale_set_create_fleet_dependency_resources = toset(concat( + [ + "arn:${var.aws_partition}:ec2:${var.aws_region}::image/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:placement-group/*", + aws_launch_template.runner.arn, + ], + [ + for subnet_id in var.config.subnet_ids : + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:subnet/${subnet_id}" + ], + )) + + scale_set_iam_statements = merge( + { + describe_ec2 = { + actions = toset([ + "ec2:DescribeInstances", + "ec2:DescribeLaunchTemplateVersions", + "ec2:DescribeTags", + ]) + # These EC2 Describe APIs do not support resource-level permissions. + resources = toset(["*"]) + conditions = [] + } + create_fleet_dependencies = { + actions = toset(["ec2:CreateFleet"]) + resources = local.scale_set_create_fleet_dependency_resources + conditions = [] + } + create_owned_fleet_capacity = { + actions = toset(["ec2:CreateFleet"]) + resources = toset([ + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:fleet/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:volume/*", + ]) + conditions = local.scale_set_owned_request_conditions + } + run_instances_dependencies = { + actions = toset(["ec2:RunInstances"]) + resources = local.scale_set_launch_dependency_resources + conditions = [] + } + run_owned_instances = { + actions = toset(["ec2:RunInstances"]) + resources = toset([ + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:volume/*", + ]) + conditions = local.scale_set_owned_request_conditions + } + tag_runners_on_create = { + actions = toset(["ec2:CreateTags"]) + resources = toset(["arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:*/*"]) + conditions = [ + { + test = "StringEquals" + variable = "ec2:CreateAction" + values = toset(["CreateFleet", "RunInstances"]) + }, + ] + } + update_owned_runner_tags = { + actions = toset(["ec2:CreateTags"]) + resources = toset(["arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*"]) + conditions = concat(local.scale_set_owned_instance_conditions, [ + { + test = "ForAllValues:StringEquals" + variable = "aws:TagKeys" + values = toset([ + "ghr:github_runner_id", + "ghr:runner_name", + "ghr:scale_set_state", + ]) + }, + ]) + } + terminate_owned_runners = { + actions = toset(["ec2:TerminateInstances"]) + resources = toset(["arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*"]) + conditions = local.scale_set_owned_instance_conditions + } + pass_runner_role = { + actions = toset(["iam:PassRole"]) + resources = toset([var.runner.iam.role.arn]) + conditions = [ + { + test = "StringEquals" + variable = "iam:PassedToService" + values = toset(["ec2.amazonaws.com"]) + }, + ] + } + publish_runner_jit_configuration = { + actions = toset([ + "ssm:AddTagsToResource", + "ssm:DeleteParameter", + "ssm:PutParameter", + ]) + resources = toset([ + "${local.ssm_parameter_arn_prefix}${var.ssm.paths.root}/${var.ssm.paths.tokens}/*", + ]) + conditions = [] + } + }, + local.ami_id_ssm_external ? { + read_external_ami_parameter = { + actions = toset(["ssm:GetParameter"]) + resources = toset([local.ami_id_ssm_parameter_arn]) + conditions = [] + } + } : {}, + local.ami_kms_key_enabled ? { + use_ami_kms_key = { + actions = toset([ + "kms:Decrypt", + "kms:DescribeKey", + "kms:ReEncryptFrom", + "kms:ReEncryptTo", + ]) + resources = toset([local.ami_kms_key_arn]) + conditions = [] + } + create_ami_kms_grant = { + actions = toset(["kms:CreateGrant"]) + resources = toset([local.ami_kms_key_arn]) + conditions = [ + { + test = "Bool" + variable = "kms:GrantIsForAWSResource" + values = toset(["true"]) + }, + ] + } + } : {}, + var.config.create_service_linked_role_spot ? { + create_spot_service_linked_role = { + actions = toset(["iam:CreateServiceLinkedRole"]) + resources = toset([ + "arn:${var.aws_partition}:iam::${data.aws_caller_identity.current.account_id}:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot", + ]) + conditions = [ + { + test = "StringEquals" + variable = "iam:AWSServiceName" + values = toset(["spot.amazonaws.com"]) + }, + ] + } + } : {}, + ) + + scale_set_capability = { + configuration_json = jsonencode(local.scale_set_runtime_configuration) + environment_variables = {} + iam_statements = local.scale_set_iam_statements + } +} diff --git a/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl b/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl index bc92537279..55775ca293 100644 --- a/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl +++ b/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl @@ -91,8 +91,112 @@ run "separates_control_plane_contract_from_ec2_resources" { command = plan assert { - condition = toset(keys(output.provider)) == toset(["environment_variables", "policies", "resources"]) - error_message = "The EC2 provider contract must expose only integration and resource data; its module identity must not be repeated in the output." + condition = toset(keys(output.provider)) == toset(["type", "capabilities", "environment_variables", "policies", "resources"]) + error_message = "The EC2 provider contract must preserve webhook fragments and expose its additive typed capabilities." + } + + assert { + condition = output.provider.type == "ec2" + error_message = "The scale-set runtime registry requires the canonical EC2 provider type." + } + + assert { + condition = toset(keys(jsondecode(output.provider.capabilities.scale_set.configuration_json))) == toset([ + "amiIdSsmParameterName", + "ec2instanceCriteria", + "environment", + "jitConfigParameterPath", + "launchTemplateName", + "onDemandFailoverOnError", + "region", + "runnerNamePrefix", + "scaleErrors", + "ssmParameterTags", + "subnets", + "useDedicatedHost", + ]) + error_message = "The EC2 scale-set payload must contain only provider-owned, non-secret runtime configuration." + } + + assert { + condition = ( + jsondecode(output.provider.capabilities.scale_set.configuration_json).runnerNamePrefix == "" + && jsondecode(output.provider.capabilities.scale_set.configuration_json).jitConfigParameterPath == "/github-runner/provider-test/tokens" + && jsondecode(output.provider.capabilities.scale_set.configuration_json).amiIdSsmParameterName == "/github-runner/ami-id" + && jsondecode(output.provider.capabilities.scale_set.configuration_json).ec2instanceCriteria.instanceTypes == ["m5.large"] + && !strcontains(output.provider.capabilities.scale_set.configuration_json, "bootTimeout") + && !strcontains(output.provider.capabilities.scale_set.configuration_json, "privateKey") + && length(output.provider.capabilities.scale_set.environment_variables) == 0 + ) + error_message = "The scale-set capability must route resolved EC2 inputs without orchestration settings, credentials, or process-global overrides." + } + + assert { + condition = ( + output.provider.capabilities.scale_set.iam_statements.pass_runner_role.resources == toset(["arn:aws:iam::123456789012:role/provider-test-runner"]) + && output.provider.capabilities.scale_set.iam_statements.pass_runner_role.conditions[0].variable == "iam:PassedToService" + && output.provider.capabilities.scale_set.iam_statements.publish_runner_jit_configuration.resources == toset(["arn:aws:ssm:eu-west-1:123456789012:parameter/github-runner/provider-test/tokens/*"]) + && output.provider.capabilities.scale_set.iam_statements.terminate_owned_runners.resources == toset(["arn:aws:ec2:eu-west-1:123456789012:instance/*"]) + && toset([ + for condition in output.provider.capabilities.scale_set.iam_statements.terminate_owned_runners.conditions : condition.variable + ]) == toset([ + "ec2:ResourceTag/ghr:Application", + "ec2:ResourceTag/ghr:created_by", + "ec2:ResourceTag/ghr:environment", + ]) + ) + error_message = "The scale-set task IAM contract must scope JIT publication, PassRole, and destructive EC2 operations." + } + + assert { + condition = ( + output.provider.capabilities.scale_set.iam_statements.create_owned_fleet_capacity.resources == toset([ + "arn:aws:ec2:eu-west-1:123456789012:fleet/*", + "arn:aws:ec2:eu-west-1:123456789012:instance/*", + "arn:aws:ec2:eu-west-1:123456789012:volume/*", + ]) + && output.provider.capabilities.scale_set.iam_statements.run_owned_instances.resources == toset([ + "arn:aws:ec2:eu-west-1:123456789012:instance/*", + "arn:aws:ec2:eu-west-1:123456789012:volume/*", + ]) + && toset([ + for condition in output.provider.capabilities.scale_set.iam_statements.create_owned_fleet_capacity.conditions : condition.variable + ]) == toset([ + "aws:RequestTag/ghr:Application", + "aws:RequestTag/ghr:created_by", + "aws:RequestTag/ghr:environment", + ]) + && toset([ + for condition in output.provider.capabilities.scale_set.iam_statements.run_owned_instances.conditions : condition.variable + ]) == toset([ + "aws:RequestTag/ghr:Application", + "aws:RequestTag/ghr:created_by", + "aws:RequestTag/ghr:environment", + ]) + ) + error_message = "Fleet and instance creation must require controller-owned request tags on every created resource." + } + + assert { + condition = ( + contains( + output.provider.capabilities.scale_set.iam_statements.create_fleet_dependencies.resources, + "arn:aws:ec2:eu-west-1:123456789012:subnet/subnet-12345678", + ) + && contains( + output.provider.capabilities.scale_set.iam_statements.run_instances_dependencies.resources, + "arn:aws:ec2:eu-west-1:123456789012:subnet/subnet-12345678", + ) + && !contains( + output.provider.capabilities.scale_set.iam_statements.create_owned_fleet_capacity.resources, + "*", + ) + && !contains( + output.provider.capabilities.scale_set.iam_statements.run_owned_instances.resources, + "*", + ) + ) + error_message = "EC2 launch dependency access must name the selected launch resources and created-capacity statements must not use a global resource wildcard." } assert { @@ -133,6 +237,23 @@ run "separates_control_plane_contract_from_ec2_resources" { error_message = "An external AMI SSM parameter must enable the scale-up managed policy attachment at plan time." } + assert { + condition = ( + contains(flatten([ + for statement in data.aws_iam_policy_document.ssm_parameters.statement : statement.actions + ]), "ssm:DeleteParameter") + && contains(flatten([ + for statement in data.aws_iam_policy_document.ssm_parameters.statement : statement.resources + ]), "arn:aws:ssm:eu-west-1:123456789012:parameter/github-runner/provider-test/tokens/*") + && contains(flatten([ + for statement in data.aws_iam_policy_document.ssm_parameters.statement : [ + for condition in statement.condition : condition.variable + ] + ]), "ec2:SourceInstanceARN") + ) + error_message = "The runner role must read and delete only its own exact JIT parameter path, fenced by source-instance identity." + } + assert { condition = output.provider.policies.pool.managed_policy_enabled error_message = "An external AMI SSM parameter must enable the pool managed policy attachment at plan time." @@ -258,6 +379,44 @@ run "accepts_partial_typed_compute_options" { } } +run "scopes_optional_scale_set_kms_and_service_role_permissions" { + command = plan + + variables { + config = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + instance_types = ["m5.large"] + ami = { + filter = { state = ["available"] } + owners = ["amazon"] + id_ssm_parameter = null + kms_key = { + arn = "arn:aws:kms:eu-west-1:123456789012:key/11111111-2222-3333-4444-555555555555" + } + } + binaries_syncer = { + enabled = false + } + create_service_linked_role_spot = true + } + } + + assert { + condition = ( + output.provider.capabilities.scale_set.iam_statements.use_ami_kms_key.resources == toset(["arn:aws:kms:eu-west-1:123456789012:key/11111111-2222-3333-4444-555555555555"]) + && output.provider.capabilities.scale_set.iam_statements.create_ami_kms_grant.conditions[0].variable == "kms:GrantIsForAWSResource" + && output.provider.capabilities.scale_set.iam_statements.create_spot_service_linked_role.resources == toset(["arn:aws:iam::123456789012:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot"]) + && alltrue(flatten([ + for statement in values(output.provider.capabilities.scale_set.iam_statements) : [ + for action in statement.actions : !strcontains(action, "*") + ] + ])) + ) + error_message = "Optional scale-set KMS and service-linked-role permissions must remain exact and contain no wildcard actions." + } +} + run "separates_provider_runner_and_ssm_tags" { command = plan diff --git a/modules/compute-providers/aws/ec2/variables.tf b/modules/compute-providers/aws/ec2/variables.tf index f538d5026e..349cebd3aa 100644 --- a/modules/compute-providers/aws/ec2/variables.tf +++ b/modules/compute-providers/aws/ec2/variables.tf @@ -251,7 +251,6 @@ variable "config" { "TargetCapacityLimitExceededException", "RequestLimitExceeded", "ResourceLimitExceeded", - "MaxSpotInstanceCountExceeded", "MaxSpotFleetRequestCountExceeded", "InsufficientInstanceCapacity", "InsufficientCapacityOnHost", diff --git a/modules/multi-runner/README.md b/modules/multi-runner/README.md index 6f01f1b345..51c332cf9b 100644 --- a/modules/multi-runner/README.md +++ b/modules/multi-runner/README.md @@ -178,6 +178,7 @@ module "multi-runner" { |------|--------|---------| | [ami\_housekeeper](#module\_ami\_housekeeper) | ../ami-housekeeper | n/a | | [instance\_termination\_watcher](#module\_instance\_termination\_watcher) | ../termination-watcher | n/a | +| [orchestration\_scale\_set](#module\_orchestration\_scale\_set) | ../orchestration-providers/scale-set | n/a | | [runner\_binaries](#module\_runner\_binaries) | ../runner-binaries-syncer | n/a | | [runner\_configs](#module\_runner\_configs) | ../runner-config | n/a | | [runners](#module\_runners) | ../runners | n/a | @@ -215,7 +216,7 @@ module "multi-runner" { | [enable\_ami\_housekeeper](#input\_enable\_ami\_housekeeper) | Option to disable the lambda to clean up old AMIs. | `bool` | `false` | no | | [enable\_managed\_runner\_security\_group](#input\_enable\_managed\_runner\_security\_group) | Enabling the default managed security group creation. Unmanaged security groups can be specified via `runner_additional_security_group_ids`. | `bool` | `true` | no | | [eventbridge](#input\_eventbridge) | Enable the use of EventBridge by the module. By enabling this feature events will be put on the EventBridge by the webhook instead of directly dispatching to queues for scaling. | 
object({
    enable        = optional(bool, true)
    accept_events = optional(list(string), [])
  })
| `{}` | no | -| [experimental](#input\_experimental) | Opt-in experimental features. Omit this object to retain only the stable `multi_runner_config` behavior. Experimental schemas can change before they become stable.

Set `experimental.multi_runner_config` to opt into provider-oriented runner configurations. A non-empty experimental map completely replaces the stable top-level `multi_runner_config`; an empty map keeps stable entries on the unchanged `runners` module.

Sibling blocks provide global v2 defaults. Global and per-configuration values use `configuration override > experimental global v2 default` precedence. Migrated v2 consumers do not inherit a matching flat module input. Singleton shared components consume the global translated values documented below; per-configuration overrides affect only their runner configuration. Component-specific inputs without a nested counterpart continue consuming their existing flat inputs. Tag maps merge from broad to narrow instead of replacing the broader map.
Nullable per-configuration values inherit their experimental global value, so only defaults that apply to every runner configuration should be placed in a global block. When a runner configuration selects an external `runner.iam.role`, inherited managed policies and additional trust policy JSON are intentionally suppressed because this module does not manage that role.
Plan-shaping ownership wrappers remain nullable where their object presence controls Terraform graph shape. `ssm.kms_key_id` is instead an ARN-valued scalar; provider-owned IAM omits null KMS statements while still accepting an ARN whose value is unknown until apply. The unchanged shared webhook retains its legacy policy handling.

Global experimental fields support the following nested properties. These values apply directly when `experimental.multi_runner_config` is non-empty; in stable mode, flat inputs are translated into the same global shape.

- `tags`: Base tags for v2 build queues, runner configurations, the shared GitHub App Parameter Store module, webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `{}`.
- `roles.path`: Default IAM path for module-managed v2 runner and Lambda roles and for the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is null.
- `roles.permissions_boundary`: Default permissions-boundary ARN for module-managed v2 runner and Lambda roles and for the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is null.
- `runner.os`: Default runner operating system. The default is null; every runner configuration must resolve this field globally or locally.
- `runner.architecture`: Default runner distribution architecture. The default is null; every runner configuration must resolve this field globally or locally.
- `runner.disable_default_labels`: Omits the default self-hosted, operating-system, and architecture labels when true. The default is `false`.
- `runner.extra_labels`: Default additional labels combined with each runner configuration's matcher labels. The default is `[]`.
- `runner.group_name`: Default GitHub runner group. The default is `Default`.
- `runner.name_prefix`: Default prefix added to registered runner names. The default is an empty string.
- `runner.run_as_root`: Runs the runner service as root when supported by the provider. The default is `false`.
- `runner.run_as`: Default operating-system user when `run_as_root` is false. The default is `ec2-user`.
- `runner.auto_update_disabled`: Disables the GitHub runner application's built-in updater. The default is `false`.
- `runner.tags`: Default tags for common runner resources, currently the module-managed runner IAM role. The default is `{}`.
- `runner.hooks.job_started`: Default script content installed as the runner job-started hook. The default is an empty string.
- `runner.hooks.job_completed`: Default script content installed as the runner job-completed hook. The default is an empty string.
- `runner.iam.role`: Optional externally managed runner-role wrapper. The default is null; wrapper presence selects external role ownership during planning.
- `runner.iam.role.arn`: ARN of the externally managed runner role.
- `runner.iam.managed_policy_arns`: Named managed-policy ARNs attached to module-managed runner roles. The default is `{}`. Keep this map empty when the global `runner.iam.role` selects an external role; a runner configuration that explicitly selects its own external role suppresses the inherited map.
- `runner.iam.additional_trust_policy_json`: Optional IAM policy document merged with the provider trust policy for module-managed runner roles. The default is null. Keep it null when the global `runner.iam.role` selects an external role; a runner configuration that explicitly selects its own external role suppresses the inherited value.
- `runner.iam.path`: Runner-role IAM path. The default is null, which falls back to `roles.path`.
- `runner.iam.permissions_boundary`: Runner-role permissions-boundary ARN. The default is null, which falls back to `roles.permissions_boundary`.
- `github.app`: Primary GitHub App credentials persisted or selected by the shared Parameter Store module and used by v2 runner configurations. The default is null, but a non-empty v2 map requires this object.
- `github.app.key_base64`: Base64-encoded GitHub App private key supplied directly.
- `github.app.key_base64_ssm`: Existing Parameter Store private-key parameter wrapper. Set this or `key_base64`.
- `github.app.key_base64_ssm.arn`: ARN of the existing private-key parameter.
- `github.app.key_base64_ssm.name`: Name of the existing private-key parameter.
- `github.app.id`: GitHub App ID supplied directly.
- `github.app.id_ssm`: Existing Parameter Store app-ID parameter wrapper. Set this or `id`.
- `github.app.id_ssm.arn`: ARN of the existing app-ID parameter.
- `github.app.id_ssm.name`: Name of the existing app-ID parameter.
- `github.app.webhook_secret`: GitHub App webhook secret supplied directly.
- `github.app.webhook_secret_ssm`: Existing Parameter Store webhook-secret parameter wrapper. Set this or `webhook_secret`.
- `github.app.webhook_secret_ssm.arn`: ARN of the existing webhook-secret parameter.
- `github.app.webhook_secret_ssm.name`: Name of the existing webhook-secret parameter.
- `github.additional_apps`: Additional GitHub App credentials persisted or selected by the shared Parameter Store module and used for API request distribution. The default is `[]`.
- `github.additional_apps[].key_base64`: Base64-encoded private key supplied directly for an additional app.
- `github.additional_apps[].key_base64_ssm`: Existing Parameter Store private-key parameter wrapper. Set this or `key_base64`.
- `github.additional_apps[].key_base64_ssm.arn`: ARN of the existing additional-app private-key parameter.
- `github.additional_apps[].key_base64_ssm.name`: Name of the existing additional-app private-key parameter.
- `github.additional_apps[].id`: Additional GitHub App ID supplied directly.
- `github.additional_apps[].id_ssm`: Existing Parameter Store app-ID parameter wrapper. Set this or `id`.
- `github.additional_apps[].id_ssm.arn`: ARN of the existing additional-app ID parameter.
- `github.additional_apps[].id_ssm.name`: Name of the existing additional-app ID parameter.
- `github.additional_apps[].installation_id`: Optional installation ID supplied directly for an additional app.
- `github.additional_apps[].installation_id_ssm`: Optional existing Parameter Store installation-ID parameter wrapper.
- `github.additional_apps[].installation_id_ssm.arn`: ARN of the existing installation-ID parameter.
- `github.additional_apps[].installation_id_ssm.name`: Name of the existing installation-ID parameter.
- `github.enterprise_server.url`: GitHub Enterprise Server URL used by v2 runner-config GitHub clients and the shared termination watcher. The default is null.
- `github.enterprise_server.ssl_verify`: Enables TLS certificate verification for v2 runner-config GitHub clients. The default is `true`.
- `github.user_agent`: HTTP User-Agent used by v2 runner-config GitHub clients. The default is `github-aws-runners`.
- `lambda.artifact.s3.bucket`: Optional shared S3 bucket containing Lambda deployment artifacts for v2 runner configurations and their SSM housekeepers, the webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is null. A component selects an object from this bucket only when its own `artifact.s3` wrapper is present.
- `lambda.runtime`: Runtime for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `nodejs24.x`.
- `lambda.architecture`: Architecture for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `arm64`.
- `lambda.principals`: Additional principals allowed to assume v2 runner-config, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is `[]`; list membership must be known during planning because it creates IAM principal blocks.
- `lambda.principals[].type`: IAM principal type.
- `lambda.principals[].identifiers`: IAM principal identifiers for the type.
- `lambda.subnet_ids`: Subnets for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `[]`.
- `lambda.security_group_ids`: Security groups for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `[]`.
- `lambda.tags`: Default tags for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `{}`.
- `lambda.role.path`: IAM path for module-managed v2 Lambda roles and the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper roles. The default is null, which falls back to `roles.path`.
- `lambda.role.permissions_boundary`: Permissions-boundary ARN for module-managed v2 Lambda roles and the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper roles. The default is null, which falls back to `roles.permissions_boundary`.
- `orchestration_provider.webhook`: Global defaults for the workflow-job webhook control plane. This is a defaults namespace; every runner configuration still selects its demand controller independently under `multi_runner_config[].orchestration_provider`, where exactly one typed provider block must be non-null.
- `orchestration_provider.webhook.queue_selection_strategy`: Queue-selection strategy when multiple runner configurations match a job equally well. The default is `first`, which deterministically selects the first matching queue by priority. `random` spreads jobs across equally matched queues. `all` dispatches to every matching queue, favoring startup speed at the cost of multiple runner launches and registrations for one job.
- `orchestration_provider.webhook.eventbridge.enable`: Routes accepted webhook events through EventBridge when true. The default is `true`, and the value must be known during planning because it selects the webhook implementation.
- `orchestration_provider.webhook.eventbridge.accept_events`: EventBridge event types accepted by the shared webhook. The default is `[]`, which accepts all supported events.
- `orchestration_provider.webhook.matcher_config_parameter_store_tier`: Parameter Store tier for the shared matcher configuration. The default is `Standard`; valid values are `Standard` and `Advanced`. The value must be known during planning because it determines the matcher-parameter chunks.
- `orchestration_provider.webhook.github.repository_white_list`: Repository full names allowed to use the shared webhook. The default is `[]`, which disables repository filtering.
- `orchestration_provider.webhook.runner.boot_time_in_minutes`: Default expected runner boot duration used by webhook scale-down and pool controls. The default is `5`.
- `orchestration_provider.webhook.runner.ephemeral`: Registers webhook-orchestrated runners in ephemeral mode by default. The default is `false`.
- `orchestration_provider.webhook.runner.jit_config_enabled`: Explicit default for just-in-time runner configuration. The default is null, which follows the resolved `ephemeral` mode.
- `orchestration_provider.webhook.runner.maximum_count`: Default maximum number of runners managed by the webhook orchestration provider per runner configuration. The default is null; every webhook runner configuration must resolve this field globally or locally.
- `orchestration_provider.webhook.lambda.artifact`: Shared runner-control-plane artifact used by webhook scale, pool, and job-retry components. Set at most one of `zip` or `s3`; when both are null, the packaged runner archive is used.
- `orchestration_provider.webhook.lambda.artifact.zip`: Optional local path to the shared runner-control-plane Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for runner-control-plane Lambdas, must be known during planning, and requires a non-null shared bucket and key.
- `orchestration_provider.webhook.lambda.artifact.s3.key`: Object key of the shared runner-control-plane Lambda archive.
- `orchestration_provider.webhook.lambda.artifact.s3.object_version`: Optional object version of the shared runner-control-plane Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.scale.up.memory_size`: Scale-up Lambda memory in MB. The default is `512`.
- `orchestration_provider.webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds. The default is `30`.
- `orchestration_provider.webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for scale-up. The default is `1`; use `-1` for unreserved concurrency.
- `orchestration_provider.webhook.lambda.scale.up.job_queued_check_enabled`: Enables queued-job verification before scaling. The default is null, which follows the resolved runner mode.
- `orchestration_provider.webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered per scale-up invocation. The default is `10`.
- `orchestration_provider.webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum build-queue batching window. The default is `0`.
- `orchestration_provider.webhook.lambda.scale.up.tags`: Default tags for scale-up resources. The default is `{}`.
- `orchestration_provider.webhook.lambda.scale.down.memory_size`: Scale-down Lambda memory in MB. The default is `512`.
- `orchestration_provider.webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds. The default is `60`.
- `orchestration_provider.webhook.lambda.scale.down.schedule_expression`: EventBridge schedule for scale-down. The default is `cron(*/5 * * * ? *)`.
- `orchestration_provider.webhook.lambda.scale.down.minimum_running_time_in_minutes`: Minimum runner age before scale-down may terminate it. The default is null, which selects the operating-system default.
- `orchestration_provider.webhook.lambda.scale.down.idle_config`: Default time-based desired idle-runner configurations. The default is `[]`.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate the cron expression.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners retained during the matching period.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed. The default is `oldest_first`.
- `orchestration_provider.webhook.lambda.scale.down.tags`: Default tags for scale-down resources. The default is `{}`.
- `orchestration_provider.webhook.lambda.webhook.artifact`: Shared-webhook artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `orchestration_provider.webhook.lambda.webhook.artifact.zip`: Optional local path to the shared-webhook Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.webhook.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for the webhook and requires a non-null shared bucket and key.
- `orchestration_provider.webhook.lambda.webhook.artifact.s3.key`: Object key of the shared-webhook Lambda archive.
- `orchestration_provider.webhook.lambda.webhook.artifact.s3.object_version`: Optional object version of the shared-webhook Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.webhook.api_gateway_access_log_settings`: Optional API Gateway access-log destination and format for the shared webhook. The default is null, and wrapper presence must be known during planning because it controls the access-log block.
- `orchestration_provider.webhook.lambda.webhook.api_gateway_access_log_settings.destination_arn`: CloudWatch Logs destination ARN for API Gateway access logs.
- `orchestration_provider.webhook.lambda.webhook.api_gateway_access_log_settings.format`: API Gateway access-log format.
- `orchestration_provider.webhook.lambda.webhook.memory_size`: Shared-webhook Lambda memory in MB. The default is `256`.
- `orchestration_provider.webhook.lambda.webhook.timeout`: Shared-webhook Lambda timeout in seconds. The default is `10`.
- `orchestration_provider.webhook.lambda.webhook.tags`: Additional tags for the shared webhook Lambda, merged after `lambda.tags`. The default is `{}`.
- `orchestration_provider.webhook.lambda.pool.memory_size`: Pool Lambda memory in MB. The default is `512`.
- `orchestration_provider.webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds. The default is `60`.
- `orchestration_provider.webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. The default is `1`; use `-1` for unreserved concurrency.
- `orchestration_provider.webhook.lambda.pool.config`: Default scheduled target pool sizes. The default is `[]`, which disables the pool component.
- `orchestration_provider.webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `orchestration_provider.webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `orchestration_provider.webhook.lambda.pool.config[].size`: Desired runner-pool size for the schedule.
- `orchestration_provider.webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity. The default is `false`.
- `orchestration_provider.webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used for pooled runners. The default is null.
- `orchestration_provider.webhook.lambda.pool.tags`: Default tags for pool resources. The default is `{}`.
- `orchestration_provider.webhook.queue.delay_webhook_event`: Default delay in seconds applied to accepted webhook jobs. The default is `30`.
- `orchestration_provider.webhook.queue.job_queue_retention_in_seconds`: Default build-queue message retention period in seconds. The default is `86400`.
- `orchestration_provider.webhook.queue.visibility_timeout_seconds`: Default build-queue visibility timeout. The default is `180`; set it to at least six times every resolved `orchestration_provider.webhook.lambda.scale.up.timeout` that inherits it.
- `orchestration_provider.webhook.queue.redrive_build_queue.enabled`: Creates and attaches a dead-letter queue to every v2 build queue by default. The default is `false`.
- `orchestration_provider.webhook.queue.redrive_build_queue.maxReceiveCount`: Default number of receives before a message moves to the dead-letter queue. The default is null while redrive is disabled and must resolve to a value greater than zero when redrive is enabled.
- `orchestration_provider.webhook.queue.tags`: Default tags for v2 build queues and dead-letter queues. The default is `{}`.
- `orchestration_provider.webhook.queue.encryption`: Global at-rest encryption configuration for the multi-runner build queues and their dead-letter queues. It does not configure runner-config job-retry queues. Omitting the whole block selects SQS-managed encryption and defaults the two KMS attributes to null. When supplying the block explicitly, provide all three leaf attributes and use null for the inactive mode.
- `orchestration_provider.webhook.queue.encryption.kms_data_key_reuse_period_seconds`: KMS data-key reuse period in seconds. This key is required syntactically in an explicit `queue.encryption` object but may be null; it is used only with `kms_master_key_id`.
- `orchestration_provider.webhook.queue.encryption.kms_master_key_id`: KMS key ARN used for queue encryption. This AWS-facing field name is retained for compatibility, but non-null values must be key ARNs because runner-config IAM policies cannot use key IDs or aliases as resources. The field is required syntactically in an explicit `queue.encryption` object but may be null, and a computed ARN may remain unknown until apply. It is independent from `ssm.kms_key_id` and is forwarded separately to each webhook runner configuration so runner-config scale-up and job-retry roles receive only the build-queue KMS permissions they require.
- `orchestration_provider.webhook.queue.encryption.sqs_managed_sse_enabled`: Selects the non-KMS mode: `true` enables SQS-managed encryption and `false` explicitly disables queue encryption. This key is required syntactically in an explicit `queue.encryption` object and must be null when `kms_master_key_id` is set. Omitting the whole encryption block defaults it to `true`.
- `ssm.paths.root`: Base Parameter Store path for shared GitHub App and webhook parameters and for all v2 runner configurations. The schema default is null, which derives `/github-action-runners/`; normalization appends the configuration key only for configuration-owned paths.
- `ssm.paths.app`: Shared GitHub App credential path segment below `ssm.paths.root`. The default is `app`.
- `ssm.paths.webhook`: Shared webhook matcher-configuration path segment below `ssm.paths.root`. The default is `webhook`.
- `ssm.paths.tokens`: Runner registration-token and JIT-configuration path segment below each runner-configuration root. The default is `runners/tokens`.
- `ssm.paths.config`: Persistent runner-configuration path segment below each runner-configuration root. The default is `runners/config`.
- `ssm.kms_key_id`: Optional global KMS key ARN that encrypts shared GitHub App parameters, configures the webhook and termination watcher, and adds matching decrypt permissions to every v2 runner configuration. The default is null and its value may be unknown until apply. It does not select encryption for runtime-created runner-configuration parameters.
- `ssm.tags`: Default tags for the shared GitHub App Parameter Store module and runner-configuration-owned SSM resources. The default is `{}`.
- `ssm.parameters.tags`: Default tags for Terraform-managed and runtime-created runner-configuration parameters. The default is `{}`.
- `ssm.housekeeper.schedule_expression`: Default EventBridge schedule for each runner-configuration SSM housekeeper. The default is `rate(1 day)`.
- `ssm.housekeeper.state`: Default EventBridge rule state for each runner-configuration SSM housekeeper. The default is `ENABLED`.
- `ssm.housekeeper.tags`: Default tags for SSM housekeeper resources. The default is `{}`.
- `ssm.housekeeper.lambda.artifact`: Default SSM-housekeeper artifact selection. Set at most one of `zip` or `s3`; when both are null, runner-config uses its packaged runner control-plane archive.
- `ssm.housekeeper.lambda.artifact.zip`: Optional local path to the SSM-housekeeper Lambda archive. The default is null.
- `ssm.housekeeper.lambda.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for the SSM housekeeper, must be known during planning, and requires a non-null shared bucket and key.
- `ssm.housekeeper.lambda.artifact.s3.key`: Object key of the SSM-housekeeper Lambda archive.
- `ssm.housekeeper.lambda.artifact.s3.object_version`: Optional object version of the SSM-housekeeper Lambda archive. The default is null.
- `ssm.housekeeper.lambda.memory_size`: Default SSM housekeeper Lambda memory in MB. The default is `512`.
- `ssm.housekeeper.lambda.timeout`: Default SSM housekeeper Lambda timeout in seconds. The default is `60`.
- `ssm.housekeeper.config.tokenPath`: Optional cleanup path shared by every runner configuration. The default is null; omit it so each runner configuration derives its isolated token path.
- `ssm.housekeeper.config.minimumDaysOld`: Minimum parameter age in days before deletion is allowed. The default is `1`.
- `ssm.housekeeper.config.dryRun`: Reports eligible parameters without deleting them when true. The default is `false`.
- `observability.logs.level`: Application log level for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `info`.
- `observability.logs.retention_in_days`: CloudWatch Logs retention for v2 runner-config resources and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `180`.
- `observability.logs.kms_key_id`: Optional KMS key ID or ARN for v2 runner-config log groups and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is null.
- `observability.logs.class`: CloudWatch log-group class for v2 runner-config resources and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `STANDARD`.
- `observability.logs.tags`: Default tags for v2 runner-config log groups. The default is `{}`; shared singleton functions receive `tags` and `lambda.tags` instead.
- `observability.tracing.mode`: Optional Lambda tracing mode for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is null; its nullness must be known during planning because it controls X-Ray IAM and tracing blocks.
- `observability.tracing.capture_http_requests`: Enables HTTP request capture in the tracing helper for v2 runner-config functions and translated shared consumers. The default is `false`.
- `observability.tracing.capture_error`: Enables error capture in the tracing helper for v2 runner-config functions and translated shared consumers. The default is `false`.
- `observability.metrics.enable`: Enables module-emitted metrics for v2 runner configurations and the shared termination watcher. The default is `false`.
- `observability.metrics.namespace`: CloudWatch namespace for v2 runner-config and termination-watcher metrics. The default is `GitHub Runners`.
- `observability.metrics.metric.enable_github_app_rate_limit`: Emits GitHub App rate-limit metrics when metrics are enabled. The default is `true`.
- `observability.metrics.metric.enable_job_retry`: Emits job-retry metrics when metrics are enabled. The default is `true`.
- `observability.metrics.metric.enable_spot_termination`: Emits Spot termination metrics from the shared termination watcher when metrics are enabled. The default is `true`.
- `observability.metrics.metric.enable_spot_termination_warning`: Emits Spot termination-warning metrics from the shared termination watcher when metrics are enabled. The default is `true`.
- `compute_provider`: Shared compute-provider defaults grouped first by cloud and then by provider type. Global defaults do not select a provider for any runner configuration.
- `compute_provider.selections`: Optional plan-shaping map keyed by runner-configuration key. Each entry identifies the namespace and type of the configuration's selected compute-provider block. The default is null, which discovers selections from the typed provider blocks. Set this map when unrelated apply-time values make that discovery unknown; its keys and values must be known during planning and cover every runner configuration exactly once.
- `compute_provider.selections[].namespace`: Compute-provider namespace. The only currently supported value is `aws`.
- `compute_provider.selections[].type`: Compute-provider type within the namespace. The only currently supported value is `ec2`.
- `compute_provider.aws`: Shared defaults for AWS compute providers.
- `compute_provider.aws.ec2`: Shared defaults for AWS EC2 runner configurations.
- `compute_provider.aws.ec2.vpc_id`: Shared VPC default for v2 EC2 runner configurations. The default is null; every EC2 runner configuration must resolve this field globally or locally.
- `compute_provider.aws.ec2.subnet_ids`: Shared subnet default for v2 EC2 runner configurations. The default is null; every EC2 runner configuration must resolve this field globally or locally.
- `compute_provider.aws.ec2.managed_security_group_enabled`: Creates the module-managed runner security group by default. The default is `true`.
- `compute_provider.aws.ec2.egress_rules`: Shared runner security-group egress rules. The default is one IPv4/IPv6 allow-all rule; v2 does not inherit flat `runner_egress_rules`.
- `compute_provider.aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].prefix_list_ids`: Prefix-list destinations for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].from_port`: Start of the egress rule port range.
- `compute_provider.aws.ec2.egress_rules[].protocol`: Egress rule protocol; `-1` allows every protocol.
- `compute_provider.aws.ec2.egress_rules[].security_groups`: Destination security-group IDs for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `compute_provider.aws.ec2.egress_rules[].to_port`: End of the egress rule port range.
- `compute_provider.aws.ec2.egress_rules[].description`: Optional egress rule description.
- `compute_provider.aws.ec2.additional_security_group_ids`: Existing security groups attached to every v2 EC2 runner configuration unless overridden. The default is `[]`.
- `compute_provider.aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration inherited by v2 EC2 runner configurations. The default is null; enablement remains configuration-owned.
- `compute_provider.aws.ec2.instance_profile_path`: IAM path for module-managed EC2 instance profiles. The default is null.
- `compute_provider.aws.ec2.key_name`: Optional EC2 key-pair name inherited by v2 EC2 runner configurations. The default is null.
- `compute_provider.aws.ec2.associate_public_ipv4_address`: Associates public IPv4 addresses with v2 EC2 runners unless overridden. The default is `false`.
- `compute_provider.aws.ec2.tags`: Default tags for runtime EC2 resources. The default is `{}` and runner-configuration EC2 tags take precedence.
- `compute_provider.aws.ec2.ami.housekeeper`: Global configuration for the shared AMI-housekeeper Lambda.
- `compute_provider.aws.ec2.ami.housekeeper.enabled`: Creates the shared AMI housekeeper when true. The default is `false`, and the value must be known during planning because it controls the module instance.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config`: AMI cleanup selection and safety settings. The default is `{}`, which resolves the leaf defaults described below in the AMI-housekeeper module.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.maxItems`: Optional maximum number of AMIs queried for cleanup. The default is null, which applies no maximum.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.minimumDaysOld`: Minimum AMI age in days before cleanup. The effective default is `30`.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.amiFilters`: AMI filters, each containing `Name` and `Values`. The effective default selects images with `state = available` and `image-type = machine`.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.launchTemplateNames`: Optional launch-template names whose referenced AMIs are retained. The default is null, which selects no launch templates.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.ssmParameterNames`: Optional Parameter Store names whose referenced AMIs are retained. The default is null, which selects no parameters.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.dryRun`: Reports eligible AMIs without deregistering them when true. The effective default is `false`.
- `compute_provider.aws.ec2.ami.housekeeper.artifact`: AMI-housekeeper artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.zip`: Optional local path to the AMI-housekeeper Lambda archive. The default is null.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 and requires a non-null shared bucket and key.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.s3.key`: Object key of the AMI-housekeeper Lambda archive.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.s3.object_version`: Optional object version of the AMI-housekeeper Lambda archive. The default is null.
- `compute_provider.aws.ec2.ami.housekeeper.lambda.memory_size`: AMI-housekeeper Lambda memory in MB. The default is `256`.
- `compute_provider.aws.ec2.ami.housekeeper.lambda.timeout`: AMI-housekeeper Lambda timeout in seconds. The default is `300`.
- `compute_provider.aws.ec2.ami.housekeeper.schedule.expression`: AMI-housekeeper EventBridge schedule expression. The default is `cron(11 7 * * ? *)`.
- `compute_provider.aws.ec2.instance_termination_watcher`: Global configuration for the shared EC2 instance-termination watcher.
- `compute_provider.aws.ec2.instance_termination_watcher.enabled`: Creates the shared EC2 termination watcher when true. The default is `false`, and the value must be known during planning because it controls the module instance.
- `compute_provider.aws.ec2.instance_termination_watcher.features.enable_spot_termination_handler`: Enables the Spot termination-event handler. The default is `true`, and the value must be known during planning because it controls child resources.
- `compute_provider.aws.ec2.instance_termination_watcher.features.enable_spot_termination_notification_watcher`: Enables the Spot interruption-warning watcher. The default is `true`, and the value must be known during planning because it controls child resources.
- `compute_provider.aws.ec2.instance_termination_watcher.enable_runner_deregistration`: Deregisters terminated runners from GitHub when true. The default is `true`, and the value must be known during planning because it controls deregistration resources.
- `compute_provider.aws.ec2.instance_termination_watcher.environment_variables`: Additional termination-watcher Lambda environment variables. The default is `{}`.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact`: Termination-watcher artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.zip`: Optional local path to the termination-watcher Lambda archive. The default is null.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 and requires a non-null shared bucket and key.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.s3.key`: Object key of the termination-watcher Lambda archive.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.s3.object_version`: Optional object version of the termination-watcher Lambda archive. The default is null.
- `compute_provider.aws.ec2.instance_termination_watcher.lambda.memory_size`: Optional watcher Lambda memory in MB. The default is null, which delegates to the termination-watcher module.
- `compute_provider.aws.ec2.instance_termination_watcher.lambda.timeout`: Optional watcher Lambda timeout in seconds. The default is null, which delegates to the termination-watcher module.
- `compute_provider.aws.ec2.runner_binaries`: Global configuration for the shared runner-distribution buckets and syncers, created once per unique enabled runner operating-system and architecture pair.
- `compute_provider.aws.ec2.runner_binaries.enabled`: Default for whether EC2 runner configurations use the synchronized runner distribution. The default is `true`; a runner configuration may override it through `compute_provider.aws.ec2.binaries_syncer.enabled`. Every resolved enable value must be known during planning because it determines the syncer module instances.
- `compute_provider.aws.ec2.runner_binaries.targets`: Optional plan-shaping map of shared runner-distribution targets, keyed by `_`. The default is null, which discovers enabled targets from runner configurations. Set this map when unrelated apply-time values make that discovery unknown. An empty map creates no shared binary syncers; every enabled runner platform must have a corresponding entry.
- `compute_provider.aws.ec2.runner_binaries.targets[].os`: Runner operating system for the target. Valid values are `linux`, `osx`, and `windows`.
- `compute_provider.aws.ec2.runner_binaries.targets[].architecture`: Runner distribution architecture for the target. Valid values are `x64` and `arm64`.
- `compute_provider.aws.ec2.runner_binaries.s3`: Settings for each shared runner-distribution bucket.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption`: Server-side encryption settings for each distribution bucket.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.enabled`: Creates an explicit distribution-bucket encryption configuration when true. The default is `true`, and the value must be known during planning because it controls resource shape. Keep `kms_master_key_id` null when this is false.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.bucket_key_enabled`: Optional S3 Bucket Key setting. The default is null.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.sse_algorithm`: Server-side encryption algorithm. The default is `AES256`; valid values are `AES256`, `aws:kms`, and `aws:kms:dsse`. When `kms_master_key_id` is set, use one of the KMS algorithms.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.kms_master_key_id`: Optional KMS key identifier for the distribution bucket. The default is null, and its nullness must be known during planning because it controls the syncer KMS policy. The syncer receives KMS access, but runner roles do not derive `kms:Decrypt` from this field; grant runner roles decrypt access separately when using a CMK.
- `compute_provider.aws.ec2.runner_binaries.s3.tags`: Additional tags for each distribution bucket. The default is `{}`; these merge after global `tags`.
- `compute_provider.aws.ec2.runner_binaries.s3.versioning`: Distribution-bucket versioning state. The default is `Disabled`; valid values are `Disabled`, `Enabled`, and `Suspended`. After enabling versioning, Terraform cannot return the bucket to `Disabled`; use `Suspended` instead.
- `compute_provider.aws.ec2.runner_binaries.s3.logging`: Optional access-logging settings for each distribution bucket.
- `compute_provider.aws.ec2.runner_binaries.s3.logging.bucket`: Existing target bucket for access logs. The default is null, and its nullness must be known during planning because it controls the logging resource.
- `compute_provider.aws.ec2.runner_binaries.s3.logging.prefix`: Optional access-log prefix. The default is null, which uses the distribution-bucket name when logging is enabled. A non-null prefix requires `logging.bucket`.
- `compute_provider.aws.ec2.runner_binaries.syncer`: Component-specific settings for the shared runner-binary syncer Lambda.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact`: Syncer Lambda artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.zip`: Optional local path to the syncer Lambda archive. The default is null.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 and requires a non-null shared bucket and key.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.s3.key`: Object key of the syncer Lambda archive. Required when the `s3` wrapper is present.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.s3.object_version`: Optional object version of the syncer Lambda archive. The default is null.
- `compute_provider.aws.ec2.runner_binaries.syncer.lambda`: Syncer Lambda sizing settings. Runtime, architecture, networking, role, tags, logging, and tracing come from their global experimental blocks.
- `compute_provider.aws.ec2.runner_binaries.syncer.lambda.memory_size`: Memory allocated to the syncer Lambda in MB. The default is `256`.
- `compute_provider.aws.ec2.runner_binaries.syncer.lambda.timeout`: Syncer Lambda timeout in seconds. The default is `300`.
- `compute_provider.aws.ec2.runner_binaries.syncer.schedule`: EventBridge schedule settings for the syncer Lambda.
- `compute_provider.aws.ec2.runner_binaries.syncer.schedule.expression`: EventBridge schedule expression. The default is `cron(27 * * * ? *)`.
- `compute_provider.aws.ec2.runner_binaries.syncer.schedule.state`: EventBridge rule state. The default is `ENABLED`; valid values are `DISABLED`, `ENABLED`, and `ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS`.

Each `experimental.multi_runner_config` entry supports the following nested fields:

- `multi_runner_config[].tags`: Configuration-wide tags. These override global `experimental.tags`; narrower component and compute-provider tag maps take precedence for their resources. Flat `tags` are not merged into v2 queues or runner configurations.
- `multi_runner_config[].runner.os`: Runner operating system.
- `multi_runner_config[].runner.architecture`: Runner distribution architecture.
- `multi_runner_config[].runner.disable_default_labels`: Prevents GitHub default labels from being registered.
- `multi_runner_config[].runner.extra_labels`: Additional labels combined with `orchestration_provider.webhook.matcherConfig.labelMatchers` for webhook runner configurations. Default self-hosted, operating-system, and architecture labels are also included unless `runner.disable_default_labels` is true.
- `multi_runner_config[].runner.group_name`: GitHub runner group used during registration.
- `multi_runner_config[].runner.name_prefix`: Prefix added to registered runner names.
- `multi_runner_config[].runner.run_as_root`: Runs the runner service as root when supported by the compute provider.
- `multi_runner_config[].runner.run_as`: Operating-system user used when `run_as_root` is false.
- `multi_runner_config[].runner.auto_update_disabled`: Disables the GitHub runner application's built-in updater.
- `multi_runner_config[].runner.tags`: Tags for common runner resources, currently the managed runner IAM role. These override entry-level `tags`.
- `multi_runner_config[].runner.hooks.job_started`: Script content installed as the runner job-started hook.
- `multi_runner_config[].runner.hooks.job_completed`: Script content installed as the runner job-completed hook.
- `multi_runner_config[].runner.iam.role`: Optional externally managed runner-role wrapper. When set, inherited managed policies and trust additions are suppressed.
- `multi_runner_config[].runner.iam.role.arn`: ARN of an externally managed runner role. When set, `runner-config` does not create or modify that role.
- `multi_runner_config[].runner.iam.managed_policy_arns`: Named managed-policy ARNs attached to the module-managed runner role. Keep this empty or null when the runner configuration selects an external `runner.iam.role`.
- `multi_runner_config[].runner.iam.additional_trust_policy_json`: Optional IAM policy document merged with the selected compute provider's default runner-role trust policy. Keep it null when the runner configuration selects an external `runner.iam.role`.
- `multi_runner_config[].runner.iam.path`: IAM path for the module-managed runner role.
- `multi_runner_config[].runner.iam.permissions_boundary`: Permissions-boundary ARN for the module-managed runner role.
- `multi_runner_config[].lambda`: Lambda runtime, architecture, networking, tags, and role substrate shared by orchestration and the runner-configuration SSM housekeeper. Scale function settings belong under `orchestration_provider.webhook.lambda`.
- `multi_runner_config[].lambda.runtime`: Per-configuration runtime override for runner-config Lambda functions.
- `multi_runner_config[].lambda.architecture`: Per-configuration architecture override for runner-config Lambda functions.
- `multi_runner_config[].lambda.subnet_ids`: Per-configuration subnet override for runner-config Lambda functions.
- `multi_runner_config[].lambda.security_group_ids`: Per-configuration security-group override for runner-config Lambda functions.
- `multi_runner_config[].lambda.tags`: Per-configuration tags for control-plane Lambda functions. Component tags override this map.
- `multi_runner_config[].lambda.role.path`: Per-configuration IAM path for module-managed Lambda roles. Null inherits `experimental.lambda.role.path`, then `experimental.roles.path`.
- `multi_runner_config[].lambda.role.permissions_boundary`: Per-configuration permissions-boundary ARN for module-managed Lambda roles. Null inherits `experimental.lambda.role.permissions_boundary`, then `experimental.roles.permissions_boundary`.
- `multi_runner_config[].orchestration_provider`: Demand-controller selection. Exactly one typed provider block must be non-null. Only `webhook` is supported today; additional providers can be added without moving the webhook contract.
- `multi_runner_config[].orchestration_provider.webhook`: Selects the workflow-job webhook control plane, including its SQS build queue, scale-up, scheduled scale-down/pool, and optional job-retry resources.
- `multi_runner_config[].orchestration_provider.webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by webhook scale-down and pool controls. Null inherits `experimental.orchestration_provider.webhook.runner.boot_time_in_minutes`.
- `multi_runner_config[].orchestration_provider.webhook.runner.ephemeral`: Registers webhook-orchestrated runners in ephemeral mode. Null inherits `experimental.orchestration_provider.webhook.runner.ephemeral`.
- `multi_runner_config[].orchestration_provider.webhook.runner.jit_config_enabled`: Explicitly enables or disables just-in-time configuration. Null inherits the global webhook value; if both are null, behavior follows the resolved webhook `ephemeral` mode.
- `multi_runner_config[].orchestration_provider.webhook.runner.maximum_count`: Maximum number of runners managed by the webhook orchestration provider for this configuration. Null inherits `experimental.orchestration_provider.webhook.runner.maximum_count`.
- `multi_runner_config[].orchestration_provider.webhook.github.organization_runners`: Registers runners at organization scope when true; otherwise repository-scoped registration is used.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.labelMatchers`: Groups of labels used to match webhook jobs to this configuration.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.exactMatch`: Deprecated one-way match. When true, every workflow-job label must appear in a configured label group, but that group may contain additional labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.bidirectionalLabelMatch`: Requires an exact two-way set match between workflow-job labels and a configured label group, with no extra or missing labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.priority`: Ordering used when multiple configurations match the same job.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.enableDynamicLabels`: Enables runtime interpretation of supported dynamic AWS labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy`: Optional policy restricting values accepted from dynamic AWS labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.blocked_keys`: Dynamic-label keys rejected for this runner configuration. The default is `[]`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys`: Per-key allow, deny, and maximum-value restrictions. The default is `{}`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys..allowed`: Values explicitly allowed for the dynamic-label key. The default is `[]`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys..denied`: Values explicitly denied for the dynamic-label key. The default is `[]`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys..max`: Optional maximum accepted value for the dynamic-label key. The default is null.
- `multi_runner_config[].orchestration_provider.webhook.queue.delay_webhook_event`: Delay in seconds applied to webhook job messages. Null inherits the global queue default.
- `multi_runner_config[].orchestration_provider.webhook.queue.job_queue_retention_in_seconds`: Build-queue message retention period in seconds. Null inherits the global queue default.
- `multi_runner_config[].orchestration_provider.webhook.queue.visibility_timeout_seconds`: Build-queue visibility timeout. Null inherits the global queue default; the resolved value must be at least six times the resolved `orchestration_provider.webhook.lambda.scale.up.timeout` so Lambda has enough time to retry throttled invocations.
- `multi_runner_config[].orchestration_provider.webhook.queue.redrive_build_queue.enabled`: Creates and attaches a dead-letter queue for the build queue. A null wrapper or null leaf inherits the corresponding global value.
- `multi_runner_config[].orchestration_provider.webhook.queue.redrive_build_queue.maxReceiveCount`: Number of receives before a build message moves to the dead-letter queue. A null wrapper or null leaf inherits the corresponding global value, and the resolved value must be greater than zero when redrive is enabled.
- `multi_runner_config[].orchestration_provider.webhook.queue.tags`: Tags for configuration-owned queue resources. These merge after global queue tags and entry-level `tags`; component tags override this map.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.memory_size`: Memory allocated to the scale-up Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for the scale-up Lambda. Use `-1` for unreserved concurrency.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.job_queued_check_enabled`: Enables the queued-job verification before scaling. Null inherits the global value; if both are null, behavior follows the resolved runner mode.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered to one scale-up Lambda invocation.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum batching window for build-queue records.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.tags`: Tags for scale-up resources. These override entry-level and shared Lambda, queue, and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.memory_size`: Memory allocated to the scale-down Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.schedule_expression`: EventBridge schedule expression that invokes scale-down.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.minimum_running_time_in_minutes`: Minimum runner age before scale-down may terminate it. Null inherits the global value; if both are null, the operating-system default is selected.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config`: Time-based desired idle-runner configurations.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate `cron`.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners to retain during the matching period.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.tags`: Tags for scale-down resources. These override entry-level and shared Lambda and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.memory_size`: Memory allocated to the pool Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. Use `-1` for unreserved concurrency.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config`: Scheduled target pool sizes. An empty list disables the pool component.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config[].size`: Desired number of runners for the schedule.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used when creating pooled runners.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.tags`: Tags for pool resources. These override entry-level and shared Lambda and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.enabled`: Creates the retry queue, Lambda function, event-source mapping, and related IAM resources.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.delay_in_seconds`: Initial delay before a queued-job retry check.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.delay_backoff`: Multiplier applied to the delay after each unsuccessful check.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.max_attempts`: Maximum retry-check attempts before the message is no longer republished.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.tags`: Tags for job-retry resources. These override entry-level and shared Lambda, queue, and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.lambda.memory_size`: Memory allocated to the job-retry Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.lambda.reserved_concurrent_executions`: Reserved concurrency for the job-retry Lambda. Use `-1` for unreserved concurrency.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.lambda.timeout`: Job-retry Lambda timeout in seconds and visibility timeout for its retry queue.
- `multi_runner_config[].ssm.paths.root`: Base Parameter Store root for this runner configuration. The configuration key is always appended to preserve configuration isolation. The omitted global root derives `/github-action-runners/`.
- `multi_runner_config[].ssm.paths.tokens`: Path segment below the runner-configuration root used for runner registration tokens and just-in-time configuration.
- `multi_runner_config[].ssm.paths.config`: Path segment below the runner-configuration root used for persistent runner configuration.
- `multi_runner_config[].ssm.tags`: Shared tags for runner-configuration-owned SSM resources. These override entry-level `tags`.
- `multi_runner_config[].ssm.parameters.tags`: Tags for Terraform-managed and runtime-created runner configuration parameters. These override `ssm.tags`.
- `multi_runner_config[].ssm.housekeeper.schedule_expression`: EventBridge schedule expression that invokes the runner-configuration SSM housekeeper.
- `multi_runner_config[].ssm.housekeeper.state`: EventBridge rule state for the runner-configuration SSM housekeeper. Valid values are `DISABLED`, `ENABLED`, and `ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS`.
- `multi_runner_config[].ssm.housekeeper.tags`: Tags for SSM housekeeper resources. These override entry-level, shared Lambda, shared log, and `ssm.tags` values.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact`: Per-configuration SSM-housekeeper artifact selection. A selected `zip` or `s3` source overrides the global `ssm.housekeeper.lambda.artifact`; when neither level selects a source, runner-config uses its packaged runner control-plane archive.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.zip`: Optional local path to the SSM-housekeeper Lambda archive.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for this SSM housekeeper, must be known during planning, and requires a non-null shared bucket and key.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.s3.key`: Object key of this runner configuration's SSM-housekeeper Lambda archive.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.s3.object_version`: Optional object version of this runner configuration's SSM-housekeeper Lambda archive.
- `multi_runner_config[].ssm.housekeeper.lambda.memory_size`: Memory allocated to the SSM housekeeper Lambda in MB.
- `multi_runner_config[].ssm.housekeeper.lambda.timeout`: SSM housekeeper Lambda timeout in seconds.
- `multi_runner_config[].ssm.housekeeper.config.tokenPath`: Optional cleanup path. A global value is shared by every runner configuration, so omit it to derive each configuration's isolated token path.
- `multi_runner_config[].ssm.housekeeper.config.minimumDaysOld`: Minimum parameter age in days before deletion is allowed.
- `multi_runner_config[].ssm.housekeeper.config.dryRun`: Reports eligible parameters without deleting them when true.
- `multi_runner_config[].observability.logs.level`: Application log level for runner-configuration control-plane functions.
- `multi_runner_config[].observability.logs.retention_in_days`: CloudWatch Logs retention period for runner-configuration resources.
- `multi_runner_config[].observability.logs.kms_key_id`: Optional KMS key ID or ARN used to encrypt runner-configuration CloudWatch log groups.
- `multi_runner_config[].observability.logs.class`: CloudWatch log-group class for runner-configuration resources.
- `multi_runner_config[].observability.logs.tags`: Shared tags for runner-configuration CloudWatch log groups. Component tags override this map.
- `multi_runner_config[].observability.tracing.mode`: Optional Lambda tracing mode. Its nullness must be known during planning because it controls X-Ray IAM and tracing blocks.
- `multi_runner_config[].observability.tracing.capture_http_requests`: Enables HTTP request capture in the tracing helper.
- `multi_runner_config[].observability.tracing.capture_error`: Enables error capture in the tracing helper.
- `multi_runner_config[].observability.metrics.enable`: Enables module-emitted metrics.
- `multi_runner_config[].observability.metrics.namespace`: CloudWatch namespace used for emitted metrics.
- `multi_runner_config[].observability.metrics.metric.enable_github_app_rate_limit`: Emits GitHub App rate-limit metrics.
- `multi_runner_config[].observability.metrics.metric.enable_job_retry`: Emits job-retry metrics.
- `multi_runner_config[].compute_provider`: Typed compute-provider namespaces. Exactly one nested provider block must be non-null, and that populated block selects the provider. Its presence must be known during planning; values inside it may remain unknown until apply.
- `multi_runner_config[].compute_provider.aws`: AWS compute-provider namespace. The namespace itself does not select a provider.
- `multi_runner_config[].compute_provider.aws.ec2`: AWS EC2-specific configuration. A non-null block selects AWS EC2 for this runner configuration.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when enabled.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `multi_runner_config[].compute_provider.aws.ec2.ami.filter`: EC2 AMI filters combined with the default AMI-name filter.
- `multi_runner_config[].compute_provider.aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `multi_runner_config[].compute_provider.aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. The wrapper's presence selects external ownership at plan time.
- `multi_runner_config[].compute_provider.aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply.
- `multi_runner_config[].compute_provider.aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence selects the KMS policy at plan time.
- `multi_runner_config[].compute_provider.aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings`: EBS mappings added to the runner launch template.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].delete_on_termination`: Deletes the volume when its runner instance terminates.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].encrypted`: Enables EBS encryption.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].iops`: Provisioned IOPS for supported volume types.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].throughput`: Provisioned throughput for supported volume types.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].volume_initialization_rate`: Fixed initialization rate in MiB/s for supported snapshot-backed volumes.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].volume_size`: Volume size in GiB.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].volume_type`: EBS volume type.
- `multi_runner_config[].compute_provider.aws.ec2.create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `multi_runner_config[].compute_provider.aws.ec2.credit_specification`: CPU credit mode for burstable instance types.
- `multi_runner_config[].compute_provider.aws.ec2.ebs_optimized`: Requests EBS-optimized runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.cloudwatch_agent.enabled`: Installs and configures the CloudWatch agent through the default bootstrap flow.
- `multi_runner_config[].compute_provider.aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration.
- `multi_runner_config[].compute_provider.aws.ec2.binaries_syncer.enabled`: Enables use of the shared synchronized runner distribution from S3. Null inherits `experimental.compute_provider.aws.ec2.runner_binaries.enabled`.
- `multi_runner_config[].compute_provider.aws.ec2.detailed_monitoring_enabled`: Enables detailed EC2 monitoring for runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.ssm_enabled`: Attaches runner permissions and policies required for AWS Systems Manager access.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.enabled`: Enables launch-template user data.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.template`: Optional path to a custom user-data template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.content`: Optional complete user-data content used instead of rendering a template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.pre_install`: Script content inserted before runner installation in the default template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.post_install`: Script content inserted after runner installation in the default template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets in logs.
- `multi_runner_config[].compute_provider.aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select capacity.
- `multi_runner_config[].compute_provider.aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price.
- `multi_runner_config[].compute_provider.aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `multi_runner_config[].compute_provider.aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `multi_runner_config[].compute_provider.aws.ec2.instance_types`: EC2 instance types available to the scale-up and pool functions.
- `multi_runner_config[].compute_provider.aws.ec2.additional_security_group_ids`: Existing security groups attached to runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.managed_security_group_enabled`: Creates the module-managed runner security group when true.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules`: Runner security-group egress rules. Null inherits `experimental.compute_provider.aws.ec2.egress_rules`, whose default is the built-in IPv4/IPv6 allow-all rule. Flat `runner_egress_rules` is not inherited by v2.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].prefix_list_ids`: Prefix-list destinations for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].from_port`: Start of the runner-configuration egress rule port range.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].protocol`: Runner-configuration egress rule protocol; `-1` allows every protocol.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].security_groups`: Destination security-group IDs for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].to_port`: End of the runner-configuration egress rule port range.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].description`: Optional runner-configuration egress rule description.
- `multi_runner_config[].compute_provider.aws.ec2.instance_profile_path`: IAM path for the module-managed EC2 instance profile.
- `multi_runner_config[].compute_provider.aws.ec2.key_name`: Optional EC2 key-pair name for runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.associate_public_ipv4_address`: Associates a public IPv4 address with runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.instance_profile.name`: Name of an externally managed instance profile. Setting it also requires `runner.iam.role`.
- `multi_runner_config[].compute_provider.aws.ec2.enable_on_demand_failover_for_errors`: EC2 error codes that trigger an on-demand fallback after a Spot launch failure.
- `multi_runner_config[].compute_provider.aws.ec2.scale_errors`: EC2 error codes treated as retryable scale-up failures.
- `multi_runner_config[].compute_provider.aws.ec2.subnet_ids`: Subnets from which scale-up may launch runners. A null configuration value inherits the experimental global value.
- `multi_runner_config[].compute_provider.aws.ec2.vpc_id`: VPC in which runner networking resources are created. A null configuration value inherits the experimental global value.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `multi_runner_config[].compute_provider.aws.ec2.placement.affinity`: Host affinity setting.
- `multi_runner_config[].compute_provider.aws.ec2.placement.availability_zone`: Availability Zone in which the instance is placed.
- `multi_runner_config[].compute_provider.aws.ec2.placement.group_id`: Placement-group ID.
- `multi_runner_config[].compute_provider.aws.ec2.placement.group_name`: Placement-group name.
- `multi_runner_config[].compute_provider.aws.ec2.placement.host_id`: Dedicated Host ID.
- `multi_runner_config[].compute_provider.aws.ec2.placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `multi_runner_config[].compute_provider.aws.ec2.placement.spread_domain`: Spread-domain placement value.
- `multi_runner_config[].compute_provider.aws.ec2.placement.tenancy`: Instance tenancy.
- `multi_runner_config[].compute_provider.aws.ec2.placement.partition_number`: Placement-group partition number.
- `multi_runner_config[].compute_provider.aws.ec2.license_specifications[].license_configuration_arn`: ARN of a License Manager license configuration.
- `multi_runner_config[].compute_provider.aws.ec2.use_dedicated_host`: Enables the dedicated-host launch path required for macOS runners.
- `multi_runner_config[].compute_provider.aws.ec2.log_files`: Optional log files collected by the CloudWatch agent.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_group_name`: CloudWatch log-group name before optional prefixing.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path when true.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].file_path`: File or glob read by the CloudWatch agent.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_stream_name`: CloudWatch log-stream name template.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_class`: CloudWatch log-group class for the collected file.
- `multi_runner_config[].compute_provider.aws.ec2.tags`: Tags for runtime EC2 instances, volumes, network interfaces, and eligible Spot requests. These override entry-level tags and the generated runner `Name`; provider-required bootstrap tags take final precedence. | 
object({
    tags = optional(map(string), {})

    roles = optional(object({
      path                 = optional(string, null)
      permissions_boundary = optional(string, null)
    }), {})

    runner = optional(object({
      os                     = optional(string, null)
      architecture           = optional(string, null)
      disable_default_labels = optional(bool, false)
      extra_labels           = optional(list(string), [])
      group_name             = optional(string, "Default")
      name_prefix            = optional(string, "")
      run_as_root            = optional(bool, false)
      run_as                 = optional(string, "ec2-user")
      auto_update_disabled   = optional(bool, false)
      tags                   = optional(map(string), {})
      hooks = optional(object({
        job_started   = optional(string, "")
        job_completed = optional(string, "")
      }), {})
      iam = optional(object({
        role = optional(object({
          arn = string
        }), null)
        managed_policy_arns          = optional(map(string), {})
        additional_trust_policy_json = optional(string, null)
        path                         = optional(string, null)
        permissions_boundary         = optional(string, null)
      }), {})
    }), {})

    github = optional(object({
      app = optional(object({
        key_base64 = optional(string)
        key_base64_ssm = optional(object({
          arn  = string
          name = string
        }))
        id = optional(string)
        id_ssm = optional(object({
          arn  = string
          name = string
        }))
        webhook_secret = optional(string)
        webhook_secret_ssm = optional(object({
          arn  = string
          name = string
        }))
      }), null)
      additional_apps = optional(list(object({
        key_base64          = optional(string)
        key_base64_ssm      = optional(object({ arn = string, name = string }))
        id                  = optional(string)
        id_ssm              = optional(object({ arn = string, name = string }))
        installation_id     = optional(string)
        installation_id_ssm = optional(object({ arn = string, name = string }))
      })), [])
      enterprise_server = optional(object({
        url        = optional(string, null)
        ssl_verify = optional(bool, true)
      }), {})
      user_agent = optional(string, "github-aws-runners")
    }), {})

    lambda = optional(object({
      artifact = optional(object({
        s3 = optional(object({
          bucket = optional(string, null)
        }), {})
      }), {})
      runtime      = optional(string, "nodejs24.x")
      architecture = optional(string, "arm64")
      principals = optional(list(object({
        type        = string
        identifiers = list(string)
      })), [])
      subnet_ids         = optional(list(string), [])
      security_group_ids = optional(list(string), [])
      tags               = optional(map(string), {})
      role = optional(object({
        path                 = optional(string, null)
        permissions_boundary = optional(string, null)
      }), {})
    }), {})

    orchestration_provider = optional(object({
      webhook = optional(object({
        queue_selection_strategy = optional(string, "first")
        eventbridge = optional(object({
          enable        = optional(bool, true)
          accept_events = optional(list(string), [])
        }), {})
        matcher_config_parameter_store_tier = optional(string, "Standard")
        runner = optional(object({
          boot_time_in_minutes = optional(number, 5)
          ephemeral            = optional(bool, false)
          jit_config_enabled   = optional(bool, null)
          maximum_count        = optional(number, null)
        }), {})

        github = optional(object({
          repository_white_list = optional(list(string), [])
        }), {})

        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          scale = optional(object({
            up = optional(object({
              memory_size                    = optional(number, 512)
              timeout                        = optional(number, 30)
              reserved_concurrent_executions = optional(number, 1)
              job_queued_check_enabled       = optional(bool, null)
              event_source_mapping = optional(object({
                batch_size                         = optional(number, 10)
                maximum_batching_window_in_seconds = optional(number, 0)
              }), {})
              tags = optional(map(string), {})
            }), {})
            down = optional(object({
              memory_size                     = optional(number, 512)
              timeout                         = optional(number, 60)
              schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
              minimum_running_time_in_minutes = optional(number, null)
              idle_config = optional(list(object({
                cron             = string
                timeZone         = string
                idleCount        = number
                evictionStrategy = optional(string, "oldest_first")
              })), [])
              tags = optional(map(string), {})
            }), {})
          }), {})
          webhook = optional(object({
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            api_gateway_access_log_settings = optional(object({
              destination_arn = string
              format          = string
            }), null)
            memory_size = optional(number, 256)
            timeout     = optional(number, 10)
            tags        = optional(map(string), {})
          }), {})
          pool = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 60)
            reserved_concurrent_executions = optional(number, 1)
            config = optional(list(object({
              schedule_expression          = string
              schedule_expression_timezone = optional(string)
              size                         = number
            })), [])
            include_busy_runners = optional(bool, false)
            runner_owner         = optional(string, null)
            tags                 = optional(map(string), {})
          }), {})
        }), {})

        queue = optional(object({
          delay_webhook_event            = optional(number, 30)
          job_queue_retention_in_seconds = optional(number, 86400)
          visibility_timeout_seconds     = optional(number, 180)
          redrive_build_queue = optional(object({
            enabled         = optional(bool, false)
            maxReceiveCount = optional(number, null)
            }), {
            enabled         = false
            maxReceiveCount = null
          })
          tags = optional(map(string), {})
          encryption = optional(object({
            kms_data_key_reuse_period_seconds = number
            kms_master_key_id                 = string
            sqs_managed_sse_enabled           = bool
            }), {
            kms_data_key_reuse_period_seconds = null
            kms_master_key_id                 = null
            sqs_managed_sse_enabled           = true
          })
        }), {})
      }), {})
    }), {})

    ssm = optional(object({
      paths = optional(object({
        root    = optional(string, null)
        app     = optional(string, "app")
        webhook = optional(string, "webhook")
        tokens  = optional(string, "runners/tokens")
        config  = optional(string, "runners/config")
      }), {})
      kms_key_id = optional(string, null)
      tags       = optional(map(string), {})
      parameters = optional(object({
        tags = optional(map(string), {})
      }), {})
      housekeeper = optional(object({
        schedule_expression = optional(string, "rate(1 day)")
        state               = optional(string, "ENABLED")
        tags                = optional(map(string), {})
        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          memory_size = optional(number, 512)
          timeout     = optional(number, 60)
        }), {})
        config = optional(object({
          tokenPath      = optional(string, null)
          minimumDaysOld = optional(number, 1)
          dryRun         = optional(bool, false)
        }), {})
      }), {})
    }), {})

    observability = optional(object({
      logs = optional(object({
        level             = optional(string, "info")
        retention_in_days = optional(number, 180)
        kms_key_id        = optional(string, null)
        class             = optional(string, "STANDARD")
        tags              = optional(map(string), {})
      }), {})
      tracing = optional(object({
        mode                  = optional(string, null)
        capture_http_requests = optional(bool, false)
        capture_error         = optional(bool, false)
      }), {})
      metrics = optional(object({
        enable    = optional(bool, false)
        namespace = optional(string, "GitHub Runners")
        metric = optional(object({
          enable_github_app_rate_limit    = optional(bool, true)
          enable_job_retry                = optional(bool, true)
          enable_spot_termination         = optional(bool, true)
          enable_spot_termination_warning = optional(bool, true)
        }), {})
      }), {})
    }), {})

    compute_provider = optional(object({
      selections = optional(map(object({
        namespace = string
        type      = string
      })), null)
      aws = optional(object({
        ec2 = optional(object({
          vpc_id                         = optional(string, null)
          subnet_ids                     = optional(list(string), null)
          managed_security_group_enabled = optional(bool, true)
          egress_rules = optional(list(object({
            cidr_blocks      = list(string)
            ipv6_cidr_blocks = list(string)
            prefix_list_ids  = list(string)
            from_port        = number
            protocol         = string
            security_groups  = list(string)
            self             = bool
            to_port          = number
            description      = string
            })), [{
            cidr_blocks      = ["0.0.0.0/0"]
            ipv6_cidr_blocks = ["::/0"]
            prefix_list_ids  = null
            from_port        = 0
            protocol         = "-1"
            security_groups  = null
            self             = null
            to_port          = 0
            description      = null
          }])
          additional_security_group_ids = optional(list(string), [])
          cloudwatch_agent = optional(object({
            config = optional(string, null)
          }), {})
          instance_profile_path         = optional(string, null)
          key_name                      = optional(string, null)
          associate_public_ipv4_address = optional(bool, false)
          tags                          = optional(map(string), {})
          ami = optional(object({
            housekeeper = optional(object({
              enabled = optional(bool, false)
              cleanup_config = optional(object({
                maxItems       = optional(number)
                minimumDaysOld = optional(number)
                amiFilters = optional(list(object({
                  Name   = string
                  Values = list(string)
                })))
                launchTemplateNames = optional(list(string))
                ssmParameterNames   = optional(list(string))
                dryRun              = optional(bool)
              }), {})
              artifact = optional(object({
                zip = optional(string, null)
                s3 = optional(object({
                  key            = string
                  object_version = optional(string, null)
                }), null)
              }), {})
              lambda = optional(object({
                memory_size = optional(number, 256)
                timeout     = optional(number, 300)
              }), {})
              schedule = optional(object({
                expression = optional(string, "cron(11 7 * * ? *)")
              }), {})
            }), {})
          }), {})
          instance_termination_watcher = optional(object({
            enabled = optional(bool, false)
            features = optional(object({
              enable_spot_termination_handler              = optional(bool, true)
              enable_spot_termination_notification_watcher = optional(bool, true)
            }), {})
            enable_runner_deregistration = optional(bool, true)
            environment_variables        = optional(map(string), {})
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            lambda = optional(object({
              memory_size = optional(number, null)
              timeout     = optional(number, null)
            }), {})
          }), {})
          runner_binaries = optional(object({
            enabled = optional(bool, true)
            targets = optional(map(object({
              os           = string
              architecture = string
            })), null)
            s3 = optional(object({
              encryption = optional(object({
                enabled            = optional(bool, true)
                bucket_key_enabled = optional(bool, null)
                sse_algorithm      = optional(string, "AES256")
                kms_master_key_id  = optional(string, null)
              }), {})
              tags       = optional(map(string), {})
              versioning = optional(string, "Disabled")
              logging = optional(object({
                bucket = optional(string, null)
                prefix = optional(string, null)
              }), {})
            }), {})
            syncer = optional(object({
              artifact = optional(object({
                zip = optional(string, null)
                s3 = optional(object({
                  key            = string
                  object_version = optional(string, null)
                }), null)
              }), {})
              lambda = optional(object({
                memory_size = optional(number, 256)
                timeout     = optional(number, 300)
              }), {})
              schedule = optional(object({
                expression = optional(string, "cron(27 * * * ? *)")
                state      = optional(string, "ENABLED")
              }), {})
            }), {})
          }), {})
        }), {})
      }), {})
    }), {})

    multi_runner_config = optional(map(object({
      tags = optional(map(string), {})

      runner = optional(object({
        os                     = optional(string, null)
        architecture           = optional(string, null)
        disable_default_labels = optional(bool, null)
        extra_labels           = optional(list(string), null)
        group_name             = optional(string, null)
        name_prefix            = optional(string, null)
        run_as_root            = optional(bool, null)
        run_as                 = optional(string, null)
        auto_update_disabled   = optional(bool, null)
        tags                   = optional(map(string), {})
        hooks = optional(object({
          job_started   = optional(string, null)
          job_completed = optional(string, null)
        }), {})
        iam = optional(object({
          role = optional(object({
            arn = string
          }), null)
          managed_policy_arns          = optional(map(string), null)
          additional_trust_policy_json = optional(string, null)
          path                         = optional(string, null)
          permissions_boundary         = optional(string, null)
        }), {})
      }), {})

      lambda = optional(object({
        runtime            = optional(string, null)
        architecture       = optional(string, null)
        subnet_ids         = optional(list(string), null)
        security_group_ids = optional(list(string), null)
        tags               = optional(map(string), {})
        role = optional(object({
          path                 = optional(string, null)
          permissions_boundary = optional(string, null)
        }), {})
      }), {})

      orchestration_provider = object({
        webhook = optional(object({
          runner = optional(object({
            boot_time_in_minutes = optional(number, null)
            ephemeral            = optional(bool, null)
            jit_config_enabled   = optional(bool, null)
            maximum_count        = optional(number, null)
          }), {})

          github = optional(object({
            organization_runners = optional(bool, false)
          }), {})

          matcherConfig = object({
            labelMatchers           = list(list(string))
            exactMatch              = optional(bool, false)
            bidirectionalLabelMatch = optional(bool, false)
            priority                = optional(number, 999)
            enableDynamicLabels     = optional(bool, false)
            awsDynamicLabelsPolicy = optional(object({
              blocked_keys = optional(list(string), [])
              restricted_keys = optional(map(object({
                allowed = optional(list(string), [])
                denied  = optional(list(string), [])
                max     = optional(string, null)
              })), {})
            }), null)
          })

          queue = optional(object({
            delay_webhook_event            = optional(number, null)
            job_queue_retention_in_seconds = optional(number, null)
            visibility_timeout_seconds     = optional(number, null)
            redrive_build_queue = optional(object({
              enabled         = optional(bool, null)
              maxReceiveCount = optional(number, null)
            }), null)
            tags = optional(map(string), {})
          }), {})

          lambda = optional(object({
            scale = optional(object({
              up = optional(object({
                memory_size                    = optional(number, null)
                timeout                        = optional(number, null)
                reserved_concurrent_executions = optional(number, null)
                job_queued_check_enabled       = optional(bool, null)
                event_source_mapping = optional(object({
                  batch_size                         = optional(number, null)
                  maximum_batching_window_in_seconds = optional(number, null)
                }), {})
                tags = optional(map(string), {})
              }), {})
              down = optional(object({
                memory_size                     = optional(number, null)
                timeout                         = optional(number, null)
                schedule_expression             = optional(string, null)
                minimum_running_time_in_minutes = optional(number, null)
                idle_config = optional(list(object({
                  cron             = string
                  timeZone         = string
                  idleCount        = number
                  evictionStrategy = optional(string, "oldest_first")
                })), null)
                tags = optional(map(string), {})
              }), {})
            }), {})
            pool = optional(object({
              memory_size                    = optional(number, null)
              timeout                        = optional(number, null)
              reserved_concurrent_executions = optional(number, null)
              config = optional(list(object({
                schedule_expression          = string
                schedule_expression_timezone = optional(string)
                size                         = number
              })), null)
              include_busy_runners = optional(bool, null)
              runner_owner         = optional(string, null)
              tags                 = optional(map(string), {})
            }), {})
          }), {})

          job_retry = optional(object({
            enabled          = optional(bool, false)
            delay_in_seconds = optional(number, 300)
            delay_backoff    = optional(number, 2)
            max_attempts     = optional(number, 1)
            tags             = optional(map(string), {})
            lambda = optional(object({
              memory_size                    = optional(number, 256)
              reserved_concurrent_executions = optional(number, 1)
              timeout                        = optional(number, 30)
            }), {})
          }), {})

        }), null)

      })

      ssm = optional(object({
        paths = optional(object({
          root   = optional(string, null)
          tokens = optional(string, null)
          config = optional(string, null)
        }), {})
        tags = optional(map(string), {})
        parameters = optional(object({
          tags = optional(map(string), {})
        }), {})
        housekeeper = optional(object({
          schedule_expression = optional(string, null)
          state               = optional(string, null)
          tags                = optional(map(string), {})
          lambda = optional(object({
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            memory_size = optional(number, null)
            timeout     = optional(number, null)
          }), {})
          config = optional(object({
            tokenPath      = optional(string, null)
            minimumDaysOld = optional(number, null)
            dryRun         = optional(bool, null)
          }), {})
        }), {})
      }), {})

      observability = optional(object({
        logs = optional(object({
          level             = optional(string, null)
          retention_in_days = optional(number, null)
          kms_key_id        = optional(string, null)
          class             = optional(string, null)
          tags              = optional(map(string), {})
        }), {})
        tracing = optional(object({
          mode                  = optional(string, null)
          capture_http_requests = optional(bool, null)
          capture_error         = optional(bool, null)
        }), {})
        metrics = optional(object({
          enable    = optional(bool, null)
          namespace = optional(string, null)
          metric = optional(object({
            enable_github_app_rate_limit = optional(bool, null)
            enable_job_retry             = optional(bool, null)
          }), {})
        }), {})
      }), {})

      compute_provider = object({
        aws = optional(object({
          ec2 = optional(object({
            metadata_options = optional(object({
              instance_metadata_tags      = optional(string, "enabled")
              http_endpoint               = optional(string, "enabled")
              http_tokens                 = optional(string, "required")
              http_put_response_hop_limit = optional(number, 1)
            }), {})
            ami = optional(object({
              filter = optional(map(list(string)), { state = ["available"] })
              owners = optional(list(string), ["amazon"])
              id_ssm_parameter = optional(object({
                arn = string
              }), null)
              kms_key = optional(object({
                arn = string
              }), null)
            }), null)
            block_device_mappings = optional(list(object({
              delete_on_termination      = optional(bool, true)
              device_name                = optional(string, "/dev/xvda")
              encrypted                  = optional(bool, true)
              iops                       = optional(number)
              kms_key_id                 = optional(string)
              snapshot_id                = optional(string)
              throughput                 = optional(number)
              volume_initialization_rate = optional(number)
              volume_size                = number
              volume_type                = optional(string, "gp3")
              })), [{
              volume_size = 30
            }])
            create_service_linked_role_spot = optional(bool, false)
            credit_specification            = optional(string, null)
            ebs_optimized                   = optional(bool, false)
            cloudwatch_agent = optional(object({
              enabled = optional(bool, true)
              config  = optional(string, null)
            }), {})
            binaries_syncer = optional(object({
              enabled = optional(bool, null)
            }), {})
            detailed_monitoring_enabled = optional(bool, false)
            ssm_enabled                 = optional(bool, false)
            user_data = optional(object({
              enabled               = optional(bool, true)
              template              = optional(string, null)
              content               = optional(string, null)
              pre_install           = optional(string, "")
              post_install          = optional(string, "")
              debug_logging_enabled = optional(bool, false)
            }), {})
            instance_allocation_strategy   = optional(string, "lowest-price")
            instance_max_spot_price        = optional(string, null)
            instance_target_capacity_type  = optional(string, "spot")
            instance_type_priorities       = optional(map(number), null)
            instance_types                 = list(string)
            additional_security_group_ids  = optional(list(string), null)
            managed_security_group_enabled = optional(bool, null)
            egress_rules = optional(list(object({
              cidr_blocks      = list(string)
              ipv6_cidr_blocks = list(string)
              prefix_list_ids  = list(string)
              from_port        = number
              protocol         = string
              security_groups  = list(string)
              self             = bool
              to_port          = number
              description      = string
            })), null)
            instance_profile_path         = optional(string, null)
            key_name                      = optional(string, null)
            associate_public_ipv4_address = optional(bool, null)
            instance_profile = optional(object({
              name = string
            }), null)
            enable_on_demand_failover_for_errors = optional(list(string), [])
            scale_errors = optional(list(string), [
              "UnfulfillableCapacity",
              "MaxSpotInstanceCountExceeded",
              "TargetCapacityLimitExceededException",
              "RequestLimitExceeded",
              "ResourceLimitExceeded",
              "MaxSpotInstanceCountExceeded",
              "MaxSpotFleetRequestCountExceeded",
              "InsufficientInstanceCapacity",
              "InsufficientCapacityOnHost",
            ])
            subnet_ids = optional(list(string), null)
            vpc_id     = optional(string, null)
            cpu_options = optional(object({
              core_count            = optional(number)
              threads_per_core      = optional(number)
              amd_sev_snp           = optional(string)
              nested_virtualization = optional(string)
            }), null)
            placement = optional(object({
              affinity                = optional(string)
              availability_zone       = optional(string)
              group_id                = optional(string)
              group_name              = optional(string)
              host_id                 = optional(string)
              host_resource_group_arn = optional(string)
              spread_domain           = optional(string)
              tenancy                 = optional(string)
              partition_number        = optional(number)
            }), null)
            license_specifications = optional(list(object({
              license_configuration_arn = string
            })), [])
            use_dedicated_host = optional(bool, false)
            log_files = optional(list(object({
              log_group_name   = string
              prefix_log_group = bool
              file_path        = string
              log_stream_name  = string
              log_class        = optional(string, "STANDARD")
            })), null)
            tags = optional(map(string), {})
          }), null)
        }), {})
      })

    })), {})
  })
| `{}` | no | +| [experimental](#input\_experimental) | Opt-in experimental features. Omit this object to retain only the stable `multi_runner_config` behavior. Experimental schemas can change before they become stable.

Set `experimental.multi_runner_config` to opt into provider-oriented runner configurations. A non-empty experimental map completely replaces the stable top-level `multi_runner_config`; an empty map keeps stable entries on the unchanged `runners` module.

Sibling blocks provide global v2 defaults. Global and per-configuration values use `configuration override > experimental global v2 default` precedence. Migrated v2 consumers do not inherit a matching flat module input. Singleton shared components consume the global translated values documented below; per-configuration overrides affect only their runner configuration. Component-specific inputs without a nested counterpart continue consuming their existing flat inputs. Tag maps merge from broad to narrow instead of replacing the broader map.
Nullable per-configuration values inherit their experimental global value, so only defaults that apply to every runner configuration should be placed in a global block. When a runner configuration selects an external `runner.iam.role`, inherited managed policies and additional trust policy JSON are intentionally suppressed because this module does not manage that role.
Plan-shaping ownership wrappers remain nullable where their object presence controls Terraform graph shape. `ssm.kms_key_id` is instead an ARN-valued scalar; provider-owned IAM omits null KMS statements while still accepting an ARN whose value is unknown until apply. The unchanged shared webhook retains its legacy policy handling.

Global experimental fields support the following nested properties. These values apply directly when `experimental.multi_runner_config` is non-empty; in stable mode, flat inputs are translated into the same global shape.

- `tags`: Base tags for v2 build queues, runner configurations, the shared GitHub App Parameter Store module, webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `{}`. When any runner selects EC2 scale-set orchestration, values inherited into its effective Parameter Store tag map participate in that runtime's tag limits and validation.
- `roles.path`: Default IAM path for module-managed v2 runner and Lambda roles and for the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is null.
- `roles.permissions_boundary`: Default permissions-boundary ARN for module-managed v2 runner and Lambda roles and for the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is null.
- `runner.os`: Default runner operating system. The default is null; every runner configuration must resolve this field globally or locally.
- `runner.architecture`: Default runner distribution architecture. The default is null; every runner configuration must resolve this field globally or locally.
- `runner.disable_default_labels`: Omits the default self-hosted, operating-system, and architecture labels when true. The default is `false`.
- `runner.extra_labels`: Default additional labels combined with each runner configuration's matcher labels. The default is `[]`.
- `runner.group_name`: Default GitHub runner group. The default is `Default`.
- `runner.name_prefix`: Default prefix added to registered runner names. The default is an empty string. EC2 scale-set orchestration requires at most 45 ASCII letters, digits, dots, underscores, or hyphens.
- `runner.run_as_root`: Runs the runner service as root when supported by the provider. The default is `false`.
- `runner.run_as`: Default operating-system user when `run_as_root` is false. The default is `ec2-user`.
- `runner.auto_update_disabled`: Disables the GitHub runner application's built-in updater. The default is `false`.
- `runner.tags`: Default tags for common runner resources, currently the module-managed runner IAM role. The default is `{}`.
- `runner.hooks.job_started`: Default script content installed as the runner job-started hook. The default is an empty string.
- `runner.hooks.job_completed`: Default script content installed as the runner job-completed hook. The default is an empty string.
- `runner.iam.role`: Optional externally managed runner-role wrapper. The default is null; wrapper presence selects external role ownership during planning.
- `runner.iam.role.arn`: ARN of the externally managed runner role.
- `runner.iam.managed_policy_arns`: Named managed-policy ARNs attached to module-managed runner roles. The default is `{}`. Keep this map empty when the global `runner.iam.role` selects an external role; a runner configuration that explicitly selects its own external role suppresses the inherited map.
- `runner.iam.additional_trust_policy_json`: Optional IAM policy document merged with the provider trust policy for module-managed runner roles. The default is null. Keep it null when the global `runner.iam.role` selects an external role; a runner configuration that explicitly selects its own external role suppresses the inherited value.
- `runner.iam.path`: Runner-role IAM path. The default is null, which falls back to `roles.path`.
- `runner.iam.permissions_boundary`: Runner-role permissions-boundary ARN. The default is null, which falls back to `roles.permissions_boundary`.
- `github.app`: Primary GitHub App credentials persisted or selected by the shared Parameter Store module and used by v2 runner configurations. The default is null, but a non-empty v2 map requires this object.
- `github.app.key_base64`: Base64-encoded GitHub App private key supplied directly.
- `github.app.key_base64_ssm`: Existing Parameter Store private-key parameter wrapper. Set this or `key_base64`.
- `github.app.key_base64_ssm.arn`: ARN of the existing private-key parameter.
- `github.app.key_base64_ssm.name`: Name of the existing private-key parameter.
- `github.app.id`: GitHub App ID supplied directly.
- `github.app.id_ssm`: Existing Parameter Store app-ID parameter wrapper. Set this or `id`.
- `github.app.id_ssm.arn`: ARN of the existing app-ID parameter.
- `github.app.id_ssm.name`: Name of the existing app-ID parameter.
- `github.app.webhook_secret`: GitHub App webhook secret supplied directly.
- `github.app.webhook_secret_ssm`: Existing Parameter Store webhook-secret parameter wrapper. Set this or `webhook_secret`.
- `github.app.webhook_secret_ssm.arn`: ARN of the existing webhook-secret parameter.
- `github.app.webhook_secret_ssm.name`: Name of the existing webhook-secret parameter.
- `github.additional_apps`: Additional GitHub App credentials persisted or selected by the shared Parameter Store module and used for API request distribution. The default is `[]`.
- `github.additional_apps[].key_base64`: Base64-encoded private key supplied directly for an additional app.
- `github.additional_apps[].key_base64_ssm`: Existing Parameter Store private-key parameter wrapper. Set this or `key_base64`.
- `github.additional_apps[].key_base64_ssm.arn`: ARN of the existing additional-app private-key parameter.
- `github.additional_apps[].key_base64_ssm.name`: Name of the existing additional-app private-key parameter.
- `github.additional_apps[].id`: Additional GitHub App ID supplied directly.
- `github.additional_apps[].id_ssm`: Existing Parameter Store app-ID parameter wrapper. Set this or `id`.
- `github.additional_apps[].id_ssm.arn`: ARN of the existing additional-app ID parameter.
- `github.additional_apps[].id_ssm.name`: Name of the existing additional-app ID parameter.
- `github.additional_apps[].installation_id`: Optional installation ID supplied directly for an additional app.
- `github.additional_apps[].installation_id_ssm`: Optional existing Parameter Store installation-ID parameter wrapper.
- `github.additional_apps[].installation_id_ssm.arn`: ARN of the existing installation-ID parameter.
- `github.additional_apps[].installation_id_ssm.name`: Name of the existing installation-ID parameter.
- `github.enterprise_server.url`: GitHub Enterprise Server URL used by v2 runner-config GitHub clients and the shared termination watcher. The default is null.
- `github.enterprise_server.ssl_verify`: Enables TLS certificate verification for v2 runner-config GitHub clients. Scale-set controllers apply a disabled value to that reconciler's GitHub App and scale-set requests without changing process-global TLS behavior. The default is `true`.
- `github.user_agent`: Client identity used by v2 runner-config GitHub clients. Scale-set controllers preserve the required structured protocol User-Agent and place this value in its `system` field. The default is `github-aws-runners`.
- `lambda.artifact.s3.bucket`: Optional shared S3 bucket containing Lambda deployment artifacts for v2 runner configurations and their SSM housekeepers, the webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is null. A component selects an object from this bucket only when its own `artifact.s3` wrapper is present.
- `lambda.runtime`: Runtime for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `nodejs24.x`.
- `lambda.architecture`: Architecture for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `arm64`.
- `lambda.principals`: Additional principals allowed to assume v2 runner-config, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is `[]`; list membership must be known during planning because it creates IAM principal blocks.
- `lambda.principals[].type`: IAM principal type.
- `lambda.principals[].identifiers`: IAM principal identifiers for the type.
- `lambda.subnet_ids`: Subnets for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `[]`.
- `lambda.security_group_ids`: Security groups for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `[]`.
- `lambda.tags`: Default tags for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `{}`.
- `lambda.role.path`: IAM path for module-managed v2 Lambda roles and the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper roles. The default is null, which falls back to `roles.path`.
- `lambda.role.permissions_boundary`: Permissions-boundary ARN for module-managed v2 Lambda roles and the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper roles. The default is null, which falls back to `roles.permissions_boundary`.
- `orchestration_provider.webhook`: Global defaults for the workflow-job webhook control plane. This is a defaults namespace; every runner configuration still selects its demand controller independently under `multi_runner_config[].orchestration_provider`, where exactly one typed provider block must be non-null.
- `orchestration_provider.webhook.queue_selection_strategy`: Queue-selection strategy when multiple runner configurations match a job equally well. The default is `first`, which deterministically selects the first matching queue by priority. `random` spreads jobs across equally matched queues. `all` dispatches to every matching queue, favoring startup speed at the cost of multiple runner launches and registrations for one job.
- `orchestration_provider.webhook.eventbridge.enable`: Routes accepted webhook events through EventBridge when true. The default is `true`, and the value must be known during planning because it selects the webhook implementation.
- `orchestration_provider.webhook.eventbridge.accept_events`: EventBridge event types accepted by the shared webhook. The default is `[]`, which accepts all supported events.
- `orchestration_provider.webhook.matcher_config_parameter_store_tier`: Parameter Store tier for the shared matcher configuration. The default is `Standard`; valid values are `Standard` and `Advanced`. The value must be known during planning because it determines the matcher-parameter chunks.
- `orchestration_provider.webhook.github.repository_white_list`: Repository full names allowed to use the shared webhook. The default is `[]`, which disables repository filtering.
- `orchestration_provider.webhook.runner.boot_time_in_minutes`: Default expected runner boot duration used by webhook scale-down and pool controls. The default is `5`.
- `orchestration_provider.webhook.runner.ephemeral`: Registers webhook-orchestrated runners in ephemeral mode by default. The default is `false`.
- `orchestration_provider.webhook.runner.jit_config_enabled`: Explicit default for just-in-time runner configuration. The default is null, which follows the resolved `ephemeral` mode.
- `orchestration_provider.webhook.runner.maximum_count`: Default maximum number of runners managed by the webhook orchestration provider per runner configuration. The default is null; every webhook runner configuration must resolve this field globally or locally.
- `orchestration_provider.webhook.lambda.artifact`: Shared runner-control-plane artifact used by webhook scale, pool, and job-retry components. Set at most one of `zip` or `s3`; when both are null, the packaged runner archive is used.
- `orchestration_provider.webhook.lambda.artifact.zip`: Optional local path to the shared runner-control-plane Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for runner-control-plane Lambdas, must be known during planning, and requires a non-null shared bucket and key.
- `orchestration_provider.webhook.lambda.artifact.s3.key`: Object key of the shared runner-control-plane Lambda archive.
- `orchestration_provider.webhook.lambda.artifact.s3.object_version`: Optional object version of the shared runner-control-plane Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.scale.up.memory_size`: Scale-up Lambda memory in MB. The default is `512`.
- `orchestration_provider.webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds. The default is `30`.
- `orchestration_provider.webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for scale-up. The default is `1`; use `-1` for unreserved concurrency.
- `orchestration_provider.webhook.lambda.scale.up.job_queued_check_enabled`: Enables queued-job verification before scaling. The default is null, which follows the resolved runner mode.
- `orchestration_provider.webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered per scale-up invocation. The default is `10`.
- `orchestration_provider.webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum build-queue batching window. The default is `0`.
- `orchestration_provider.webhook.lambda.scale.up.tags`: Default tags for scale-up resources. The default is `{}`.
- `orchestration_provider.webhook.lambda.scale.down.memory_size`: Scale-down Lambda memory in MB. The default is `512`.
- `orchestration_provider.webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds. The default is `60`.
- `orchestration_provider.webhook.lambda.scale.down.schedule_expression`: EventBridge schedule for scale-down. The default is `cron(*/5 * * * ? *)`.
- `orchestration_provider.webhook.lambda.scale.down.minimum_running_time_in_minutes`: Minimum runner age before scale-down may terminate it. The default is null, which selects the operating-system default.
- `orchestration_provider.webhook.lambda.scale.down.idle_config`: Default time-based desired idle-runner configurations. The default is `[]`.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate the cron expression.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners retained during the matching period.
- `orchestration_provider.webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed. The default is `oldest_first`.
- `orchestration_provider.webhook.lambda.scale.down.tags`: Default tags for scale-down resources. The default is `{}`.
- `orchestration_provider.webhook.lambda.webhook.artifact`: Shared-webhook artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `orchestration_provider.webhook.lambda.webhook.artifact.zip`: Optional local path to the shared-webhook Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.webhook.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for the webhook and requires a non-null shared bucket and key.
- `orchestration_provider.webhook.lambda.webhook.artifact.s3.key`: Object key of the shared-webhook Lambda archive.
- `orchestration_provider.webhook.lambda.webhook.artifact.s3.object_version`: Optional object version of the shared-webhook Lambda archive. The default is null.
- `orchestration_provider.webhook.lambda.webhook.api_gateway_access_log_settings`: Optional API Gateway access-log destination and format for the shared webhook. The default is null, and wrapper presence must be known during planning because it controls the access-log block.
- `orchestration_provider.webhook.lambda.webhook.api_gateway_access_log_settings.destination_arn`: CloudWatch Logs destination ARN for API Gateway access logs.
- `orchestration_provider.webhook.lambda.webhook.api_gateway_access_log_settings.format`: API Gateway access-log format.
- `orchestration_provider.webhook.lambda.webhook.memory_size`: Shared-webhook Lambda memory in MB. The default is `256`.
- `orchestration_provider.webhook.lambda.webhook.timeout`: Shared-webhook Lambda timeout in seconds. The default is `10`.
- `orchestration_provider.webhook.lambda.webhook.tags`: Additional tags for the shared webhook Lambda, merged after `lambda.tags`. The default is `{}`.
- `orchestration_provider.webhook.lambda.pool.memory_size`: Pool Lambda memory in MB. The default is `512`.
- `orchestration_provider.webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds. The default is `60`.
- `orchestration_provider.webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. The default is `1`; use `-1` for unreserved concurrency.
- `orchestration_provider.webhook.lambda.pool.config`: Default scheduled target pool sizes. The default is `[]`, which disables the pool component.
- `orchestration_provider.webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `orchestration_provider.webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `orchestration_provider.webhook.lambda.pool.config[].size`: Desired runner-pool size for the schedule.
- `orchestration_provider.webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity. The default is `false`.
- `orchestration_provider.webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used for pooled runners. The default is null.
- `orchestration_provider.webhook.lambda.pool.tags`: Default tags for pool resources. The default is `{}`.
- `orchestration_provider.webhook.queue.delay_webhook_event`: Default delay in seconds applied to accepted webhook jobs. The default is `30`.
- `orchestration_provider.webhook.queue.job_queue_retention_in_seconds`: Default build-queue message retention period in seconds. The default is `86400`.
- `orchestration_provider.webhook.queue.visibility_timeout_seconds`: Default build-queue visibility timeout. The default is `180`; set it to at least six times every resolved `orchestration_provider.webhook.lambda.scale.up.timeout` that inherits it.
- `orchestration_provider.webhook.queue.redrive_build_queue.enabled`: Creates and attaches a dead-letter queue to every v2 build queue by default. The default is `false`.
- `orchestration_provider.webhook.queue.redrive_build_queue.maxReceiveCount`: Default number of receives before a message moves to the dead-letter queue. The default is null while redrive is disabled and must resolve to a value greater than zero when redrive is enabled.
- `orchestration_provider.webhook.queue.tags`: Default tags for v2 build queues and dead-letter queues. The default is `{}`.
- `orchestration_provider.webhook.queue.encryption`: Global at-rest encryption configuration for the multi-runner build queues and their dead-letter queues. It does not configure runner-config job-retry queues. Omitting the whole block selects SQS-managed encryption and defaults the two KMS attributes to null. When supplying the block explicitly, provide all three leaf attributes and use null for the inactive mode.
- `orchestration_provider.webhook.queue.encryption.kms_data_key_reuse_period_seconds`: KMS data-key reuse period in seconds. This key is required syntactically in an explicit `queue.encryption` object but may be null; it is used only with `kms_master_key_id`.
- `orchestration_provider.webhook.queue.encryption.kms_master_key_id`: KMS key ARN used for queue encryption. This AWS-facing field name is retained for compatibility, but non-null values must be key ARNs because runner-config IAM policies cannot use key IDs or aliases as resources. The field is required syntactically in an explicit `queue.encryption` object but may be null, and a computed ARN may remain unknown until apply. It is independent from `ssm.kms_key_id` and is forwarded separately to each webhook runner configuration so runner-config scale-up and job-retry roles receive only the build-queue KMS permissions they require.
- `orchestration_provider.webhook.queue.encryption.sqs_managed_sse_enabled`: Selects the non-KMS mode: `true` enables SQS-managed encryption and `false` explicitly disables queue encryption. This key is required syntactically in an explicit `queue.encryption` object and must be null when `kms_master_key_id` is set. Omitting the whole encryption block defaults it to `true`.
- `orchestration_provider.scale_set`: Shared scale-set controller topology and runtime defaults. Multi-runner creates this provider once when at least one runner config selects `scale_set`; the provider then packs the selected reconcilers into controller groups.
- `orchestration_provider.scale_set.grouping.strategy`: Controller grouping strategy. `compute_provider` (the default) creates one controller group per compute-provider type, `runner_config` creates one group per runner config, and `custom` uses the explicit group map.
- `orchestration_provider.scale_set.grouping.custom`: Explicit controller groups. This must be non-null only when the strategy is `custom`, and membership must cover every selected scale-set runner config exactly once.
- `orchestration_provider.scale_set.grouping.custom.groups`: Controller groups keyed by stable group name.
- `orchestration_provider.scale_set.grouping.custom.groups..runner_configs`: Set of scale-set runner-config keys assigned to the group.
- `orchestration_provider.scale_set.container.image`: Controller image reference. The default is null, which selects the release's official scale-set service image; callers can override it with a compatible image.
- `orchestration_provider.scale_set.container.user`: Numeric user and optional group used by the hardened Fargate container. The default is `10001:10001`.
- `orchestration_provider.scale_set.container.health_port`: Loopback HTTP health-listener port. The default is `8080`.
- `orchestration_provider.scale_set.container.health_path`: ECS liveness endpoint. The only supported value is `/healthz`.
- `orchestration_provider.scale_set.container.health_check_command`: Optional ECS container health-check command. Null uses the built-in Node probe against `/healthz`.
- `orchestration_provider.scale_set.container.health_check_interval`: ECS health-check interval in seconds. The default is `30`.
- `orchestration_provider.scale_set.container.health_check_timeout`: ECS health-check timeout in seconds. The default is `5`.
- `orchestration_provider.scale_set.container.health_check_retries`: Consecutive failed checks before ECS marks the task unhealthy. The default is `3`.
- `orchestration_provider.scale_set.container.health_check_start_period`: Startup grace period for ECS health checks in seconds. The default is `30`.
- `orchestration_provider.scale_set.container.health_stale_after_seconds`: Maximum allowed age of a successful controller reconciliation before liveness fails. The default is `180`.
- `orchestration_provider.scale_set.container.shutdown_timeout_seconds`: Maximum controller shutdown-drain period. The default is `110`.
- `orchestration_provider.scale_set.container.session_close_timeout_seconds`: Maximum wait for a GitHub scale-set session to close during shutdown. The default is `10`.
- `orchestration_provider.scale_set.container.reconnect_initial_backoff_seconds`: Initial reconnect delay after a transient session failure. The default is `1`.
- `orchestration_provider.scale_set.container.reconnect_max_backoff_seconds`: Maximum transient-session reconnect delay. The default is `30`.
- `orchestration_provider.scale_set.container.stop_timeout_seconds`: ECS stop timeout. The default is `120` and must leave enough time for the configured controller shutdown.
- `orchestration_provider.scale_set.container.ecr_repository`: Optional private-ECR repository wrapper. Its presence adds narrowly scoped image-pull permissions to the execution role.
- `orchestration_provider.scale_set.container.ecr_repository.arn`: ARN of the private ECR repository containing the selected image.
- `orchestration_provider.scale_set.config_store.path_prefix`: SSM path prefix for non-secret reconciler manifests. Null derives `//scale-set-controller`.
- `orchestration_provider.scale_set.config_store.tier`: Parameter Store tier for reconciler manifests. The default is `Standard`; `Advanced` permits larger manifests.
- `orchestration_provider.scale_set.config_store.tags`: Tags applied to reconciler manifest parameters. The default is `{}`.
- `orchestration_provider.scale_set.ecs.cluster.mode`: ECS cluster ownership mode. `managed` (the default) creates a cluster; `external` uses `cluster.arn`.
- `orchestration_provider.scale_set.ecs.cluster.arn`: Existing ECS cluster ARN required in external mode.
- `orchestration_provider.scale_set.ecs.cluster.name`: Optional name for a managed ECS cluster. Null derives a stable name.
- `orchestration_provider.scale_set.ecs.cluster.container_insights`: Enables Container Insights on a managed cluster. The default is `true`.
- `orchestration_provider.scale_set.ecs.task.cpu`: Fargate task CPU units per controller group. The default is `512`.
- `orchestration_provider.scale_set.ecs.task.memory`: Fargate task memory in MiB per controller group. The default is `1024`.
- `orchestration_provider.scale_set.ecs.task.cpu_architecture`: Fargate CPU architecture. The default is `X86_64`; `ARM64` is also supported when the selected image is compatible.
- `orchestration_provider.scale_set.ecs.task.ephemeral_storage.size_in_gib`: Optional Fargate ephemeral-storage size in GiB. Null uses the AWS default.
- `orchestration_provider.scale_set.ecs.service.platform_version`: Fargate platform version. The default is `LATEST`.
- `orchestration_provider.scale_set.ecs.iam.path`: IAM path for controller task and execution roles. The default is `/`.
- `orchestration_provider.scale_set.ecs.iam.permissions_boundary`: Optional permissions-boundary ARN for controller task and execution roles. The default is null.
- `orchestration_provider.scale_set.network.vpc_id`: VPC for private Fargate controller tasks. It is required when any runner config selects `scale_set`.
- `orchestration_provider.scale_set.network.subnet_ids`: Private subnets for Fargate controller tasks. At least one is required when any runner config selects `scale_set`; tasks never receive public IP addresses.
- `orchestration_provider.scale_set.network.https_egress.ipv4_cidrs`: IPv4 CIDRs allowed for outbound HTTPS. The default is `0.0.0.0/0`; use controlled NAT, firewall, or proxy routing when required.
- `orchestration_provider.scale_set.network.https_egress.ipv6_cidrs`: IPv6 CIDRs allowed for outbound HTTPS. The default is `[]`.
- `orchestration_provider.scale_set.logging.retention_in_days`: CloudWatch Logs retention period for controller groups. The default is `30`.
- `orchestration_provider.scale_set.logging.kms_key_arn`: Optional customer-managed KMS key ARN for controller log groups. The default is null.
- `orchestration_provider.scale_set.logging.log_group_class`: Controller log-group class. The default is `STANDARD`.
- `orchestration_provider.scale_set.logging.tags`: Tags applied to controller log groups. The default is `{}`.
- `orchestration_provider.scale_set.tags`: Tags applied to shared scale-set controller resources after global experimental tags. The default is `{}`.
- `ssm.paths.root`: Base Parameter Store path for shared GitHub App and webhook parameters and for all v2 runner configurations. The schema default is null, which derives `/github-action-runners/`; normalization appends the configuration key only for configuration-owned paths.
- `ssm.paths.app`: Shared GitHub App credential path segment below `ssm.paths.root`. The default is `app`.
- `ssm.paths.webhook`: Shared webhook matcher-configuration path segment below `ssm.paths.root`. The default is `webhook`.
- `ssm.paths.tokens`: Runner registration-token and JIT-configuration path segment below each runner-configuration root. The default is `runners/tokens`.
- `ssm.paths.config`: Persistent runner-configuration path segment below each runner-configuration root. The default is `runners/config`.
- `ssm.kms_key_id`: Optional global KMS key ARN that encrypts shared GitHub App parameters, configures the webhook and termination watcher, and adds matching decrypt permissions to every v2 runner configuration. The default is null and its value may be unknown until apply. It does not select encryption for runtime-created runner-configuration parameters.
- `ssm.tags`: Default tags for the shared GitHub App Parameter Store module and runner-configuration-owned SSM resources. The default is `{}`. For EC2 scale-set orchestration, inherited values participate in the effective 45-tag runtime limit and key/value validation.
- `ssm.parameters.tags`: Default tags for Terraform-managed and runtime-created runner-configuration parameters. The default is `{}` and inherited values participate in the EC2 scale-set runtime tag limit.
- `ssm.housekeeper.schedule_expression`: Default EventBridge schedule for each runner-configuration SSM housekeeper. The default is `rate(1 day)`.
- `ssm.housekeeper.state`: Default EventBridge rule state for each runner-configuration SSM housekeeper. The default is `ENABLED`.
- `ssm.housekeeper.tags`: Default tags for SSM housekeeper resources. The default is `{}`.
- `ssm.housekeeper.lambda.artifact`: Default SSM-housekeeper artifact selection. Set at most one of `zip` or `s3`; when both are null, runner-config uses its packaged runner control-plane archive.
- `ssm.housekeeper.lambda.artifact.zip`: Optional local path to the SSM-housekeeper Lambda archive. The default is null.
- `ssm.housekeeper.lambda.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for the SSM housekeeper, must be known during planning, and requires a non-null shared bucket and key.
- `ssm.housekeeper.lambda.artifact.s3.key`: Object key of the SSM-housekeeper Lambda archive.
- `ssm.housekeeper.lambda.artifact.s3.object_version`: Optional object version of the SSM-housekeeper Lambda archive. The default is null.
- `ssm.housekeeper.lambda.memory_size`: Default SSM housekeeper Lambda memory in MB. The default is `512`.
- `ssm.housekeeper.lambda.timeout`: Default SSM housekeeper Lambda timeout in seconds. The default is `60`.
- `ssm.housekeeper.config.tokenPath`: Optional cleanup path shared by every runner configuration. The default is null; omit it so each runner configuration derives its isolated token path.
- `ssm.housekeeper.config.minimumDaysOld`: Minimum parameter age in days before deletion is allowed. The default is `1`.
- `ssm.housekeeper.config.dryRun`: Reports eligible parameters without deleting them when true. The default is `false`.
- `observability.logs.level`: Application log level for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `info`.
- `observability.logs.retention_in_days`: CloudWatch Logs retention for v2 runner-config resources and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `180`.
- `observability.logs.kms_key_id`: Optional KMS key ID or ARN for v2 runner-config log groups and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is null.
- `observability.logs.class`: CloudWatch log-group class for v2 runner-config resources and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `STANDARD`.
- `observability.logs.tags`: Default tags for v2 runner-config log groups. The default is `{}`; shared singleton functions receive `tags` and `lambda.tags` instead.
- `observability.tracing.mode`: Optional Lambda tracing mode for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is null; its nullness must be known during planning because it controls X-Ray IAM and tracing blocks.
- `observability.tracing.capture_http_requests`: Enables HTTP request capture in the tracing helper for v2 runner-config functions and translated shared consumers. The default is `false`.
- `observability.tracing.capture_error`: Enables error capture in the tracing helper for v2 runner-config functions and translated shared consumers. The default is `false`.
- `observability.metrics.enable`: Enables module-emitted metrics for v2 runner configurations and the shared termination watcher. The default is `false`.
- `observability.metrics.namespace`: CloudWatch namespace for v2 runner-config and termination-watcher metrics. The default is `GitHub Runners`.
- `observability.metrics.metric.enable_github_app_rate_limit`: Emits GitHub App rate-limit metrics when metrics are enabled. The default is `true`.
- `observability.metrics.metric.enable_job_retry`: Emits job-retry metrics when metrics are enabled. The default is `true`.
- `observability.metrics.metric.enable_spot_termination`: Emits Spot termination metrics from the shared termination watcher when metrics are enabled. The default is `true`.
- `observability.metrics.metric.enable_spot_termination_warning`: Emits Spot termination-warning metrics from the shared termination watcher when metrics are enabled. The default is `true`.
- `compute_provider`: Shared compute-provider defaults grouped first by cloud and then by provider type. Global defaults do not select a provider for any runner configuration.
- `compute_provider.selections`: Optional plan-shaping map keyed by runner-configuration key. Each entry identifies the namespace and type of the configuration's selected compute-provider block. The default is null, which discovers selections from the typed provider blocks. Set this map when unrelated apply-time values make that discovery unknown; its keys and values must be known during planning and cover every runner configuration exactly once.
- `compute_provider.selections[].namespace`: Compute-provider namespace. The only currently supported value is `aws`.
- `compute_provider.selections[].type`: Compute-provider type within the namespace. The only currently supported value is `ec2`.
- `compute_provider.aws`: Shared defaults for AWS compute providers.
- `compute_provider.aws.ec2`: Shared defaults for AWS EC2 runner configurations.
- `compute_provider.aws.ec2.vpc_id`: Shared VPC default for v2 EC2 runner configurations. The default is null; every EC2 runner configuration must resolve this field globally or locally.
- `compute_provider.aws.ec2.subnet_ids`: Shared subnet default for v2 EC2 runner configurations. The default is null; every EC2 runner configuration must resolve this field globally or locally.
- `compute_provider.aws.ec2.managed_security_group_enabled`: Creates the module-managed runner security group by default. The default is `true`.
- `compute_provider.aws.ec2.egress_rules`: Shared runner security-group egress rules. The default is one IPv4/IPv6 allow-all rule; v2 does not inherit flat `runner_egress_rules`.
- `compute_provider.aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].prefix_list_ids`: Prefix-list destinations for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].from_port`: Start of the egress rule port range.
- `compute_provider.aws.ec2.egress_rules[].protocol`: Egress rule protocol; `-1` allows every protocol.
- `compute_provider.aws.ec2.egress_rules[].security_groups`: Destination security-group IDs for an egress rule.
- `compute_provider.aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `compute_provider.aws.ec2.egress_rules[].to_port`: End of the egress rule port range.
- `compute_provider.aws.ec2.egress_rules[].description`: Optional egress rule description.
- `compute_provider.aws.ec2.additional_security_group_ids`: Existing security groups attached to every v2 EC2 runner configuration unless overridden. The default is `[]`.
- `compute_provider.aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration inherited by v2 EC2 runner configurations. The default is null; enablement remains configuration-owned.
- `compute_provider.aws.ec2.instance_profile_path`: IAM path for module-managed EC2 instance profiles. The default is null.
- `compute_provider.aws.ec2.key_name`: Optional EC2 key-pair name inherited by v2 EC2 runner configurations. The default is null.
- `compute_provider.aws.ec2.associate_public_ipv4_address`: Associates public IPv4 addresses with v2 EC2 runners unless overridden. The default is `false`.
- `compute_provider.aws.ec2.tags`: Default tags for runtime EC2 resources. The default is `{}` and runner-configuration EC2 tags take precedence.
- `compute_provider.aws.ec2.ami.housekeeper`: Global configuration for the shared AMI-housekeeper Lambda.
- `compute_provider.aws.ec2.ami.housekeeper.enabled`: Creates the shared AMI housekeeper when true. The default is `false`, and the value must be known during planning because it controls the module instance.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config`: AMI cleanup selection and safety settings. The default is `{}`, which resolves the leaf defaults described below in the AMI-housekeeper module.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.maxItems`: Optional maximum number of AMIs queried for cleanup. The default is null, which applies no maximum.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.minimumDaysOld`: Minimum AMI age in days before cleanup. The effective default is `30`.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.amiFilters`: AMI filters, each containing `Name` and `Values`. The effective default selects images with `state = available` and `image-type = machine`.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.launchTemplateNames`: Optional launch-template names whose referenced AMIs are retained. The default is null, which selects no launch templates.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.ssmParameterNames`: Optional Parameter Store names whose referenced AMIs are retained. The default is null, which selects no parameters.
- `compute_provider.aws.ec2.ami.housekeeper.cleanup_config.dryRun`: Reports eligible AMIs without deregistering them when true. The effective default is `false`.
- `compute_provider.aws.ec2.ami.housekeeper.artifact`: AMI-housekeeper artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.zip`: Optional local path to the AMI-housekeeper Lambda archive. The default is null.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 and requires a non-null shared bucket and key.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.s3.key`: Object key of the AMI-housekeeper Lambda archive.
- `compute_provider.aws.ec2.ami.housekeeper.artifact.s3.object_version`: Optional object version of the AMI-housekeeper Lambda archive. The default is null.
- `compute_provider.aws.ec2.ami.housekeeper.lambda.memory_size`: AMI-housekeeper Lambda memory in MB. The default is `256`.
- `compute_provider.aws.ec2.ami.housekeeper.lambda.timeout`: AMI-housekeeper Lambda timeout in seconds. The default is `300`.
- `compute_provider.aws.ec2.ami.housekeeper.schedule.expression`: AMI-housekeeper EventBridge schedule expression. The default is `cron(11 7 * * ? *)`.
- `compute_provider.aws.ec2.instance_termination_watcher`: Global configuration for the shared EC2 instance-termination watcher.
- `compute_provider.aws.ec2.instance_termination_watcher.enabled`: Creates the shared EC2 termination watcher when true. The default is `false`, and the value must be known during planning because it controls the module instance. Scale-set runner configs may use the watcher for logging and metrics only, with runner deregistration disabled.
- `compute_provider.aws.ec2.instance_termination_watcher.features.enable_spot_termination_handler`: Enables the Spot termination-event handler. The default is `true`, and the value must be known during planning because it controls child resources.
- `compute_provider.aws.ec2.instance_termination_watcher.features.enable_spot_termination_notification_watcher`: Enables the Spot interruption-warning watcher. The default is `true`, and the value must be known during planning because it controls child resources.
- `compute_provider.aws.ec2.instance_termination_watcher.enable_runner_deregistration`: Deregisters terminated runners from GitHub when true. The default is `true`, and the value must be known during planning because it controls deregistration resources. This must be false when any runner config selects scale-set orchestration because the scale-set reconciler owns GitHub runner deregistration.
- `compute_provider.aws.ec2.instance_termination_watcher.environment_variables`: Additional termination-watcher Lambda environment variables. The default is `{}`.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact`: Termination-watcher artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.zip`: Optional local path to the termination-watcher Lambda archive. The default is null.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 and requires a non-null shared bucket and key.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.s3.key`: Object key of the termination-watcher Lambda archive.
- `compute_provider.aws.ec2.instance_termination_watcher.artifact.s3.object_version`: Optional object version of the termination-watcher Lambda archive. The default is null.
- `compute_provider.aws.ec2.instance_termination_watcher.lambda.memory_size`: Optional watcher Lambda memory in MB. The default is null, which delegates to the termination-watcher module.
- `compute_provider.aws.ec2.instance_termination_watcher.lambda.timeout`: Optional watcher Lambda timeout in seconds. The default is null, which delegates to the termination-watcher module.
- `compute_provider.aws.ec2.runner_binaries`: Global configuration for the shared runner-distribution buckets and syncers, created once per unique enabled runner operating-system and architecture pair.
- `compute_provider.aws.ec2.runner_binaries.enabled`: Default for whether EC2 runner configurations use the synchronized runner distribution. The default is `true`; a runner configuration may override it through `compute_provider.aws.ec2.binaries_syncer.enabled`. Every resolved enable value must be known during planning because it determines the syncer module instances.
- `compute_provider.aws.ec2.runner_binaries.targets`: Optional plan-shaping map of shared runner-distribution targets, keyed by `_`. The default is null, which discovers enabled targets from runner configurations. Set this map when unrelated apply-time values make that discovery unknown. An empty map creates no shared binary syncers; every enabled runner platform must have a corresponding entry.
- `compute_provider.aws.ec2.runner_binaries.targets[].os`: Runner operating system for the target. Valid values are `linux`, `osx`, and `windows`.
- `compute_provider.aws.ec2.runner_binaries.targets[].architecture`: Runner distribution architecture for the target. Valid values are `x64` and `arm64`.
- `compute_provider.aws.ec2.runner_binaries.s3`: Settings for each shared runner-distribution bucket.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption`: Server-side encryption settings for each distribution bucket.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.enabled`: Creates an explicit distribution-bucket encryption configuration when true. The default is `true`, and the value must be known during planning because it controls resource shape. Keep `kms_master_key_id` null when this is false.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.bucket_key_enabled`: Optional S3 Bucket Key setting. The default is null.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.sse_algorithm`: Server-side encryption algorithm. The default is `AES256`; valid values are `AES256`, `aws:kms`, and `aws:kms:dsse`. When `kms_master_key_id` is set, use one of the KMS algorithms.
- `compute_provider.aws.ec2.runner_binaries.s3.encryption.kms_master_key_id`: Optional KMS key identifier for the distribution bucket. The default is null, and its nullness must be known during planning because it controls the syncer KMS policy. The syncer receives KMS access, but runner roles do not derive `kms:Decrypt` from this field; grant runner roles decrypt access separately when using a CMK.
- `compute_provider.aws.ec2.runner_binaries.s3.tags`: Additional tags for each distribution bucket. The default is `{}`; these merge after global `tags`.
- `compute_provider.aws.ec2.runner_binaries.s3.versioning`: Distribution-bucket versioning state. The default is `Disabled`; valid values are `Disabled`, `Enabled`, and `Suspended`. After enabling versioning, Terraform cannot return the bucket to `Disabled`; use `Suspended` instead.
- `compute_provider.aws.ec2.runner_binaries.s3.logging`: Optional access-logging settings for each distribution bucket.
- `compute_provider.aws.ec2.runner_binaries.s3.logging.bucket`: Existing target bucket for access logs. The default is null, and its nullness must be known during planning because it controls the logging resource.
- `compute_provider.aws.ec2.runner_binaries.s3.logging.prefix`: Optional access-log prefix. The default is null, which uses the distribution-bucket name when logging is enabled. A non-null prefix requires `logging.bucket`.
- `compute_provider.aws.ec2.runner_binaries.syncer`: Component-specific settings for the shared runner-binary syncer Lambda.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact`: Syncer Lambda artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.zip`: Optional local path to the syncer Lambda archive. The default is null.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 and requires a non-null shared bucket and key.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.s3.key`: Object key of the syncer Lambda archive. Required when the `s3` wrapper is present.
- `compute_provider.aws.ec2.runner_binaries.syncer.artifact.s3.object_version`: Optional object version of the syncer Lambda archive. The default is null.
- `compute_provider.aws.ec2.runner_binaries.syncer.lambda`: Syncer Lambda sizing settings. Runtime, architecture, networking, role, tags, logging, and tracing come from their global experimental blocks.
- `compute_provider.aws.ec2.runner_binaries.syncer.lambda.memory_size`: Memory allocated to the syncer Lambda in MB. The default is `256`.
- `compute_provider.aws.ec2.runner_binaries.syncer.lambda.timeout`: Syncer Lambda timeout in seconds. The default is `300`.
- `compute_provider.aws.ec2.runner_binaries.syncer.schedule`: EventBridge schedule settings for the syncer Lambda.
- `compute_provider.aws.ec2.runner_binaries.syncer.schedule.expression`: EventBridge schedule expression. The default is `cron(27 * * * ? *)`.
- `compute_provider.aws.ec2.runner_binaries.syncer.schedule.state`: EventBridge rule state. The default is `ENABLED`; valid values are `DISABLED`, `ENABLED`, and `ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS`.

Each `experimental.multi_runner_config` entry supports the following nested fields:

- `multi_runner_config[].tags`: Configuration-wide tags. These override global `experimental.tags`; narrower component and compute-provider tag maps take precedence for their resources. Flat `tags` are not merged into v2 queues or runner configurations.
- `multi_runner_config[].runner.os`: Runner operating system.
- `multi_runner_config[].runner.architecture`: Runner distribution architecture.
- `multi_runner_config[].runner.disable_default_labels`: Prevents GitHub default labels from being registered.
- `multi_runner_config[].runner.extra_labels`: Additional labels combined with `orchestration_provider.webhook.matcherConfig.labelMatchers` for webhook runner configurations. Default self-hosted, operating-system, and architecture labels are also included unless `runner.disable_default_labels` is true.
- `multi_runner_config[].runner.group_name`: GitHub runner group used during registration.
- `multi_runner_config[].runner.name_prefix`: Prefix added to registered runner names. EC2 scale-set orchestration requires at most 45 ASCII letters, digits, dots, underscores, or hyphens.
- `multi_runner_config[].runner.run_as_root`: Runs the runner service as root when supported by the compute provider.
- `multi_runner_config[].runner.run_as`: Operating-system user used when `run_as_root` is false.
- `multi_runner_config[].runner.auto_update_disabled`: Disables the GitHub runner application's built-in updater.
- `multi_runner_config[].runner.tags`: Tags for common runner resources, currently the managed runner IAM role. These override entry-level `tags`.
- `multi_runner_config[].runner.hooks.job_started`: Script content installed as the runner job-started hook.
- `multi_runner_config[].runner.hooks.job_completed`: Script content installed as the runner job-completed hook.
- `multi_runner_config[].runner.iam.role`: Optional externally managed runner-role wrapper. When set, inherited managed policies and trust additions are suppressed.
- `multi_runner_config[].runner.iam.role.arn`: ARN of an externally managed runner role. When set, `runner-config` does not create or modify that role.
- `multi_runner_config[].runner.iam.managed_policy_arns`: Named managed-policy ARNs attached to the module-managed runner role. Keep this empty or null when the runner configuration selects an external `runner.iam.role`.
- `multi_runner_config[].runner.iam.additional_trust_policy_json`: Optional IAM policy document merged with the selected compute provider's default runner-role trust policy. Keep it null when the runner configuration selects an external `runner.iam.role`.
- `multi_runner_config[].runner.iam.path`: IAM path for the module-managed runner role.
- `multi_runner_config[].runner.iam.permissions_boundary`: Permissions-boundary ARN for the module-managed runner role.
- `multi_runner_config[].lambda`: Lambda runtime, architecture, networking, tags, and role substrate shared by orchestration and the runner-configuration SSM housekeeper. Scale function settings belong under `orchestration_provider.webhook.lambda`.
- `multi_runner_config[].lambda.runtime`: Per-configuration runtime override for runner-config Lambda functions.
- `multi_runner_config[].lambda.architecture`: Per-configuration architecture override for runner-config Lambda functions.
- `multi_runner_config[].lambda.subnet_ids`: Per-configuration subnet override for runner-config Lambda functions.
- `multi_runner_config[].lambda.security_group_ids`: Per-configuration security-group override for runner-config Lambda functions.
- `multi_runner_config[].lambda.tags`: Per-configuration tags for control-plane Lambda functions. Component tags override this map.
- `multi_runner_config[].lambda.role.path`: Per-configuration IAM path for module-managed Lambda roles. Null inherits `experimental.lambda.role.path`, then `experimental.roles.path`.
- `multi_runner_config[].lambda.role.permissions_boundary`: Per-configuration permissions-boundary ARN for module-managed Lambda roles. Null inherits `experimental.lambda.role.permissions_boundary`, then `experimental.roles.permissions_boundary`.
- `multi_runner_config[].orchestration_provider`: Demand-controller selection. Exactly one of the typed `webhook` or `scale_set` blocks must be non-null, and the selected wrapper must be known during planning.
- `multi_runner_config[].orchestration_provider.scale_set`: Selects stateful GitHub Actions runner scale-set orchestration. Runner-config applies fixed ephemeral JIT lifecycle settings, while multi-runner aggregates all selected runner configs into the shared controller topology.
- `multi_runner_config[].orchestration_provider.scale_set.github.config_url`: HTTPS GitHub organization, repository, or enterprise scope URL owned by this scale set.
- `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm`: Existing Parameter Store reference containing the GitHub App installation ID for this runner config's GitHub scope. The manifest contains only the parameter name, never the credential value.
- `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm.name`: Absolute Parameter Store name containing the installation ID.
- `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm.arn`: Exact same-account, same-region ARN of the installation-ID parameter.
- `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm.kms_key_arn`: Optional KMS key ARN needed to decrypt the installation-ID parameter. The default is null and is independent from global `ssm.kms_key_id`.
- `multi_runner_config[].orchestration_provider.scale_set.github.force_ghes`: Optional explicit GitHub Enterprise Server mode. Null enables it when global `github.enterprise_server.url` is set and otherwise uses GitHub.com behavior.
- `multi_runner_config[].orchestration_provider.scale_set.name`: Expected GitHub runner scale-set name used to verify controller ownership.
- `multi_runner_config[].orchestration_provider.scale_set.id`: Existing GitHub runner scale-set numeric ID. The controller does not create or discover scale sets in this foundation.
- `multi_runner_config[].orchestration_provider.scale_set.runner_group_id`: Optional expected GitHub runner-group ID. The default is null.
- `multi_runner_config[].orchestration_provider.scale_set.min_runners`: Minimum desired runner count reconciled for the scale set. The default is `0`.
- `multi_runner_config[].orchestration_provider.scale_set.max_runners`: Maximum desired runner count reconciled for the scale set. The default is `10`.
- `multi_runner_config[].orchestration_provider.scale_set.boot_time_in_minutes`: Expected compute boot time used to expire stale pending runner requests. The default is `10`; valid values are integers from `1` through `120`.
- `multi_runner_config[].orchestration_provider.scale_set.session_owner`: Optional stable owner used for the GitHub message session. Null derives one from the controller group and runner-config key.
- `multi_runner_config[].orchestration_provider.scale_set.work_folder`: Optional relative runner work folder. The default is null, which lets the compute provider use its default.
- `multi_runner_config[].orchestration_provider.webhook`: Selects the workflow-job webhook control plane, including its SQS build queue, scale-up, scheduled scale-down/pool, and optional job-retry resources.
- `multi_runner_config[].orchestration_provider.webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by webhook scale-down and pool controls. Null inherits `experimental.orchestration_provider.webhook.runner.boot_time_in_minutes`.
- `multi_runner_config[].orchestration_provider.webhook.runner.ephemeral`: Registers webhook-orchestrated runners in ephemeral mode. Null inherits `experimental.orchestration_provider.webhook.runner.ephemeral`.
- `multi_runner_config[].orchestration_provider.webhook.runner.jit_config_enabled`: Explicitly enables or disables just-in-time configuration. Null inherits the global webhook value; if both are null, behavior follows the resolved webhook `ephemeral` mode.
- `multi_runner_config[].orchestration_provider.webhook.runner.maximum_count`: Maximum number of runners managed by the webhook orchestration provider for this configuration. Null inherits `experimental.orchestration_provider.webhook.runner.maximum_count`.
- `multi_runner_config[].orchestration_provider.webhook.github.organization_runners`: Registers runners at organization scope when true; otherwise repository-scoped registration is used.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.labelMatchers`: Groups of labels used to match webhook jobs to this configuration.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.exactMatch`: Deprecated one-way match. When true, every workflow-job label must appear in a configured label group, but that group may contain additional labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.bidirectionalLabelMatch`: Requires an exact two-way set match between workflow-job labels and a configured label group, with no extra or missing labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.priority`: Ordering used when multiple configurations match the same job.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.enableDynamicLabels`: Enables runtime interpretation of supported dynamic AWS labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy`: Optional policy restricting values accepted from dynamic AWS labels.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.blocked_keys`: Dynamic-label keys rejected for this runner configuration. The default is `[]`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys`: Per-key allow, deny, and maximum-value restrictions. The default is `{}`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys..allowed`: Values explicitly allowed for the dynamic-label key. The default is `[]`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys..denied`: Values explicitly denied for the dynamic-label key. The default is `[]`.
- `multi_runner_config[].orchestration_provider.webhook.matcherConfig.awsDynamicLabelsPolicy.restricted_keys..max`: Optional maximum accepted value for the dynamic-label key. The default is null.
- `multi_runner_config[].orchestration_provider.webhook.queue.delay_webhook_event`: Delay in seconds applied to webhook job messages. Null inherits the global queue default.
- `multi_runner_config[].orchestration_provider.webhook.queue.job_queue_retention_in_seconds`: Build-queue message retention period in seconds. Null inherits the global queue default.
- `multi_runner_config[].orchestration_provider.webhook.queue.visibility_timeout_seconds`: Build-queue visibility timeout. Null inherits the global queue default; the resolved value must be at least six times the resolved `orchestration_provider.webhook.lambda.scale.up.timeout` so Lambda has enough time to retry throttled invocations.
- `multi_runner_config[].orchestration_provider.webhook.queue.redrive_build_queue.enabled`: Creates and attaches a dead-letter queue for the build queue. A null wrapper or null leaf inherits the corresponding global value.
- `multi_runner_config[].orchestration_provider.webhook.queue.redrive_build_queue.maxReceiveCount`: Number of receives before a build message moves to the dead-letter queue. A null wrapper or null leaf inherits the corresponding global value, and the resolved value must be greater than zero when redrive is enabled.
- `multi_runner_config[].orchestration_provider.webhook.queue.tags`: Tags for configuration-owned queue resources. These merge after global queue tags and entry-level `tags`; component tags override this map.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.memory_size`: Memory allocated to the scale-up Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for the scale-up Lambda. Use `-1` for unreserved concurrency.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.job_queued_check_enabled`: Enables the queued-job verification before scaling. Null inherits the global value; if both are null, behavior follows the resolved runner mode.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered to one scale-up Lambda invocation.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum batching window for build-queue records.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.up.tags`: Tags for scale-up resources. These override entry-level and shared Lambda, queue, and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.memory_size`: Memory allocated to the scale-down Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.schedule_expression`: EventBridge schedule expression that invokes scale-down.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.minimum_running_time_in_minutes`: Minimum runner age before scale-down may terminate it. Null inherits the global value; if both are null, the operating-system default is selected.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config`: Time-based desired idle-runner configurations.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate `cron`.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners to retain during the matching period.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed.
- `multi_runner_config[].orchestration_provider.webhook.lambda.scale.down.tags`: Tags for scale-down resources. These override entry-level and shared Lambda and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.memory_size`: Memory allocated to the pool Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. Use `-1` for unreserved concurrency.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config`: Scheduled target pool sizes. An empty list disables the pool component.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.config[].size`: Desired number of runners for the schedule.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used when creating pooled runners.
- `multi_runner_config[].orchestration_provider.webhook.lambda.pool.tags`: Tags for pool resources. These override entry-level and shared Lambda and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.enabled`: Creates the retry queue, Lambda function, event-source mapping, and related IAM resources.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.delay_in_seconds`: Initial delay before a queued-job retry check.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.delay_backoff`: Multiplier applied to the delay after each unsuccessful check.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.max_attempts`: Maximum retry-check attempts before the message is no longer republished.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.tags`: Tags for job-retry resources. These override entry-level and shared Lambda, queue, and log-group tags within their resource scopes.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.lambda.memory_size`: Memory allocated to the job-retry Lambda in MB.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.lambda.reserved_concurrent_executions`: Reserved concurrency for the job-retry Lambda. Use `-1` for unreserved concurrency.
- `multi_runner_config[].orchestration_provider.webhook.job_retry.lambda.timeout`: Job-retry Lambda timeout in seconds and visibility timeout for its retry queue.
- `multi_runner_config[].ssm.paths.root`: Base Parameter Store root for this runner configuration. The configuration key is always appended to preserve configuration isolation. The omitted global root derives `/github-action-runners/`.
- `multi_runner_config[].ssm.paths.tokens`: Path segment below the runner-configuration root used for runner registration tokens and just-in-time configuration.
- `multi_runner_config[].ssm.paths.config`: Path segment below the runner-configuration root used for persistent runner configuration.
- `multi_runner_config[].ssm.tags`: Shared tags for runner-configuration-owned SSM resources. These override entry-level `tags`. For EC2 scale-set orchestration, the effective Parameter Store tag map contains at most 45 runtime-compatible keys and values.
- `multi_runner_config[].ssm.parameters.tags`: Tags for Terraform-managed and runtime-created runner configuration parameters. These override `ssm.tags` and participate in the EC2 scale-set runtime tag limit.
- `multi_runner_config[].ssm.housekeeper.schedule_expression`: EventBridge schedule expression that invokes the runner-configuration SSM housekeeper.
- `multi_runner_config[].ssm.housekeeper.state`: EventBridge rule state for the runner-configuration SSM housekeeper. Valid values are `DISABLED`, `ENABLED`, and `ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS`.
- `multi_runner_config[].ssm.housekeeper.tags`: Tags for SSM housekeeper resources. These override entry-level, shared Lambda, shared log, and `ssm.tags` values.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact`: Per-configuration SSM-housekeeper artifact selection. A selected `zip` or `s3` source overrides the global `ssm.housekeeper.lambda.artifact`; when neither level selects a source, runner-config uses its packaged runner control-plane archive.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.zip`: Optional local path to the SSM-housekeeper Lambda archive.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.s3`: Optional key and object version in the shared `lambda.artifact.s3.bucket`. Wrapper presence selects S3 for this SSM housekeeper, must be known during planning, and requires a non-null shared bucket and key.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.s3.key`: Object key of this runner configuration's SSM-housekeeper Lambda archive.
- `multi_runner_config[].ssm.housekeeper.lambda.artifact.s3.object_version`: Optional object version of this runner configuration's SSM-housekeeper Lambda archive.
- `multi_runner_config[].ssm.housekeeper.lambda.memory_size`: Memory allocated to the SSM housekeeper Lambda in MB.
- `multi_runner_config[].ssm.housekeeper.lambda.timeout`: SSM housekeeper Lambda timeout in seconds.
- `multi_runner_config[].ssm.housekeeper.config.tokenPath`: Optional cleanup path. A global value is shared by every runner configuration, so omit it to derive each configuration's isolated token path.
- `multi_runner_config[].ssm.housekeeper.config.minimumDaysOld`: Minimum parameter age in days before deletion is allowed.
- `multi_runner_config[].ssm.housekeeper.config.dryRun`: Reports eligible parameters without deleting them when true.
- `multi_runner_config[].observability.logs.level`: Application log level for runner-configuration control-plane functions.
- `multi_runner_config[].observability.logs.retention_in_days`: CloudWatch Logs retention period for runner-configuration resources.
- `multi_runner_config[].observability.logs.kms_key_id`: Optional KMS key ID or ARN used to encrypt runner-configuration CloudWatch log groups.
- `multi_runner_config[].observability.logs.class`: CloudWatch log-group class for runner-configuration resources.
- `multi_runner_config[].observability.logs.tags`: Shared tags for runner-configuration CloudWatch log groups. Component tags override this map.
- `multi_runner_config[].observability.tracing.mode`: Optional Lambda tracing mode. Its nullness must be known during planning because it controls X-Ray IAM and tracing blocks.
- `multi_runner_config[].observability.tracing.capture_http_requests`: Enables HTTP request capture in the tracing helper.
- `multi_runner_config[].observability.tracing.capture_error`: Enables error capture in the tracing helper.
- `multi_runner_config[].observability.metrics.enable`: Enables module-emitted metrics.
- `multi_runner_config[].observability.metrics.namespace`: CloudWatch namespace used for emitted metrics.
- `multi_runner_config[].observability.metrics.metric.enable_github_app_rate_limit`: Emits GitHub App rate-limit metrics.
- `multi_runner_config[].observability.metrics.metric.enable_job_retry`: Emits job-retry metrics.
- `multi_runner_config[].compute_provider`: Typed compute-provider namespaces. Exactly one nested provider block must be non-null, and that populated block selects the provider. Its presence must be known during planning; values inside it may remain unknown until apply.
- `multi_runner_config[].compute_provider.aws`: AWS compute-provider namespace. The namespace itself does not select a provider.
- `multi_runner_config[].compute_provider.aws.ec2`: AWS EC2-specific configuration. A non-null block selects AWS EC2 for this runner configuration.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when enabled.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `multi_runner_config[].compute_provider.aws.ec2.metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `multi_runner_config[].compute_provider.aws.ec2.ami.filter`: EC2 AMI filters combined with the default AMI-name filter.
- `multi_runner_config[].compute_provider.aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `multi_runner_config[].compute_provider.aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. The wrapper's presence selects external ownership at plan time.
- `multi_runner_config[].compute_provider.aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply. Scale-set selections require an exact same-account, same-region ARN whose extracted absolute name matches the EC2 runtime grammar.
- `multi_runner_config[].compute_provider.aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence selects the KMS policy at plan time.
- `multi_runner_config[].compute_provider.aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings`: EBS mappings added to the runner launch template.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].delete_on_termination`: Deletes the volume when its runner instance terminates.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].encrypted`: Enables EBS encryption.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].iops`: Provisioned IOPS for supported volume types.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].throughput`: Provisioned throughput for supported volume types.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].volume_initialization_rate`: Fixed initialization rate in MiB/s for supported snapshot-backed volumes.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].volume_size`: Volume size in GiB.
- `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings[].volume_type`: EBS volume type.
- `multi_runner_config[].compute_provider.aws.ec2.create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `multi_runner_config[].compute_provider.aws.ec2.credit_specification`: CPU credit mode for burstable instance types.
- `multi_runner_config[].compute_provider.aws.ec2.ebs_optimized`: Requests EBS-optimized runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.cloudwatch_agent.enabled`: Installs and configures the CloudWatch agent through the default bootstrap flow.
- `multi_runner_config[].compute_provider.aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration.
- `multi_runner_config[].compute_provider.aws.ec2.binaries_syncer.enabled`: Enables use of the shared synchronized runner distribution from S3. Null inherits `experimental.compute_provider.aws.ec2.runner_binaries.enabled`.
- `multi_runner_config[].compute_provider.aws.ec2.detailed_monitoring_enabled`: Enables detailed EC2 monitoring for runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.ssm_enabled`: Attaches runner permissions and policies required for AWS Systems Manager access.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.enabled`: Enables launch-template user data.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.template`: Optional path to a custom user-data template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.content`: Optional complete user-data content used instead of rendering a template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.pre_install`: Script content inserted before runner installation in the default template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.post_install`: Script content inserted after runner installation in the default template.
- `multi_runner_config[].compute_provider.aws.ec2.user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets in logs.
- `multi_runner_config[].compute_provider.aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select capacity. Scale-set selections allow only `lowest-price` or `prioritized` with `on-demand`; Spot supports the complete EC2 provider strategy set.
- `multi_runner_config[].compute_provider.aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price.
- `multi_runner_config[].compute_provider.aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `multi_runner_config[].compute_provider.aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `multi_runner_config[].compute_provider.aws.ec2.instance_types`: EC2 instance types available to the scale-up and pool functions.
- `multi_runner_config[].compute_provider.aws.ec2.additional_security_group_ids`: Existing security groups attached to runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.managed_security_group_enabled`: Creates the module-managed runner security group when true.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules`: Runner security-group egress rules. Null inherits `experimental.compute_provider.aws.ec2.egress_rules`, whose default is the built-in IPv4/IPv6 allow-all rule. Flat `runner_egress_rules` is not inherited by v2.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].prefix_list_ids`: Prefix-list destinations for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].from_port`: Start of the runner-configuration egress rule port range.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].protocol`: Runner-configuration egress rule protocol; `-1` allows every protocol.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].security_groups`: Destination security-group IDs for the runner-configuration egress rule.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].to_port`: End of the runner-configuration egress rule port range.
- `multi_runner_config[].compute_provider.aws.ec2.egress_rules[].description`: Optional runner-configuration egress rule description.
- `multi_runner_config[].compute_provider.aws.ec2.instance_profile_path`: IAM path for the module-managed EC2 instance profile.
- `multi_runner_config[].compute_provider.aws.ec2.key_name`: Optional EC2 key-pair name for runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.associate_public_ipv4_address`: Associates a public IPv4 address with runner instances.
- `multi_runner_config[].compute_provider.aws.ec2.instance_profile.name`: Name of an externally managed instance profile. Setting it also requires `runner.iam.role`.
- `multi_runner_config[].compute_provider.aws.ec2.enable_on_demand_failover_for_errors`: EC2 error codes that trigger an on-demand fallback after a Spot launch failure.
- `multi_runner_config[].compute_provider.aws.ec2.scale_errors`: EC2 error codes treated as retryable scale-up failures.
- `multi_runner_config[].compute_provider.aws.ec2.subnet_ids`: Subnets from which scale-up may launch runners. A null configuration value inherits the experimental global value.
- `multi_runner_config[].compute_provider.aws.ec2.vpc_id`: VPC in which runner networking resources are created. A null configuration value inherits the experimental global value.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `multi_runner_config[].compute_provider.aws.ec2.cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `multi_runner_config[].compute_provider.aws.ec2.placement.affinity`: Host affinity setting.
- `multi_runner_config[].compute_provider.aws.ec2.placement.availability_zone`: Availability Zone in which the instance is placed.
- `multi_runner_config[].compute_provider.aws.ec2.placement.group_id`: Placement-group ID.
- `multi_runner_config[].compute_provider.aws.ec2.placement.group_name`: Placement-group name.
- `multi_runner_config[].compute_provider.aws.ec2.placement.host_id`: Dedicated Host ID.
- `multi_runner_config[].compute_provider.aws.ec2.placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `multi_runner_config[].compute_provider.aws.ec2.placement.spread_domain`: Spread-domain placement value.
- `multi_runner_config[].compute_provider.aws.ec2.placement.tenancy`: Instance tenancy.
- `multi_runner_config[].compute_provider.aws.ec2.placement.partition_number`: Placement-group partition number.
- `multi_runner_config[].compute_provider.aws.ec2.license_specifications[].license_configuration_arn`: ARN of a License Manager license configuration.
- `multi_runner_config[].compute_provider.aws.ec2.use_dedicated_host`: Enables the dedicated-host launch path required for macOS runners.
- `multi_runner_config[].compute_provider.aws.ec2.log_files`: Optional log files collected by the CloudWatch agent.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_group_name`: CloudWatch log-group name before optional prefixing.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path when true.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].file_path`: File or glob read by the CloudWatch agent.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_stream_name`: CloudWatch log-stream name template.
- `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_class`: CloudWatch log-group class for the collected file.
- `multi_runner_config[].compute_provider.aws.ec2.tags`: Tags for runtime EC2 instances, volumes, network interfaces, and eligible Spot requests. These override entry-level tags and the generated runner `Name`; provider-required bootstrap tags take final precedence. Scale-set selections reject caller values for scale-set ownership and lifecycle keys. | 
object({
    tags = optional(map(string), {})

    roles = optional(object({
      path                 = optional(string, null)
      permissions_boundary = optional(string, null)
    }), {})

    runner = optional(object({
      os                     = optional(string, null)
      architecture           = optional(string, null)
      disable_default_labels = optional(bool, false)
      extra_labels           = optional(list(string), [])
      group_name             = optional(string, "Default")
      name_prefix            = optional(string, "")
      run_as_root            = optional(bool, false)
      run_as                 = optional(string, "ec2-user")
      auto_update_disabled   = optional(bool, false)
      tags                   = optional(map(string), {})
      hooks = optional(object({
        job_started   = optional(string, "")
        job_completed = optional(string, "")
      }), {})
      iam = optional(object({
        role = optional(object({
          arn = string
        }), null)
        managed_policy_arns          = optional(map(string), {})
        additional_trust_policy_json = optional(string, null)
        path                         = optional(string, null)
        permissions_boundary         = optional(string, null)
      }), {})
    }), {})

    github = optional(object({
      app = optional(object({
        key_base64 = optional(string)
        key_base64_ssm = optional(object({
          arn  = string
          name = string
        }))
        id = optional(string)
        id_ssm = optional(object({
          arn  = string
          name = string
        }))
        webhook_secret = optional(string)
        webhook_secret_ssm = optional(object({
          arn  = string
          name = string
        }))
      }), null)
      additional_apps = optional(list(object({
        key_base64          = optional(string)
        key_base64_ssm      = optional(object({ arn = string, name = string }))
        id                  = optional(string)
        id_ssm              = optional(object({ arn = string, name = string }))
        installation_id     = optional(string)
        installation_id_ssm = optional(object({ arn = string, name = string }))
      })), [])
      enterprise_server = optional(object({
        url        = optional(string, null)
        ssl_verify = optional(bool, true)
      }), {})
      user_agent = optional(string, "github-aws-runners")
    }), {})

    lambda = optional(object({
      artifact = optional(object({
        s3 = optional(object({
          bucket = optional(string, null)
        }), {})
      }), {})
      runtime      = optional(string, "nodejs24.x")
      architecture = optional(string, "arm64")
      principals = optional(list(object({
        type        = string
        identifiers = list(string)
      })), [])
      subnet_ids         = optional(list(string), [])
      security_group_ids = optional(list(string), [])
      tags               = optional(map(string), {})
      role = optional(object({
        path                 = optional(string, null)
        permissions_boundary = optional(string, null)
      }), {})
    }), {})

    orchestration_provider = optional(object({
      webhook = optional(object({
        queue_selection_strategy = optional(string, "first")
        eventbridge = optional(object({
          enable        = optional(bool, true)
          accept_events = optional(list(string), [])
        }), {})
        matcher_config_parameter_store_tier = optional(string, "Standard")
        runner = optional(object({
          boot_time_in_minutes = optional(number, 5)
          ephemeral            = optional(bool, false)
          jit_config_enabled   = optional(bool, null)
          maximum_count        = optional(number, null)
        }), {})

        github = optional(object({
          repository_white_list = optional(list(string), [])
        }), {})

        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          scale = optional(object({
            up = optional(object({
              memory_size                    = optional(number, 512)
              timeout                        = optional(number, 30)
              reserved_concurrent_executions = optional(number, 1)
              job_queued_check_enabled       = optional(bool, null)
              event_source_mapping = optional(object({
                batch_size                         = optional(number, 10)
                maximum_batching_window_in_seconds = optional(number, 0)
              }), {})
              tags = optional(map(string), {})
            }), {})
            down = optional(object({
              memory_size                     = optional(number, 512)
              timeout                         = optional(number, 60)
              schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
              minimum_running_time_in_minutes = optional(number, null)
              idle_config = optional(list(object({
                cron             = string
                timeZone         = string
                idleCount        = number
                evictionStrategy = optional(string, "oldest_first")
              })), [])
              tags = optional(map(string), {})
            }), {})
          }), {})
          webhook = optional(object({
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            api_gateway_access_log_settings = optional(object({
              destination_arn = string
              format          = string
            }), null)
            memory_size = optional(number, 256)
            timeout     = optional(number, 10)
            tags        = optional(map(string), {})
          }), {})
          pool = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 60)
            reserved_concurrent_executions = optional(number, 1)
            config = optional(list(object({
              schedule_expression          = string
              schedule_expression_timezone = optional(string)
              size                         = number
            })), [])
            include_busy_runners = optional(bool, false)
            runner_owner         = optional(string, null)
            tags                 = optional(map(string), {})
          }), {})
        }), {})

        queue = optional(object({
          delay_webhook_event            = optional(number, 30)
          job_queue_retention_in_seconds = optional(number, 86400)
          visibility_timeout_seconds     = optional(number, 180)
          redrive_build_queue = optional(object({
            enabled         = optional(bool, false)
            maxReceiveCount = optional(number, null)
            }), {
            enabled         = false
            maxReceiveCount = null
          })
          tags = optional(map(string), {})
          encryption = optional(object({
            kms_data_key_reuse_period_seconds = number
            kms_master_key_id                 = string
            sqs_managed_sse_enabled           = bool
            }), {
            kms_data_key_reuse_period_seconds = null
            kms_master_key_id                 = null
            sqs_managed_sse_enabled           = true
          })
        }), {})
      }), {})
      scale_set = optional(object({
        grouping = optional(object({
          strategy = optional(string, "compute_provider")
          custom = optional(object({
            groups = map(object({
              runner_configs = set(string)
            }))
          }), null)
        }), {})
        container = optional(object({
          image                             = optional(string, null)
          user                              = optional(string, "10001:10001")
          health_port                       = optional(number, 8080)
          health_path                       = optional(string, "/healthz")
          health_check_command              = optional(list(string), null)
          health_check_interval             = optional(number, 30)
          health_check_timeout              = optional(number, 5)
          health_check_retries              = optional(number, 3)
          health_check_start_period         = optional(number, 30)
          health_stale_after_seconds        = optional(number, 180)
          shutdown_timeout_seconds          = optional(number, 110)
          session_close_timeout_seconds     = optional(number, 10)
          reconnect_initial_backoff_seconds = optional(number, 1)
          reconnect_max_backoff_seconds     = optional(number, 30)
          stop_timeout_seconds              = optional(number, 120)
          ecr_repository = optional(object({
            arn = string
          }), null)
        }), {})
        config_store = optional(object({
          path_prefix = optional(string, null)
          tier        = optional(string, "Standard")
          tags        = optional(map(string), {})
        }), {})
        ecs = optional(object({
          cluster = optional(object({
            mode               = optional(string, "managed")
            arn                = optional(string, null)
            name               = optional(string, null)
            container_insights = optional(bool, true)
          }), {})
          task = optional(object({
            cpu              = optional(number, 512)
            memory           = optional(number, 1024)
            cpu_architecture = optional(string, "X86_64")
            ephemeral_storage = optional(object({
              size_in_gib = number
            }), null)
          }), {})
          service = optional(object({
            platform_version = optional(string, "LATEST")
          }), {})
          iam = optional(object({
            path                 = optional(string, "/")
            permissions_boundary = optional(string, null)
          }), {})
        }), {})
        network = optional(object({
          vpc_id     = optional(string, null)
          subnet_ids = optional(set(string), null)
          https_egress = optional(object({
            ipv4_cidrs = optional(set(string), ["0.0.0.0/0"])
            ipv6_cidrs = optional(set(string), [])
          }), {})
        }), {})
        logging = optional(object({
          retention_in_days = optional(number, 30)
          kms_key_arn       = optional(string, null)
          log_group_class   = optional(string, "STANDARD")
          tags              = optional(map(string), {})
        }), {})
        tags = optional(map(string), {})
      }), {})
    }), {})

    ssm = optional(object({
      paths = optional(object({
        root    = optional(string, null)
        app     = optional(string, "app")
        webhook = optional(string, "webhook")
        tokens  = optional(string, "runners/tokens")
        config  = optional(string, "runners/config")
      }), {})
      kms_key_id = optional(string, null)
      tags       = optional(map(string), {})
      parameters = optional(object({
        tags = optional(map(string), {})
      }), {})
      housekeeper = optional(object({
        schedule_expression = optional(string, "rate(1 day)")
        state               = optional(string, "ENABLED")
        tags                = optional(map(string), {})
        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          memory_size = optional(number, 512)
          timeout     = optional(number, 60)
        }), {})
        config = optional(object({
          tokenPath      = optional(string, null)
          minimumDaysOld = optional(number, 1)
          dryRun         = optional(bool, false)
        }), {})
      }), {})
    }), {})

    observability = optional(object({
      logs = optional(object({
        level             = optional(string, "info")
        retention_in_days = optional(number, 180)
        kms_key_id        = optional(string, null)
        class             = optional(string, "STANDARD")
        tags              = optional(map(string), {})
      }), {})
      tracing = optional(object({
        mode                  = optional(string, null)
        capture_http_requests = optional(bool, false)
        capture_error         = optional(bool, false)
      }), {})
      metrics = optional(object({
        enable    = optional(bool, false)
        namespace = optional(string, "GitHub Runners")
        metric = optional(object({
          enable_github_app_rate_limit    = optional(bool, true)
          enable_job_retry                = optional(bool, true)
          enable_spot_termination         = optional(bool, true)
          enable_spot_termination_warning = optional(bool, true)
        }), {})
      }), {})
    }), {})

    compute_provider = optional(object({
      selections = optional(map(object({
        namespace = string
        type      = string
      })), null)
      aws = optional(object({
        ec2 = optional(object({
          vpc_id                         = optional(string, null)
          subnet_ids                     = optional(list(string), null)
          managed_security_group_enabled = optional(bool, true)
          egress_rules = optional(list(object({
            cidr_blocks      = list(string)
            ipv6_cidr_blocks = list(string)
            prefix_list_ids  = list(string)
            from_port        = number
            protocol         = string
            security_groups  = list(string)
            self             = bool
            to_port          = number
            description      = string
            })), [{
            cidr_blocks      = ["0.0.0.0/0"]
            ipv6_cidr_blocks = ["::/0"]
            prefix_list_ids  = null
            from_port        = 0
            protocol         = "-1"
            security_groups  = null
            self             = null
            to_port          = 0
            description      = null
          }])
          additional_security_group_ids = optional(list(string), [])
          cloudwatch_agent = optional(object({
            config = optional(string, null)
          }), {})
          instance_profile_path         = optional(string, null)
          key_name                      = optional(string, null)
          associate_public_ipv4_address = optional(bool, false)
          tags                          = optional(map(string), {})
          ami = optional(object({
            housekeeper = optional(object({
              enabled = optional(bool, false)
              cleanup_config = optional(object({
                maxItems       = optional(number)
                minimumDaysOld = optional(number)
                amiFilters = optional(list(object({
                  Name   = string
                  Values = list(string)
                })))
                launchTemplateNames = optional(list(string))
                ssmParameterNames   = optional(list(string))
                dryRun              = optional(bool)
              }), {})
              artifact = optional(object({
                zip = optional(string, null)
                s3 = optional(object({
                  key            = string
                  object_version = optional(string, null)
                }), null)
              }), {})
              lambda = optional(object({
                memory_size = optional(number, 256)
                timeout     = optional(number, 300)
              }), {})
              schedule = optional(object({
                expression = optional(string, "cron(11 7 * * ? *)")
              }), {})
            }), {})
          }), {})
          instance_termination_watcher = optional(object({
            enabled = optional(bool, false)
            features = optional(object({
              enable_spot_termination_handler              = optional(bool, true)
              enable_spot_termination_notification_watcher = optional(bool, true)
            }), {})
            enable_runner_deregistration = optional(bool, true)
            environment_variables        = optional(map(string), {})
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            lambda = optional(object({
              memory_size = optional(number, null)
              timeout     = optional(number, null)
            }), {})
          }), {})
          runner_binaries = optional(object({
            enabled = optional(bool, true)
            targets = optional(map(object({
              os           = string
              architecture = string
            })), null)
            s3 = optional(object({
              encryption = optional(object({
                enabled            = optional(bool, true)
                bucket_key_enabled = optional(bool, null)
                sse_algorithm      = optional(string, "AES256")
                kms_master_key_id  = optional(string, null)
              }), {})
              tags       = optional(map(string), {})
              versioning = optional(string, "Disabled")
              logging = optional(object({
                bucket = optional(string, null)
                prefix = optional(string, null)
              }), {})
            }), {})
            syncer = optional(object({
              artifact = optional(object({
                zip = optional(string, null)
                s3 = optional(object({
                  key            = string
                  object_version = optional(string, null)
                }), null)
              }), {})
              lambda = optional(object({
                memory_size = optional(number, 256)
                timeout     = optional(number, 300)
              }), {})
              schedule = optional(object({
                expression = optional(string, "cron(27 * * * ? *)")
                state      = optional(string, "ENABLED")
              }), {})
            }), {})
          }), {})
        }), {})
      }), {})
    }), {})

    multi_runner_config = optional(map(object({
      tags = optional(map(string), {})

      runner = optional(object({
        os                     = optional(string, null)
        architecture           = optional(string, null)
        disable_default_labels = optional(bool, null)
        extra_labels           = optional(list(string), null)
        group_name             = optional(string, null)
        name_prefix            = optional(string, null)
        run_as_root            = optional(bool, null)
        run_as                 = optional(string, null)
        auto_update_disabled   = optional(bool, null)
        tags                   = optional(map(string), {})
        hooks = optional(object({
          job_started   = optional(string, null)
          job_completed = optional(string, null)
        }), {})
        iam = optional(object({
          role = optional(object({
            arn = string
          }), null)
          managed_policy_arns          = optional(map(string), null)
          additional_trust_policy_json = optional(string, null)
          path                         = optional(string, null)
          permissions_boundary         = optional(string, null)
        }), {})
      }), {})

      lambda = optional(object({
        runtime            = optional(string, null)
        architecture       = optional(string, null)
        subnet_ids         = optional(list(string), null)
        security_group_ids = optional(list(string), null)
        tags               = optional(map(string), {})
        role = optional(object({
          path                 = optional(string, null)
          permissions_boundary = optional(string, null)
        }), {})
      }), {})

      orchestration_provider = object({
        webhook = optional(object({
          runner = optional(object({
            boot_time_in_minutes = optional(number, null)
            ephemeral            = optional(bool, null)
            jit_config_enabled   = optional(bool, null)
            maximum_count        = optional(number, null)
          }), {})

          github = optional(object({
            organization_runners = optional(bool, false)
          }), {})

          matcherConfig = object({
            labelMatchers           = list(list(string))
            exactMatch              = optional(bool, false)
            bidirectionalLabelMatch = optional(bool, false)
            priority                = optional(number, 999)
            enableDynamicLabels     = optional(bool, false)
            awsDynamicLabelsPolicy = optional(object({
              blocked_keys = optional(list(string), [])
              restricted_keys = optional(map(object({
                allowed = optional(list(string), [])
                denied  = optional(list(string), [])
                max     = optional(string, null)
              })), {})
            }), null)
          })

          queue = optional(object({
            delay_webhook_event            = optional(number, null)
            job_queue_retention_in_seconds = optional(number, null)
            visibility_timeout_seconds     = optional(number, null)
            redrive_build_queue = optional(object({
              enabled         = optional(bool, null)
              maxReceiveCount = optional(number, null)
            }), null)
            tags = optional(map(string), {})
          }), {})

          lambda = optional(object({
            scale = optional(object({
              up = optional(object({
                memory_size                    = optional(number, null)
                timeout                        = optional(number, null)
                reserved_concurrent_executions = optional(number, null)
                job_queued_check_enabled       = optional(bool, null)
                event_source_mapping = optional(object({
                  batch_size                         = optional(number, null)
                  maximum_batching_window_in_seconds = optional(number, null)
                }), {})
                tags = optional(map(string), {})
              }), {})
              down = optional(object({
                memory_size                     = optional(number, null)
                timeout                         = optional(number, null)
                schedule_expression             = optional(string, null)
                minimum_running_time_in_minutes = optional(number, null)
                idle_config = optional(list(object({
                  cron             = string
                  timeZone         = string
                  idleCount        = number
                  evictionStrategy = optional(string, "oldest_first")
                })), null)
                tags = optional(map(string), {})
              }), {})
            }), {})
            pool = optional(object({
              memory_size                    = optional(number, null)
              timeout                        = optional(number, null)
              reserved_concurrent_executions = optional(number, null)
              config = optional(list(object({
                schedule_expression          = string
                schedule_expression_timezone = optional(string)
                size                         = number
              })), null)
              include_busy_runners = optional(bool, null)
              runner_owner         = optional(string, null)
              tags                 = optional(map(string), {})
            }), {})
          }), {})

          job_retry = optional(object({
            enabled          = optional(bool, false)
            delay_in_seconds = optional(number, 300)
            delay_backoff    = optional(number, 2)
            max_attempts     = optional(number, 1)
            tags             = optional(map(string), {})
            lambda = optional(object({
              memory_size                    = optional(number, 256)
              reserved_concurrent_executions = optional(number, 1)
              timeout                        = optional(number, 30)
            }), {})
          }), {})

        }), null)

        scale_set = optional(object({
          github = object({
            config_url = string
            installation_id_ssm = object({
              name        = string
              arn         = string
              kms_key_arn = optional(string, null)
            })
            force_ghes = optional(bool, null)
          })
          name                 = string
          id                   = number
          runner_group_id      = optional(number, null)
          min_runners          = optional(number, 0)
          max_runners          = optional(number, 10)
          boot_time_in_minutes = optional(number, 10)
          session_owner        = optional(string, null)
          work_folder          = optional(string, null)
        }), null)

      })

      ssm = optional(object({
        paths = optional(object({
          root   = optional(string, null)
          tokens = optional(string, null)
          config = optional(string, null)
        }), {})
        tags = optional(map(string), {})
        parameters = optional(object({
          tags = optional(map(string), {})
        }), {})
        housekeeper = optional(object({
          schedule_expression = optional(string, null)
          state               = optional(string, null)
          tags                = optional(map(string), {})
          lambda = optional(object({
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            memory_size = optional(number, null)
            timeout     = optional(number, null)
          }), {})
          config = optional(object({
            tokenPath      = optional(string, null)
            minimumDaysOld = optional(number, null)
            dryRun         = optional(bool, null)
          }), {})
        }), {})
      }), {})

      observability = optional(object({
        logs = optional(object({
          level             = optional(string, null)
          retention_in_days = optional(number, null)
          kms_key_id        = optional(string, null)
          class             = optional(string, null)
          tags              = optional(map(string), {})
        }), {})
        tracing = optional(object({
          mode                  = optional(string, null)
          capture_http_requests = optional(bool, null)
          capture_error         = optional(bool, null)
        }), {})
        metrics = optional(object({
          enable    = optional(bool, null)
          namespace = optional(string, null)
          metric = optional(object({
            enable_github_app_rate_limit = optional(bool, null)
            enable_job_retry             = optional(bool, null)
          }), {})
        }), {})
      }), {})

      compute_provider = object({
        aws = optional(object({
          ec2 = optional(object({
            metadata_options = optional(object({
              instance_metadata_tags      = optional(string, "enabled")
              http_endpoint               = optional(string, "enabled")
              http_tokens                 = optional(string, "required")
              http_put_response_hop_limit = optional(number, 1)
            }), {})
            ami = optional(object({
              filter = optional(map(list(string)), { state = ["available"] })
              owners = optional(list(string), ["amazon"])
              id_ssm_parameter = optional(object({
                arn = string
              }), null)
              kms_key = optional(object({
                arn = string
              }), null)
            }), null)
            block_device_mappings = optional(list(object({
              delete_on_termination      = optional(bool, true)
              device_name                = optional(string, "/dev/xvda")
              encrypted                  = optional(bool, true)
              iops                       = optional(number)
              kms_key_id                 = optional(string)
              snapshot_id                = optional(string)
              throughput                 = optional(number)
              volume_initialization_rate = optional(number)
              volume_size                = number
              volume_type                = optional(string, "gp3")
              })), [{
              volume_size = 30
            }])
            create_service_linked_role_spot = optional(bool, false)
            credit_specification            = optional(string, null)
            ebs_optimized                   = optional(bool, false)
            cloudwatch_agent = optional(object({
              enabled = optional(bool, true)
              config  = optional(string, null)
            }), {})
            binaries_syncer = optional(object({
              enabled = optional(bool, null)
            }), {})
            detailed_monitoring_enabled = optional(bool, false)
            ssm_enabled                 = optional(bool, false)
            user_data = optional(object({
              enabled               = optional(bool, true)
              template              = optional(string, null)
              content               = optional(string, null)
              pre_install           = optional(string, "")
              post_install          = optional(string, "")
              debug_logging_enabled = optional(bool, false)
            }), {})
            instance_allocation_strategy   = optional(string, "lowest-price")
            instance_max_spot_price        = optional(string, null)
            instance_target_capacity_type  = optional(string, "spot")
            instance_type_priorities       = optional(map(number), null)
            instance_types                 = list(string)
            additional_security_group_ids  = optional(list(string), null)
            managed_security_group_enabled = optional(bool, null)
            egress_rules = optional(list(object({
              cidr_blocks      = list(string)
              ipv6_cidr_blocks = list(string)
              prefix_list_ids  = list(string)
              from_port        = number
              protocol         = string
              security_groups  = list(string)
              self             = bool
              to_port          = number
              description      = string
            })), null)
            instance_profile_path         = optional(string, null)
            key_name                      = optional(string, null)
            associate_public_ipv4_address = optional(bool, null)
            instance_profile = optional(object({
              name = string
            }), null)
            enable_on_demand_failover_for_errors = optional(list(string), [])
            scale_errors = optional(list(string), [
              "UnfulfillableCapacity",
              "MaxSpotInstanceCountExceeded",
              "TargetCapacityLimitExceededException",
              "RequestLimitExceeded",
              "ResourceLimitExceeded",
              "MaxSpotFleetRequestCountExceeded",
              "InsufficientInstanceCapacity",
              "InsufficientCapacityOnHost",
            ])
            subnet_ids = optional(list(string), null)
            vpc_id     = optional(string, null)
            cpu_options = optional(object({
              core_count            = optional(number)
              threads_per_core      = optional(number)
              amd_sev_snp           = optional(string)
              nested_virtualization = optional(string)
            }), null)
            placement = optional(object({
              affinity                = optional(string)
              availability_zone       = optional(string)
              group_id                = optional(string)
              group_name              = optional(string)
              host_id                 = optional(string)
              host_resource_group_arn = optional(string)
              spread_domain           = optional(string)
              tenancy                 = optional(string)
              partition_number        = optional(number)
            }), null)
            license_specifications = optional(list(object({
              license_configuration_arn = string
            })), [])
            use_dedicated_host = optional(bool, false)
            log_files = optional(list(object({
              log_group_name   = string
              prefix_log_group = bool
              file_path        = string
              log_stream_name  = string
              log_class        = optional(string, "STANDARD")
            })), null)
            tags = optional(map(string), {})
          }), null)
        }), {})
      })

    })), {})
  })
| `{}` | no | | [ghes\_ssl\_verify](#input\_ghes\_ssl\_verify) | GitHub Enterprise SSL verification. Set to 'false' when custom certificate (chains) is used for GitHub Enterprise Server (insecure). | `bool` | `true` | no | | [ghes\_url](#input\_ghes\_url) | GitHub Enterprise Server URL. Example: https://github.internal.co - DO NOT SET IF USING PUBLIC GITHUB. .However if you are using GitHub Enterprise Cloud with data-residency (ghe.com), set the endpoint here. Example - https://companyname.ghe.com\| | `string` | `null` | no | | [github\_app](#input\_github\_app) | GitHub app parameters, see your github app.
You can optionally create the SSM parameters yourself and provide the ARN and name here, through the `*_ssm` attributes.
If you chose to provide the configuration values directly here,
please ensure the key is the base64-encoded `.pem` file (the output of `base64 app.private-key.pem`, not the content of `private-key.pem`).
Note: the provided SSM parameters arn and name have a precedence over the actual value (i.e `key_base64_ssm` has a precedence over `key_base64` etc). | 
object({
    key_base64 = optional(string)
    key_base64_ssm = optional(object({
      arn  = string
      name = string
    }))
    id = optional(string)
    id_ssm = optional(object({
      arn  = string
      name = string
    }))
    webhook_secret = optional(string)
    webhook_secret_ssm = optional(object({
      arn  = string
      name = string
    }))
  })
| `null` | no | @@ -290,6 +291,7 @@ module "multi-runner" { | [instance\_termination\_watcher](#output\_instance\_termination\_watcher) | n/a | | [runners\_map](#output\_runners\_map) | Stable v1 runner resources keyed by runner configuration. Entries retain the historical flat output shape. | | [runners\_map\_v2](#output\_runners\_map\_v2) | Experimental v2 runner resources keyed by runner configuration. Compute resources are grouped under `provider..`, currently `provider.aws.ec2`. The `orchestration_provider` object is canonical; `scale_up`, `scale_down`, and `pool` remain compatibility aliases. | +| [scale\_set](#output\_scale\_set) | Shared scale-set orchestration resources. Null when no experimental runner config selects scale\_set; controller\_groups are keyed by the resolved cross-runner grouping. | | [ssm\_parameters](#output\_ssm\_parameters) | n/a | | [webhook](#output\_webhook) | n/a | diff --git a/modules/multi-runner/config.experimental.translation.tf b/modules/multi-runner/config.experimental.translation.tf index 8423f7aefd..e886890b20 100644 --- a/modules/multi-runner/config.experimental.translation.tf +++ b/modules/multi-runner/config.experimental.translation.tf @@ -146,6 +146,71 @@ locals { encryption = var.queue_encryption } } + scale_set = { + grouping = { + strategy = "compute_provider" + custom = null + } + container = { + image = null + user = "10001:10001" + health_port = 8080 + health_path = "/healthz" + health_check_command = null + health_check_interval = 30 + health_check_timeout = 5 + health_check_retries = 3 + health_check_start_period = 30 + health_stale_after_seconds = 180 + shutdown_timeout_seconds = 110 + session_close_timeout_seconds = 10 + reconnect_initial_backoff_seconds = 1 + reconnect_max_backoff_seconds = 30 + stop_timeout_seconds = 120 + ecr_repository = null + } + config_store = { + path_prefix = null + tier = "Standard" + tags = {} + } + ecs = { + cluster = { + mode = "managed" + arn = null + name = null + container_insights = true + } + task = { + cpu = 512 + memory = 1024 + cpu_architecture = "X86_64" + ephemeral_storage = null + } + service = { + platform_version = "LATEST" + } + iam = { + path = "/" + permissions_boundary = null + } + } + network = { + vpc_id = null + subnet_ids = null + https_egress = { + ipv4_cidrs = ["0.0.0.0/0"] + ipv6_cidrs = [] + } + } + logging = { + retention_in_days = 30 + kms_key_arn = null + log_group_class = "STANDARD" + tags = {} + } + tags = {} + } } ssm = { @@ -411,6 +476,7 @@ locals { } } } + scale_set = null } ssm = { @@ -669,6 +735,7 @@ locals { tags = merge(local.raw_translated_experimental.orchestration_provider.webhook.queue.tags, v.orchestration_provider.webhook.queue.tags) }) }) + scale_set = v.orchestration_provider.scale_set } ssm = merge(v.ssm, { @@ -807,6 +874,7 @@ locals { artifact = local.translated_experimental_base.orchestration_provider.webhook.lambda.artifact }) }) + scale_set = v.orchestration_provider.scale_set } ssm = merge(v.ssm, { diff --git a/modules/multi-runner/orchestration-provider.scale-set.tf b/modules/multi-runner/orchestration-provider.scale-set.tf new file mode 100644 index 0000000000..479bd0d0b4 --- /dev/null +++ b/modules/multi-runner/orchestration-provider.scale-set.tf @@ -0,0 +1,74 @@ +locals { + scale_set_runner_config = { + for runner_name, runner_config in local.translated_experimental.multi_runner_config : + runner_name => runner_config + if runner_config.orchestration_provider.scale_set != null + } + + scale_set_runner_configs = { + for runner_name, runner_config in local.scale_set_runner_config : runner_name => { + github = { + config_url = runner_config.orchestration_provider.scale_set.github.config_url + app = { + app_id = { + name = local.primary_app_id.name + arn = local.primary_app_id.arn + kms_key_arn = local.translated_experimental.ssm.kms_key_id + } + private_key = { + name = local.primary_app_key_base64.name + arn = local.primary_app_key_base64.arn + kms_key_arn = local.translated_experimental.ssm.kms_key_id + } + installation_id = { + name = runner_config.orchestration_provider.scale_set.github.installation_id_ssm.name + arn = runner_config.orchestration_provider.scale_set.github.installation_id_ssm.arn + kms_key_arn = runner_config.orchestration_provider.scale_set.github.installation_id_ssm.kms_key_arn + } + } + force_ghes = try(coalesce( + runner_config.orchestration_provider.scale_set.github.force_ghes, + local.translated_experimental.github.enterprise_server.url != null, + ), false) + ssl_verify = local.translated_experimental.github.enterprise_server.ssl_verify + user_agent = local.translated_experimental.github.user_agent + } + scale_set = { + name = runner_config.orchestration_provider.scale_set.name + id = runner_config.orchestration_provider.scale_set.id + runner_group_id = runner_config.orchestration_provider.scale_set.runner_group_id + min_runners = runner_config.orchestration_provider.scale_set.min_runners + max_runners = runner_config.orchestration_provider.scale_set.max_runners + boot_time_in_minutes = runner_config.orchestration_provider.scale_set.boot_time_in_minutes + session_owner = runner_config.orchestration_provider.scale_set.session_owner + } + work_folder = runner_config.orchestration_provider.scale_set.work_folder + } + } + + scale_set_compute_provider_contracts = { + for runner_name in keys(local.scale_set_runner_configs) : + runner_name => module.runner_configs[runner_name].compute_provider_contract + } +} + +module "orchestration_scale_set" { + source = "../orchestration-providers/scale-set" + count = length(local.scale_set_runner_configs) > 0 ? 1 : 0 + + prefix = var.prefix + runner_configs = local.scale_set_runner_configs + compute_provider_contracts = local.scale_set_compute_provider_contracts + + grouping = local.translated_experimental.orchestration_provider.scale_set.grouping + container = local.translated_experimental.orchestration_provider.scale_set.container + config_store = local.translated_experimental.orchestration_provider.scale_set.config_store + ecs = local.translated_experimental.orchestration_provider.scale_set.ecs + network = local.translated_experimental.orchestration_provider.scale_set.network + logging = local.translated_experimental.orchestration_provider.scale_set.logging + tags = merge( + local.translated_experimental.tags, + local.translated_experimental.orchestration_provider.scale_set.tags, + { "ghr:environment" = var.prefix }, + ) +} diff --git a/modules/multi-runner/outputs.tf b/modules/multi-runner/outputs.tf index 15aeaf172f..d711b0085b 100644 --- a/modules/multi-runner/outputs.tf +++ b/modules/multi-runner/outputs.tf @@ -35,6 +35,16 @@ output "runners_map_v2" { } } +output "scale_set" { + description = "Shared scale-set orchestration resources. Null when no experimental runner config selects scale_set; controller_groups are keyed by the resolved cross-runner grouping." + value = length(module.orchestration_scale_set) == 0 ? null : { + cluster = one(module.orchestration_scale_set[*].cluster) + controller_groups = one(module.orchestration_scale_set[*].controller_groups) + reconciler_config_parameters = one(module.orchestration_scale_set[*].reconciler_config_parameters) + resolved_container_image = one(module.orchestration_scale_set[*].resolved_container_image) + } +} + output "binaries_syncer_map" { value = { for runner_binary_key, runner_binary in module.runner_binaries : runner_binary_key => { lambda = runner_binary.lambda diff --git a/modules/multi-runner/runners.experimental.tf b/modules/multi-runner/runners.experimental.tf index b889c6a467..6df5764507 100644 --- a/modules/multi-runner/runners.experimental.tf +++ b/modules/multi-runner/runners.experimental.tf @@ -31,6 +31,7 @@ module "runner_configs" { lambda = each.value.orchestration_provider.webhook.lambda job_retry = each.value.orchestration_provider.webhook.job_retry } + scale_set = each.value.orchestration_provider.scale_set == null ? null : {} } ssm = each.value.ssm observability = each.value.observability diff --git a/modules/multi-runner/tests/provider-routing-scale-set.tftest.hcl b/modules/multi-runner/tests/provider-routing-scale-set.tftest.hcl new file mode 100644 index 0000000000..0e8a7a5c8d --- /dev/null +++ b/modules/multi-runner/tests/provider-routing-scale-set.tftest.hcl @@ -0,0 +1,624 @@ +mock_provider "aws" { + mock_data "aws_caller_identity" { + defaults = { + account_id = "123456789012" + } + } + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_region" { + defaults = { + region = "eu-west-1" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/scale-set-integration" + } + } + + mock_resource "aws_ecs_cluster" { + defaults = { + arn = "arn:aws:ecs:eu-west-1:123456789012:cluster/scale-set-integration" + } + } +} + +mock_provider "random" {} +mock_provider "null" {} + +variables { + aws_region = "eu-west-1" + vpc_id = "vpc-flat-unused" + subnet_ids = ["subnet-flat-unused"] + + github_app = { + id = "flat-unused" + key_base64 = "dGVzdA==" + webhook_secret = "flat-unused" + } + + lambda_s3_bucket = "flat-unused" + webhook_lambda_s3_key = "flat-unused.zip" + runners_lambda_zip = "README.md" + runners_lambda_s3_key = "flat-unused.zip" + syncer_lambda_s3_key = "flat-unused.zip" +} + +run "rejects_multiple_orchestration_selections" { + command = plan + + plan_options { + target = [terraform_data.validate_experimental] + } + + variables { + experimental = { + github = { + app = { + id = "123456" + key_base64 = "dGVzdA==" + webhook_secret = "test-secret" + } + } + runner = { + os = "linux" + architecture = "x64" + } + orchestration_provider = { + scale_set = { + network = { + vpc_id = "vpc-controller" + subnet_ids = ["subnet-controller"] + } + } + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-runners" + subnet_ids = ["subnet-runners"] + runner_binaries = { + enabled = false + } + } + } + } + multi_runner_config = { + invalid = { + orchestration_provider = { + webhook = { + runner = { + maximum_count = 2 + } + matcherConfig = { + labelMatchers = [["linux"]] + } + } + scale_set = { + github = { + config_url = "https://github.com/example" + installation_id_ssm = { + name = "/scale-set/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set/installation-id" + } + } + name = "invalid" + id = 301 + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + } + + expect_failures = [terraform_data.validate_experimental] +} + +run "rejects_scale_set_without_controller_network" { + command = plan + + plan_options { + target = [terraform_data.validate_experimental] + } + + variables { + experimental = { + github = { + app = { + id = "123456" + key_base64 = "dGVzdA==" + webhook_secret = "test-secret" + } + } + runner = { + os = "linux" + architecture = "x64" + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-runners" + subnet_ids = ["subnet-runners"] + runner_binaries = { + enabled = false + } + } + } + } + multi_runner_config = { + scale = { + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example" + installation_id_ssm = { + name = "/scale-set/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set/installation-id" + } + } + name = "scale" + id = 302 + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + } + + expect_failures = [terraform_data.validate_experimental] +} + +run "rejects_scale_set_with_mutating_termination_watcher" { + command = plan + + plan_options { + target = [terraform_data.validate_experimental] + } + + variables { + experimental = { + github = { + app = { + id = "123456" + key_base64 = "dGVzdA==" + webhook_secret = "test-secret" + } + } + runner = { + os = "linux" + architecture = "x64" + } + orchestration_provider = { + scale_set = { + network = { + vpc_id = "vpc-controller" + subnet_ids = ["subnet-controller"] + } + } + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-runners" + subnet_ids = ["subnet-runners"] + instance_termination_watcher = { + enabled = true + enable_runner_deregistration = true + } + runner_binaries = { + enabled = false + } + } + } + } + multi_runner_config = { + scale = { + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example" + installation_id_ssm = { + name = "/scale-set/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set/installation-id" + } + } + name = "scale" + id = 303 + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + } + + expect_failures = [terraform_data.validate_experimental] +} + +run "webhook_and_scale_set_coexist_with_one_grouped_controller" { + command = plan + + variables { + experimental = { + runner = { + os = "linux" + architecture = "x64" + } + github = { + app = { + id = "123456" + key_base64 = "dGVzdA==" + webhook_secret = "test-secret" + } + enterprise_server = { + ssl_verify = false + } + } + orchestration_provider = { + webhook = { + lambda = { + artifact = { + zip = "README.md" + } + webhook = { + artifact = { + zip = "README.md" + } + } + } + } + scale_set = { + network = { + vpc_id = "vpc-controller" + subnet_ids = ["subnet-controller-a", "subnet-controller-b"] + } + tags = { + Controller = "shared" + } + } + } + ssm = { + kms_key_id = "arn:aws:kms:eu-west-1:123456789012:key/11111111-1111-1111-1111-111111111111" + housekeeper = { + lambda = { + artifact = { + zip = "README.md" + } + } + } + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-runners" + subnet_ids = ["subnet-runners"] + instance_termination_watcher = { + enabled = true + enable_runner_deregistration = false + artifact = { + zip = "README.md" + } + } + runner_binaries = { + enabled = false + } + } + } + } + multi_runner_config = { + webhook = { + orchestration_provider = { + webhook = { + runner = { + maximum_count = 2 + } + github = { + organization_runners = true + } + matcherConfig = { + labelMatchers = [["self-hosted", "linux", "x64", "webhook"]] + } + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + binaries_syncer = { + enabled = false + } + } + } + } + } + scale_a = { + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example-a" + installation_id_ssm = { + name = "/scale-set/a/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set/a/installation-id" + kms_key_arn = "arn:aws:kms:eu-west-1:123456789012:key/22222222-2222-2222-2222-222222222222" + } + } + name = "scale-a" + id = 101 + min_runners = 1 + max_runners = 5 + boot_time_in_minutes = 12 + work_folder = "_work/a" + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + binaries_syncer = { + enabled = false + } + } + } + } + } + scale_b = { + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example-b/repository" + installation_id_ssm = { + name = "/scale-set/b/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set/b/installation-id" + } + } + name = "scale-b" + id = 102 + max_runners = 8 + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.xlarge"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + } + + assert { + condition = ( + toset(keys(local.webhook_runner_config)) == toset(["webhook"]) && + toset(keys(local.scale_set_runner_config)) == toset(["scale_a", "scale_b"]) && + toset(keys(aws_sqs_queue.queued_builds)) == toset(["webhook"]) && + toset(keys(local.runner_matcher_config)) == toset(["webhook"]) + ) + error_message = "Webhook queues and matcher routing must contain only webhook selections while scale-set selections remain in their own aggregate." + } + + assert { + condition = ( + length(module.orchestration_scale_set) == 1 && + toset(keys(module.orchestration_scale_set[0].controller_groups)) == toset(["ec2"]) && + toset(module.orchestration_scale_set[0].controller_groups["ec2"].runner_configs) == toset(["scale_a", "scale_b"]) && + toset(keys(local.scale_set_compute_provider_contracts)) == toset(["scale_a", "scale_b"]) + ) + error_message = "Multi-runner must call one scale-set orchestration module and allow it to group all selected runner configs across runner-config children." + } + + assert { + condition = ( + local.scale_set_runner_configs.scale_a.scale_set.boot_time_in_minutes == 12 && + local.scale_set_runner_configs.scale_b.scale_set.boot_time_in_minutes == 10 && + !local.scale_set_runner_configs.scale_a.github.ssl_verify && + local.scale_set_runner_configs.scale_a.github.app.installation_id.name == "/scale-set/a/installation-id" && + local.scale_set_runner_configs.scale_a.github.app.app_id.kms_key_arn == "arn:aws:kms:eu-west-1:123456789012:key/11111111-1111-1111-1111-111111111111" && + local.scale_set_runner_configs.scale_a.github.app.installation_id.kms_key_arn == "arn:aws:kms:eu-west-1:123456789012:key/22222222-2222-2222-2222-222222222222" && + local.scale_set_compute_provider_contracts.scale_a.type == "ec2" && + local.scale_set_compute_provider_contracts.scale_a.capabilities.scale_set != null + ) + error_message = "Per-runner scale-set identity, capacity, boot timeout, credentials, and exact compute capabilities must reach the aggregated controller contract." + } + + assert { + condition = ( + output.runners_map_v2.scale_a.scale_up == null && + output.runners_map_v2.scale_a.scale_down == null && + output.runners_map_v2.scale_a.pool == null && + output.runners_map_v2.scale_a.orchestration_provider.webhook == null && + output.runners_map_v2.scale_a.orchestration_provider.scale_set != null && + output.runners_map_v2.webhook.orchestration_provider.scale_set == null && + output.scale_set != null && + output.scale_set.controller_groups["ec2"] != null + ) + error_message = "Scale-set runners must keep webhook aliases null while grouped controller resources are exposed separately without changing runners_map_v2 entry shape." + } + + assert { + condition = ( + length(module.instance_termination_watcher) == 1 && + !local.translated_experimental.compute_provider.aws.ec2.instance_termination_watcher.enable_runner_deregistration + ) + error_message = "Mixed webhook and scale-set deployments may retain the shared termination watcher only in metrics-only mode." + } +} + +run "only_scale_set_keeps_shared_ingress_and_supports_custom_groups" { + command = plan + + variables { + experimental = { + runner = { + os = "linux" + architecture = "x64" + } + github = { + app = { + id = "123456" + key_base64 = "dGVzdA==" + webhook_secret = "test-secret" + } + } + orchestration_provider = { + webhook = { + lambda = { + webhook = { + artifact = { + zip = "README.md" + } + } + } + } + scale_set = { + grouping = { + strategy = "custom" + custom = { + groups = { + general = { + runner_configs = ["scale_a"] + } + isolated = { + runner_configs = ["scale_b"] + } + } + } + } + network = { + vpc_id = "vpc-controller" + subnet_ids = ["subnet-controller"] + } + } + } + ssm = { + housekeeper = { + lambda = { + artifact = { + zip = "README.md" + } + } + } + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-runners" + subnet_ids = ["subnet-runners"] + runner_binaries = { + enabled = false + } + } + } + } + multi_runner_config = { + scale_a = { + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example-a" + installation_id_ssm = { + name = "/scale-set/a/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set/a/installation-id" + } + } + name = "scale-a" + id = 201 + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + binaries_syncer = { + enabled = false + } + } + } + } + } + scale_b = { + orchestration_provider = { + scale_set = { + github = { + config_url = "https://github.com/example-b" + installation_id_ssm = { + name = "/scale-set/b/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set/b/installation-id" + } + } + name = "scale-b" + id = 202 + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m7i.large"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + } + + assert { + condition = ( + length(local.webhook_runner_config) == 0 && + length(aws_sqs_queue.queued_builds) == 0 && + length(local.runner_matcher_config) == 0 && + output.webhook != null && + toset(keys(module.orchestration_scale_set[0].controller_groups)) == toset(["general", "isolated"]) + ) + error_message = "A scale-set-only deployment must create no webhook runner queues, retain the unconditional shared ingress, and honor custom cross-runner grouping." + } +} diff --git a/modules/multi-runner/tests/provider-routing-v1.tftest.hcl b/modules/multi-runner/tests/provider-routing-v1.tftest.hcl index ebe79fb8aa..f9a42c048f 100644 --- a/modules/multi-runner/tests/provider-routing-v1.tftest.hcl +++ b/modules/multi-runner/tests/provider-routing-v1.tftest.hcl @@ -45,10 +45,13 @@ run "empty_runner_configurations_return_empty_output_maps" { length(local.raw_translated_experimental.multi_runner_config) == 0 && length(local.translated_experimental.multi_runner_config) == 0 && length(local.webhook_runner_config) == 0 + && length(local.scale_set_runner_config) == 0 && length(local.runner_matcher_config) == 0 && length(module.runner_configs) == 0 + && length(module.orchestration_scale_set) == 0 + && output.scale_set == null ) - error_message = "An empty stable and experimental configuration must translate to an empty raw runner-configuration map without selecting a v2 runner configuration." + error_message = "An empty stable and experimental configuration must not select v2 runner or scale-set orchestration resources." } assert { @@ -342,6 +345,7 @@ run "stable_v1_keeps_legacy_runner_module" { ]) && toset(keys(local.raw_translated_experimental.orchestration_provider)) == toset([ "webhook", + "scale_set", ]) && toset(keys(local.raw_translated_experimental.orchestration_provider.webhook)) == toset([ "queue_selection_strategy", @@ -390,6 +394,7 @@ run "stable_v1_keeps_legacy_runner_module" { ]) && toset(keys(local.raw_translated_experimental.multi_runner_config["linux"].orchestration_provider)) == toset([ "webhook", + "scale_set", ]) && toset(keys(local.raw_translated_experimental.multi_runner_config["linux"].orchestration_provider.webhook)) == toset([ "runner", @@ -790,6 +795,15 @@ run "stable_v1_keeps_legacy_runner_module" { error_message = "Stable multi_runner_config must not add entries to the experimental runners_map_v2 output." } + assert { + condition = ( + length(local.scale_set_runner_config) == 0 + && length(module.orchestration_scale_set) == 0 + && output.scale_set == null + ) + error_message = "Stable v1 must not select, instantiate, or expose the experimental scale-set orchestration provider." + } + assert { condition = toset(keys(output.runners_map["linux"])) == toset( [ diff --git a/modules/multi-runner/tests/provider-routing-v2.tftest.hcl b/modules/multi-runner/tests/provider-routing-v2.tftest.hcl index 531d32ee12..5b68bfc554 100644 --- a/modules/multi-runner/tests/provider-routing-v2.tftest.hcl +++ b/modules/multi-runner/tests/provider-routing-v2.tftest.hcl @@ -842,8 +842,9 @@ run "experimental_v2_rejects_missing_orchestration_provider" { experimental = { github = { app = { - id = "123456" - key_base64 = "dGVzdA==" + id = "123456" + key_base64 = "dGVzdA==" + webhook_secret = "test-secret" } } compute_provider = { @@ -851,6 +852,9 @@ run "experimental_v2_rejects_missing_orchestration_provider" { ec2 = { vpc_id = "vpc-missing-orchestration-provider" subnet_ids = ["subnet-missing-orchestration-provider"] + runner_binaries = { + enabled = false + } } } } @@ -897,6 +901,9 @@ run "experimental_v2_requires_webhook_maximum_count" { ec2 = { vpc_id = "vpc-missing-webhook-maximum" subnet_ids = ["subnet-missing-webhook-maximum"] + runner_binaries = { + enabled = false + } } } } @@ -3457,6 +3464,9 @@ run "experimental_v2_rejects_conflicting_queue_encryption" { ec2 = { vpc_id = "vpc-invalid-encryption" subnet_ids = ["subnet-invalid-encryption"] + runner_binaries = { + enabled = false + } } } } @@ -3820,6 +3830,9 @@ run "experimental_v2_rejects_runner_artifact_zip_and_s3" { ec2 = { vpc_id = "vpc-conflicting-runner-artifact" subnet_ids = ["subnet-conflicting-runner-artifact"] + runner_binaries = { + enabled = false + } } } } @@ -3897,6 +3910,9 @@ run "experimental_v2_rejects_runner_artifact_bucket_without_key" { ec2 = { vpc_id = "vpc-missing-runner-artifact-key" subnet_ids = ["subnet-missing-runner-artifact-key"] + runner_binaries = { + enabled = false + } } } } @@ -3966,6 +3982,9 @@ run "experimental_v2_rejects_runner_artifact_s3_without_bucket" { ec2 = { vpc_id = "vpc-missing-runner-artifact-bucket" subnet_ids = ["subnet-missing-runner-artifact-bucket"] + runner_binaries = { + enabled = false + } } } } @@ -4043,6 +4062,9 @@ run "experimental_v2_rejects_ssm_housekeeper_artifact_zip_and_s3" { ec2 = { vpc_id = "vpc-conflicting-ssm-housekeeper-artifact" subnet_ids = ["subnet-conflicting-ssm-housekeeper-artifact"] + runner_binaries = { + enabled = false + } } } } @@ -4110,6 +4132,9 @@ run "experimental_v2_rejects_ssm_housekeeper_artifact_s3_without_bucket" { ec2 = { vpc_id = "vpc-missing-ssm-housekeeper-artifact-bucket" subnet_ids = ["subnet-missing-ssm-housekeeper-artifact-bucket"] + runner_binaries = { + enabled = false + } } } } @@ -4184,6 +4209,9 @@ run "experimental_v2_rejects_ssm_housekeeper_artifact_s3_without_key" { ec2 = { vpc_id = "vpc-missing-ssm-housekeeper-artifact-key" subnet_ids = ["subnet-missing-ssm-housekeeper-artifact-key"] + runner_binaries = { + enabled = false + } } } } @@ -4540,6 +4568,9 @@ run "experimental_v2_rejects_invalid_ssm_housekeeper_state" { ec2 = { vpc_id = "vpc-invalid-housekeeper" subnet_ids = ["subnet-invalid-housekeeper"] + runner_binaries = { + enabled = false + } } } } diff --git a/modules/multi-runner/validations.experimental.tf b/modules/multi-runner/validations.experimental.tf index 2f4d4e061e..42e7d9734c 100644 --- a/modules/multi-runner/validations.experimental.tf +++ b/modules/multi-runner/validations.experimental.tf @@ -100,7 +100,54 @@ resource "terraform_data" "validate_experimental" { if orchestration_config != null ]) == 1 ]) - error_message = "Each experimental runner configuration must set exactly one orchestration block. Supported orchestration blocks: webhook." + error_message = "Each experimental runner configuration must set exactly one orchestration block. Supported orchestration blocks: webhook and scale_set." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.experimental.multi_runner_config) : + runner_config.orchestration_provider.scale_set == null ? true : ( + runner_config.orchestration_provider.scale_set.boot_time_in_minutes >= 1 && + runner_config.orchestration_provider.scale_set.boot_time_in_minutes <= 120 && + floor(runner_config.orchestration_provider.scale_set.boot_time_in_minutes) == runner_config.orchestration_provider.scale_set.boot_time_in_minutes + ) + ]) + error_message = "Each experimental scale_set boot_time_in_minutes must be an integer between 1 and 120." + } + + precondition { + condition = length(local.scale_set_runner_config) == 0 ? true : ( + try(length(trimspace(var.experimental.orchestration_provider.scale_set.network.vpc_id)) > 0, false) && + try(length(var.experimental.orchestration_provider.scale_set.network.subnet_ids) > 0, false) + ) + error_message = "experimental.orchestration_provider.scale_set.network.vpc_id and subnet_ids are required when a runner configuration selects scale_set." + } + + precondition { + condition = length(local.scale_set_runner_config) == 0 ? true : ( + !var.experimental.compute_provider.aws.ec2.instance_termination_watcher.enabled || + !var.experimental.compute_provider.aws.ec2.instance_termination_watcher.enable_runner_deregistration + ) + error_message = "experimental.compute_provider.aws.ec2.instance_termination_watcher.enable_runner_deregistration must be false when a runner configuration selects scale_set; the scale-set reconciler owns GitHub runner deregistration." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.experimental.multi_runner_config) : + runner_config.orchestration_provider.scale_set == null ? true : ( + can(regex("^https://", runner_config.orchestration_provider.scale_set.github.config_url)) && + length(trimspace(runner_config.orchestration_provider.scale_set.github.installation_id_ssm.name)) > 0 && + length(trimspace(runner_config.orchestration_provider.scale_set.github.installation_id_ssm.arn)) > 0 && + ( + runner_config.orchestration_provider.scale_set.github.installation_id_ssm.kms_key_arn == null || + can(regex( + "^arn:[^:]+:kms:[^:]+:[0-9]{12}:key/.+$", + runner_config.orchestration_provider.scale_set.github.installation_id_ssm.kms_key_arn, + )) + ) + ) + ]) + error_message = "Each experimental scale_set selection must use an HTTPS github.config_url, a non-empty installation_id_ssm name and ARN, and a KMS key ARN when installation_id_ssm.kms_key_arn is set." } precondition { diff --git a/modules/multi-runner/variables.experimental.tf b/modules/multi-runner/variables.experimental.tf index ae19eab604..c1795ba84b 100644 --- a/modules/multi-runner/variables.experimental.tf +++ b/modules/multi-runner/variables.experimental.tf @@ -10,7 +10,7 @@ variable "experimental" { Global experimental fields support the following nested properties. These values apply directly when `experimental.multi_runner_config` is non-empty; in stable mode, flat inputs are translated into the same global shape. - - `tags`: Base tags for v2 build queues, runner configurations, the shared GitHub App Parameter Store module, webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `{}`. + - `tags`: Base tags for v2 build queues, runner configurations, the shared GitHub App Parameter Store module, webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `{}`. When any runner selects EC2 scale-set orchestration, values inherited into its effective Parameter Store tag map participate in that runtime's tag limits and validation. - `roles.path`: Default IAM path for module-managed v2 runner and Lambda roles and for the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is null. - `roles.permissions_boundary`: Default permissions-boundary ARN for module-managed v2 runner and Lambda roles and for the shared webhook, runner-binary syncer, termination-watcher, and AMI-housekeeper Lambda roles. The default is null. - `runner.os`: Default runner operating system. The default is null; every runner configuration must resolve this field globally or locally. @@ -18,7 +18,7 @@ variable "experimental" { - `runner.disable_default_labels`: Omits the default self-hosted, operating-system, and architecture labels when true. The default is `false`. - `runner.extra_labels`: Default additional labels combined with each runner configuration's matcher labels. The default is `[]`. - `runner.group_name`: Default GitHub runner group. The default is `Default`. - - `runner.name_prefix`: Default prefix added to registered runner names. The default is an empty string. + - `runner.name_prefix`: Default prefix added to registered runner names. The default is an empty string. EC2 scale-set orchestration requires at most 45 ASCII letters, digits, dots, underscores, or hyphens. - `runner.run_as_root`: Runs the runner service as root when supported by the provider. The default is `false`. - `runner.run_as`: Default operating-system user when `run_as_root` is false. The default is `ec2-user`. - `runner.auto_update_disabled`: Disables the GitHub runner application's built-in updater. The default is `false`. @@ -58,8 +58,8 @@ variable "experimental" { - `github.additional_apps[].installation_id_ssm.arn`: ARN of the existing installation-ID parameter. - `github.additional_apps[].installation_id_ssm.name`: Name of the existing installation-ID parameter. - `github.enterprise_server.url`: GitHub Enterprise Server URL used by v2 runner-config GitHub clients and the shared termination watcher. The default is null. - - `github.enterprise_server.ssl_verify`: Enables TLS certificate verification for v2 runner-config GitHub clients. The default is `true`. - - `github.user_agent`: HTTP User-Agent used by v2 runner-config GitHub clients. The default is `github-aws-runners`. + - `github.enterprise_server.ssl_verify`: Enables TLS certificate verification for v2 runner-config GitHub clients. Scale-set controllers apply a disabled value to that reconciler's GitHub App and scale-set requests without changing process-global TLS behavior. The default is `true`. + - `github.user_agent`: Client identity used by v2 runner-config GitHub clients. Scale-set controllers preserve the required structured protocol User-Agent and place this value in its `system` field. The default is `github-aws-runners`. - `lambda.artifact.s3.bucket`: Optional shared S3 bucket containing Lambda deployment artifacts for v2 runner configurations and their SSM housekeepers, the webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is null. A component selects an object from this bucket only when its own `artifact.s3` wrapper is present. - `lambda.runtime`: Runtime for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `nodejs24.x`. - `lambda.architecture`: Architecture for v2 runner-config functions and the shared webhook, runner-binary syncer, termination watcher, and AMI housekeeper. The default is `arm64`. @@ -134,14 +134,59 @@ variable "experimental" { - `orchestration_provider.webhook.queue.encryption.kms_data_key_reuse_period_seconds`: KMS data-key reuse period in seconds. This key is required syntactically in an explicit `queue.encryption` object but may be null; it is used only with `kms_master_key_id`. - `orchestration_provider.webhook.queue.encryption.kms_master_key_id`: KMS key ARN used for queue encryption. This AWS-facing field name is retained for compatibility, but non-null values must be key ARNs because runner-config IAM policies cannot use key IDs or aliases as resources. The field is required syntactically in an explicit `queue.encryption` object but may be null, and a computed ARN may remain unknown until apply. It is independent from `ssm.kms_key_id` and is forwarded separately to each webhook runner configuration so runner-config scale-up and job-retry roles receive only the build-queue KMS permissions they require. - `orchestration_provider.webhook.queue.encryption.sqs_managed_sse_enabled`: Selects the non-KMS mode: `true` enables SQS-managed encryption and `false` explicitly disables queue encryption. This key is required syntactically in an explicit `queue.encryption` object and must be null when `kms_master_key_id` is set. Omitting the whole encryption block defaults it to `true`. + - `orchestration_provider.scale_set`: Shared scale-set controller topology and runtime defaults. Multi-runner creates this provider once when at least one runner config selects `scale_set`; the provider then packs the selected reconcilers into controller groups. + - `orchestration_provider.scale_set.grouping.strategy`: Controller grouping strategy. `compute_provider` (the default) creates one controller group per compute-provider type, `runner_config` creates one group per runner config, and `custom` uses the explicit group map. + - `orchestration_provider.scale_set.grouping.custom`: Explicit controller groups. This must be non-null only when the strategy is `custom`, and membership must cover every selected scale-set runner config exactly once. + - `orchestration_provider.scale_set.grouping.custom.groups`: Controller groups keyed by stable group name. + - `orchestration_provider.scale_set.grouping.custom.groups..runner_configs`: Set of scale-set runner-config keys assigned to the group. + - `orchestration_provider.scale_set.container.image`: Controller image reference. The default is null, which selects the release's official scale-set service image; callers can override it with a compatible image. + - `orchestration_provider.scale_set.container.user`: Numeric user and optional group used by the hardened Fargate container. The default is `10001:10001`. + - `orchestration_provider.scale_set.container.health_port`: Loopback HTTP health-listener port. The default is `8080`. + - `orchestration_provider.scale_set.container.health_path`: ECS liveness endpoint. The only supported value is `/healthz`. + - `orchestration_provider.scale_set.container.health_check_command`: Optional ECS container health-check command. Null uses the built-in Node probe against `/healthz`. + - `orchestration_provider.scale_set.container.health_check_interval`: ECS health-check interval in seconds. The default is `30`. + - `orchestration_provider.scale_set.container.health_check_timeout`: ECS health-check timeout in seconds. The default is `5`. + - `orchestration_provider.scale_set.container.health_check_retries`: Consecutive failed checks before ECS marks the task unhealthy. The default is `3`. + - `orchestration_provider.scale_set.container.health_check_start_period`: Startup grace period for ECS health checks in seconds. The default is `30`. + - `orchestration_provider.scale_set.container.health_stale_after_seconds`: Maximum allowed age of a successful controller reconciliation before liveness fails. The default is `180`. + - `orchestration_provider.scale_set.container.shutdown_timeout_seconds`: Maximum controller shutdown-drain period. The default is `110`. + - `orchestration_provider.scale_set.container.session_close_timeout_seconds`: Maximum wait for a GitHub scale-set session to close during shutdown. The default is `10`. + - `orchestration_provider.scale_set.container.reconnect_initial_backoff_seconds`: Initial reconnect delay after a transient session failure. The default is `1`. + - `orchestration_provider.scale_set.container.reconnect_max_backoff_seconds`: Maximum transient-session reconnect delay. The default is `30`. + - `orchestration_provider.scale_set.container.stop_timeout_seconds`: ECS stop timeout. The default is `120` and must leave enough time for the configured controller shutdown. + - `orchestration_provider.scale_set.container.ecr_repository`: Optional private-ECR repository wrapper. Its presence adds narrowly scoped image-pull permissions to the execution role. + - `orchestration_provider.scale_set.container.ecr_repository.arn`: ARN of the private ECR repository containing the selected image. + - `orchestration_provider.scale_set.config_store.path_prefix`: SSM path prefix for non-secret reconciler manifests. Null derives `//scale-set-controller`. + - `orchestration_provider.scale_set.config_store.tier`: Parameter Store tier for reconciler manifests. The default is `Standard`; `Advanced` permits larger manifests. + - `orchestration_provider.scale_set.config_store.tags`: Tags applied to reconciler manifest parameters. The default is `{}`. + - `orchestration_provider.scale_set.ecs.cluster.mode`: ECS cluster ownership mode. `managed` (the default) creates a cluster; `external` uses `cluster.arn`. + - `orchestration_provider.scale_set.ecs.cluster.arn`: Existing ECS cluster ARN required in external mode. + - `orchestration_provider.scale_set.ecs.cluster.name`: Optional name for a managed ECS cluster. Null derives a stable name. + - `orchestration_provider.scale_set.ecs.cluster.container_insights`: Enables Container Insights on a managed cluster. The default is `true`. + - `orchestration_provider.scale_set.ecs.task.cpu`: Fargate task CPU units per controller group. The default is `512`. + - `orchestration_provider.scale_set.ecs.task.memory`: Fargate task memory in MiB per controller group. The default is `1024`. + - `orchestration_provider.scale_set.ecs.task.cpu_architecture`: Fargate CPU architecture. The default is `X86_64`; `ARM64` is also supported when the selected image is compatible. + - `orchestration_provider.scale_set.ecs.task.ephemeral_storage.size_in_gib`: Optional Fargate ephemeral-storage size in GiB. Null uses the AWS default. + - `orchestration_provider.scale_set.ecs.service.platform_version`: Fargate platform version. The default is `LATEST`. + - `orchestration_provider.scale_set.ecs.iam.path`: IAM path for controller task and execution roles. The default is `/`. + - `orchestration_provider.scale_set.ecs.iam.permissions_boundary`: Optional permissions-boundary ARN for controller task and execution roles. The default is null. + - `orchestration_provider.scale_set.network.vpc_id`: VPC for private Fargate controller tasks. It is required when any runner config selects `scale_set`. + - `orchestration_provider.scale_set.network.subnet_ids`: Private subnets for Fargate controller tasks. At least one is required when any runner config selects `scale_set`; tasks never receive public IP addresses. + - `orchestration_provider.scale_set.network.https_egress.ipv4_cidrs`: IPv4 CIDRs allowed for outbound HTTPS. The default is `0.0.0.0/0`; use controlled NAT, firewall, or proxy routing when required. + - `orchestration_provider.scale_set.network.https_egress.ipv6_cidrs`: IPv6 CIDRs allowed for outbound HTTPS. The default is `[]`. + - `orchestration_provider.scale_set.logging.retention_in_days`: CloudWatch Logs retention period for controller groups. The default is `30`. + - `orchestration_provider.scale_set.logging.kms_key_arn`: Optional customer-managed KMS key ARN for controller log groups. The default is null. + - `orchestration_provider.scale_set.logging.log_group_class`: Controller log-group class. The default is `STANDARD`. + - `orchestration_provider.scale_set.logging.tags`: Tags applied to controller log groups. The default is `{}`. + - `orchestration_provider.scale_set.tags`: Tags applied to shared scale-set controller resources after global experimental tags. The default is `{}`. - `ssm.paths.root`: Base Parameter Store path for shared GitHub App and webhook parameters and for all v2 runner configurations. The schema default is null, which derives `/github-action-runners/`; normalization appends the configuration key only for configuration-owned paths. - `ssm.paths.app`: Shared GitHub App credential path segment below `ssm.paths.root`. The default is `app`. - `ssm.paths.webhook`: Shared webhook matcher-configuration path segment below `ssm.paths.root`. The default is `webhook`. - `ssm.paths.tokens`: Runner registration-token and JIT-configuration path segment below each runner-configuration root. The default is `runners/tokens`. - `ssm.paths.config`: Persistent runner-configuration path segment below each runner-configuration root. The default is `runners/config`. - `ssm.kms_key_id`: Optional global KMS key ARN that encrypts shared GitHub App parameters, configures the webhook and termination watcher, and adds matching decrypt permissions to every v2 runner configuration. The default is null and its value may be unknown until apply. It does not select encryption for runtime-created runner-configuration parameters. - - `ssm.tags`: Default tags for the shared GitHub App Parameter Store module and runner-configuration-owned SSM resources. The default is `{}`. - - `ssm.parameters.tags`: Default tags for Terraform-managed and runtime-created runner-configuration parameters. The default is `{}`. + - `ssm.tags`: Default tags for the shared GitHub App Parameter Store module and runner-configuration-owned SSM resources. The default is `{}`. For EC2 scale-set orchestration, inherited values participate in the effective 45-tag runtime limit and key/value validation. + - `ssm.parameters.tags`: Default tags for Terraform-managed and runtime-created runner-configuration parameters. The default is `{}` and inherited values participate in the EC2 scale-set runtime tag limit. - `ssm.housekeeper.schedule_expression`: Default EventBridge schedule for each runner-configuration SSM housekeeper. The default is `rate(1 day)`. - `ssm.housekeeper.state`: Default EventBridge rule state for each runner-configuration SSM housekeeper. The default is `ENABLED`. - `ssm.housekeeper.tags`: Default tags for SSM housekeeper resources. The default is `{}`. @@ -212,10 +257,10 @@ variable "experimental" { - `compute_provider.aws.ec2.ami.housekeeper.lambda.timeout`: AMI-housekeeper Lambda timeout in seconds. The default is `300`. - `compute_provider.aws.ec2.ami.housekeeper.schedule.expression`: AMI-housekeeper EventBridge schedule expression. The default is `cron(11 7 * * ? *)`. - `compute_provider.aws.ec2.instance_termination_watcher`: Global configuration for the shared EC2 instance-termination watcher. - - `compute_provider.aws.ec2.instance_termination_watcher.enabled`: Creates the shared EC2 termination watcher when true. The default is `false`, and the value must be known during planning because it controls the module instance. + - `compute_provider.aws.ec2.instance_termination_watcher.enabled`: Creates the shared EC2 termination watcher when true. The default is `false`, and the value must be known during planning because it controls the module instance. Scale-set runner configs may use the watcher for logging and metrics only, with runner deregistration disabled. - `compute_provider.aws.ec2.instance_termination_watcher.features.enable_spot_termination_handler`: Enables the Spot termination-event handler. The default is `true`, and the value must be known during planning because it controls child resources. - `compute_provider.aws.ec2.instance_termination_watcher.features.enable_spot_termination_notification_watcher`: Enables the Spot interruption-warning watcher. The default is `true`, and the value must be known during planning because it controls child resources. - - `compute_provider.aws.ec2.instance_termination_watcher.enable_runner_deregistration`: Deregisters terminated runners from GitHub when true. The default is `true`, and the value must be known during planning because it controls deregistration resources. + - `compute_provider.aws.ec2.instance_termination_watcher.enable_runner_deregistration`: Deregisters terminated runners from GitHub when true. The default is `true`, and the value must be known during planning because it controls deregistration resources. This must be false when any runner config selects scale-set orchestration because the scale-set reconciler owns GitHub runner deregistration. - `compute_provider.aws.ec2.instance_termination_watcher.environment_variables`: Additional termination-watcher Lambda environment variables. The default is `{}`. - `compute_provider.aws.ec2.instance_termination_watcher.artifact`: Termination-watcher artifact selection. Set at most one of `zip` or `s3`; when both are null, the packaged archive is used. - `compute_provider.aws.ec2.instance_termination_watcher.artifact.zip`: Optional local path to the termination-watcher Lambda archive. The default is null. @@ -261,7 +306,7 @@ variable "experimental" { - `multi_runner_config[].runner.disable_default_labels`: Prevents GitHub default labels from being registered. - `multi_runner_config[].runner.extra_labels`: Additional labels combined with `orchestration_provider.webhook.matcherConfig.labelMatchers` for webhook runner configurations. Default self-hosted, operating-system, and architecture labels are also included unless `runner.disable_default_labels` is true. - `multi_runner_config[].runner.group_name`: GitHub runner group used during registration. - - `multi_runner_config[].runner.name_prefix`: Prefix added to registered runner names. + - `multi_runner_config[].runner.name_prefix`: Prefix added to registered runner names. EC2 scale-set orchestration requires at most 45 ASCII letters, digits, dots, underscores, or hyphens. - `multi_runner_config[].runner.run_as_root`: Runs the runner service as root when supported by the compute provider. - `multi_runner_config[].runner.run_as`: Operating-system user used when `run_as_root` is false. - `multi_runner_config[].runner.auto_update_disabled`: Disables the GitHub runner application's built-in updater. @@ -282,7 +327,22 @@ variable "experimental" { - `multi_runner_config[].lambda.tags`: Per-configuration tags for control-plane Lambda functions. Component tags override this map. - `multi_runner_config[].lambda.role.path`: Per-configuration IAM path for module-managed Lambda roles. Null inherits `experimental.lambda.role.path`, then `experimental.roles.path`. - `multi_runner_config[].lambda.role.permissions_boundary`: Per-configuration permissions-boundary ARN for module-managed Lambda roles. Null inherits `experimental.lambda.role.permissions_boundary`, then `experimental.roles.permissions_boundary`. - - `multi_runner_config[].orchestration_provider`: Demand-controller selection. Exactly one typed provider block must be non-null. Only `webhook` is supported today; additional providers can be added without moving the webhook contract. + - `multi_runner_config[].orchestration_provider`: Demand-controller selection. Exactly one of the typed `webhook` or `scale_set` blocks must be non-null, and the selected wrapper must be known during planning. + - `multi_runner_config[].orchestration_provider.scale_set`: Selects stateful GitHub Actions runner scale-set orchestration. Runner-config applies fixed ephemeral JIT lifecycle settings, while multi-runner aggregates all selected runner configs into the shared controller topology. + - `multi_runner_config[].orchestration_provider.scale_set.github.config_url`: HTTPS GitHub organization, repository, or enterprise scope URL owned by this scale set. + - `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm`: Existing Parameter Store reference containing the GitHub App installation ID for this runner config's GitHub scope. The manifest contains only the parameter name, never the credential value. + - `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm.name`: Absolute Parameter Store name containing the installation ID. + - `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm.arn`: Exact same-account, same-region ARN of the installation-ID parameter. + - `multi_runner_config[].orchestration_provider.scale_set.github.installation_id_ssm.kms_key_arn`: Optional KMS key ARN needed to decrypt the installation-ID parameter. The default is null and is independent from global `ssm.kms_key_id`. + - `multi_runner_config[].orchestration_provider.scale_set.github.force_ghes`: Optional explicit GitHub Enterprise Server mode. Null enables it when global `github.enterprise_server.url` is set and otherwise uses GitHub.com behavior. + - `multi_runner_config[].orchestration_provider.scale_set.name`: Expected GitHub runner scale-set name used to verify controller ownership. + - `multi_runner_config[].orchestration_provider.scale_set.id`: Existing GitHub runner scale-set numeric ID. The controller does not create or discover scale sets in this foundation. + - `multi_runner_config[].orchestration_provider.scale_set.runner_group_id`: Optional expected GitHub runner-group ID. The default is null. + - `multi_runner_config[].orchestration_provider.scale_set.min_runners`: Minimum desired runner count reconciled for the scale set. The default is `0`. + - `multi_runner_config[].orchestration_provider.scale_set.max_runners`: Maximum desired runner count reconciled for the scale set. The default is `10`. + - `multi_runner_config[].orchestration_provider.scale_set.boot_time_in_minutes`: Expected compute boot time used to expire stale pending runner requests. The default is `10`; valid values are integers from `1` through `120`. + - `multi_runner_config[].orchestration_provider.scale_set.session_owner`: Optional stable owner used for the GitHub message session. Null derives one from the controller group and runner-config key. + - `multi_runner_config[].orchestration_provider.scale_set.work_folder`: Optional relative runner work folder. The default is null, which lets the compute provider use its default. - `multi_runner_config[].orchestration_provider.webhook`: Selects the workflow-job webhook control plane, including its SQS build queue, scale-up, scheduled scale-down/pool, and optional job-retry resources. - `multi_runner_config[].orchestration_provider.webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by webhook scale-down and pool controls. Null inherits `experimental.orchestration_provider.webhook.runner.boot_time_in_minutes`. - `multi_runner_config[].orchestration_provider.webhook.runner.ephemeral`: Registers webhook-orchestrated runners in ephemeral mode. Null inherits `experimental.orchestration_provider.webhook.runner.ephemeral`. @@ -344,8 +404,8 @@ variable "experimental" { - `multi_runner_config[].ssm.paths.root`: Base Parameter Store root for this runner configuration. The configuration key is always appended to preserve configuration isolation. The omitted global root derives `/github-action-runners/`. - `multi_runner_config[].ssm.paths.tokens`: Path segment below the runner-configuration root used for runner registration tokens and just-in-time configuration. - `multi_runner_config[].ssm.paths.config`: Path segment below the runner-configuration root used for persistent runner configuration. - - `multi_runner_config[].ssm.tags`: Shared tags for runner-configuration-owned SSM resources. These override entry-level `tags`. - - `multi_runner_config[].ssm.parameters.tags`: Tags for Terraform-managed and runtime-created runner configuration parameters. These override `ssm.tags`. + - `multi_runner_config[].ssm.tags`: Shared tags for runner-configuration-owned SSM resources. These override entry-level `tags`. For EC2 scale-set orchestration, the effective Parameter Store tag map contains at most 45 runtime-compatible keys and values. + - `multi_runner_config[].ssm.parameters.tags`: Tags for Terraform-managed and runtime-created runner configuration parameters. These override `ssm.tags` and participate in the EC2 scale-set runtime tag limit. - `multi_runner_config[].ssm.housekeeper.schedule_expression`: EventBridge schedule expression that invokes the runner-configuration SSM housekeeper. - `multi_runner_config[].ssm.housekeeper.state`: EventBridge rule state for the runner-configuration SSM housekeeper. Valid values are `DISABLED`, `ENABLED`, and `ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS`. - `multi_runner_config[].ssm.housekeeper.tags`: Tags for SSM housekeeper resources. These override entry-level, shared Lambda, shared log, and `ssm.tags` values. @@ -381,7 +441,7 @@ variable "experimental" { - `multi_runner_config[].compute_provider.aws.ec2.ami.filter`: EC2 AMI filters combined with the default AMI-name filter. - `multi_runner_config[].compute_provider.aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI. - `multi_runner_config[].compute_provider.aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. The wrapper's presence selects external ownership at plan time. - - `multi_runner_config[].compute_provider.aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply. + - `multi_runner_config[].compute_provider.aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply. Scale-set selections require an exact same-account, same-region ARN whose extracted absolute name matches the EC2 runtime grammar. - `multi_runner_config[].compute_provider.aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence selects the KMS policy at plan time. - `multi_runner_config[].compute_provider.aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply. - `multi_runner_config[].compute_provider.aws.ec2.block_device_mappings`: EBS mappings added to the runner launch template. @@ -409,7 +469,7 @@ variable "experimental" { - `multi_runner_config[].compute_provider.aws.ec2.user_data.pre_install`: Script content inserted before runner installation in the default template. - `multi_runner_config[].compute_provider.aws.ec2.user_data.post_install`: Script content inserted after runner installation in the default template. - `multi_runner_config[].compute_provider.aws.ec2.user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets in logs. - - `multi_runner_config[].compute_provider.aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select capacity. + - `multi_runner_config[].compute_provider.aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select capacity. Scale-set selections allow only `lowest-price` or `prioritized` with `on-demand`; Spot supports the complete EC2 provider strategy set. - `multi_runner_config[].compute_provider.aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price. - `multi_runner_config[].compute_provider.aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`. - `multi_runner_config[].compute_provider.aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type. @@ -455,7 +515,7 @@ variable "experimental" { - `multi_runner_config[].compute_provider.aws.ec2.log_files[].file_path`: File or glob read by the CloudWatch agent. - `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_stream_name`: CloudWatch log-stream name template. - `multi_runner_config[].compute_provider.aws.ec2.log_files[].log_class`: CloudWatch log-group class for the collected file. - - `multi_runner_config[].compute_provider.aws.ec2.tags`: Tags for runtime EC2 instances, volumes, network interfaces, and eligible Spot requests. These override entry-level tags and the generated runner `Name`; provider-required bootstrap tags take final precedence. + - `multi_runner_config[].compute_provider.aws.ec2.tags`: Tags for runtime EC2 instances, volumes, network interfaces, and eligible Spot requests. These override entry-level tags and the generated runner `Name`; provider-required bootstrap tags take final precedence. Scale-set selections reject caller values for scale-set ownership and lifecycle keys. EOT type = object({ @@ -653,6 +713,79 @@ variable "experimental" { }) }), {}) }), {}) + scale_set = optional(object({ + grouping = optional(object({ + strategy = optional(string, "compute_provider") + custom = optional(object({ + groups = map(object({ + runner_configs = set(string) + })) + }), null) + }), {}) + container = optional(object({ + image = optional(string, null) + user = optional(string, "10001:10001") + health_port = optional(number, 8080) + health_path = optional(string, "/healthz") + health_check_command = optional(list(string), null) + health_check_interval = optional(number, 30) + health_check_timeout = optional(number, 5) + health_check_retries = optional(number, 3) + health_check_start_period = optional(number, 30) + health_stale_after_seconds = optional(number, 180) + shutdown_timeout_seconds = optional(number, 110) + session_close_timeout_seconds = optional(number, 10) + reconnect_initial_backoff_seconds = optional(number, 1) + reconnect_max_backoff_seconds = optional(number, 30) + stop_timeout_seconds = optional(number, 120) + ecr_repository = optional(object({ + arn = string + }), null) + }), {}) + config_store = optional(object({ + path_prefix = optional(string, null) + tier = optional(string, "Standard") + tags = optional(map(string), {}) + }), {}) + ecs = optional(object({ + cluster = optional(object({ + mode = optional(string, "managed") + arn = optional(string, null) + name = optional(string, null) + container_insights = optional(bool, true) + }), {}) + task = optional(object({ + cpu = optional(number, 512) + memory = optional(number, 1024) + cpu_architecture = optional(string, "X86_64") + ephemeral_storage = optional(object({ + size_in_gib = number + }), null) + }), {}) + service = optional(object({ + platform_version = optional(string, "LATEST") + }), {}) + iam = optional(object({ + path = optional(string, "/") + permissions_boundary = optional(string, null) + }), {}) + }), {}) + network = optional(object({ + vpc_id = optional(string, null) + subnet_ids = optional(set(string), null) + https_egress = optional(object({ + ipv4_cidrs = optional(set(string), ["0.0.0.0/0"]) + ipv6_cidrs = optional(set(string), []) + }), {}) + }), {}) + logging = optional(object({ + retention_in_days = optional(number, 30) + kms_key_arn = optional(string, null) + log_group_class = optional(string, "STANDARD") + tags = optional(map(string), {}) + }), {}) + tags = optional(map(string), {}) + }), {}) }), {}) ssm = optional(object({ @@ -985,6 +1118,26 @@ variable "experimental" { }), null) + scale_set = optional(object({ + github = object({ + config_url = string + installation_id_ssm = object({ + name = string + arn = string + kms_key_arn = optional(string, null) + }) + force_ghes = optional(bool, null) + }) + name = string + id = number + runner_group_id = optional(number, null) + min_runners = optional(number, 0) + max_runners = optional(number, 10) + boot_time_in_minutes = optional(number, 10) + session_owner = optional(string, null) + work_folder = optional(string, null) + }), null) + }) ssm = optional(object({ @@ -1127,7 +1280,6 @@ variable "experimental" { "TargetCapacityLimitExceededException", "RequestLimitExceeded", "ResourceLimitExceeded", - "MaxSpotInstanceCountExceeded", "MaxSpotFleetRequestCountExceeded", "InsufficientInstanceCapacity", "InsufficientCapacityOnHost", diff --git a/modules/orchestration-providers/scale-set/README.md b/modules/orchestration-providers/scale-set/README.md new file mode 100644 index 0000000000..13d0476599 --- /dev/null +++ b/modules/orchestration-providers/scale-set/README.md @@ -0,0 +1,215 @@ +# Scale-set orchestration provider + +This internal module deploys long-running GitHub Actions runner scale-set controllers on ECS Fargate. It creates one deployment unit per resolved **controller group**: + +```text +1 ECS service +1 task definition +1 running task during normal operation +1 application container +1 ScaleSetController supervising N independent reconcilers +``` + +Each reconciler still owns exactly one GitHub scale-set ID and one message session. Grouping only packs reconcilers into a shared task; it does not merge scale-set identity, session state, or compute-provider behavior. It does, however, intentionally union task IAM permissions and failure/deployment blast radius across all members of that controller group. + +This foundation adopts scale sets that were created elsewhere. It validates the supplied ID, expected name, and optional runner-group ID at runtime, but it does not create or delete the GitHub scale-set resource. The complete compute-provider contract must likewise come from its Terraform adapter; until that adapter and the public runner-config selection are wired, this internal module is not an end-to-end deployment interface. + +The normalized `(github.config_url, scale_set.id)` ownership tuple must be globally unique across all groups. Duplicate detection normalizes URL case, one trailing slash, and an explicit default `:443` port, so equivalent spellings cannot accidentally deploy two services against one GitHub message session. Numeric scale-set IDs may repeat under different GitHub scopes. + +## Grouping + +`grouping.strategy` selects a plan-known grouping implementation: + +- `compute_provider` (default): one group per `compute_provider_contracts[*].type`. +- `runner_config`: one group per runner-config key. +- `custom`: explicit groups whose membership covers every runner config exactly once. + +```hcl +grouping = { + strategy = "custom" + custom = { + groups = { + general = { + runner_configs = ["linux-small", "linux-large"] + } + critical = { + runner_configs = ["production"] + } + } + } +} +``` + +Group names and memberships become Terraform `for_each` identities and must be known during planning. A group may contain at most 1000 runner configs, matching the service loader limit. Additional grouping algorithms can be added later by producing the same internal `map(list(runner_config_name))` shape. + +## Compute-provider capability boundary + +`compute_provider_contracts` is keyed exactly like `runner_configs`. A compute provider implements the scale-set desired-capacity interface by returning: + +```hcl +{ + type = "ec2" # plan-known grouping and runtime registry key + capabilities = { + scale_set = { + configuration_json = local.provider_owned_runtime_configuration + environment_variables = local.provider_owned_non_secret_environment + iam_statements = local.provider_owned_task_role_statements + } + } +} +``` + +The symbolic locals above represent outputs from the selected compute-provider Terraform adapter; callers should not recreate the provider payload by hand. The provider-specific adapter owns the runtime configuration schema and the complete IAM statement set. This orchestration module treats configuration JSON as an opaque, non-secret object and combines only the selected group's statements into that group's task role. Provider-owned process environment variables are also non-secret: duplicate names within a group must resolve to the same value, and reserved runtime names cannot be overridden. Runner-config-specific values stay in the SSM reconciler document, while credentials stay behind SSM references. Wildcard IAM actions are rejected. The rendered per-group policy is checked against AWS's 10,240-byte inline role-policy quota with an explicit split-the-group error; group splitting remains the escape hatch when the union is too large or too broad. + +## Configuration delivery + +Large groups do not embed their full manifest in an ECS task definition. The module writes one non-secret SSM `String` parameter per reconciler: + +```text +//scale-set-controller// +``` + +Each leaf is the flat `ScaleSetReconcilerConfig` consumed by the service: + +```json +{ + "schemaVersion": 1, + "runnerConfigName": "linux-small", + "githubConfigUrl": "https://github.com/example", + "scaleSetId": 123, + "expectedScaleSetName": "linux-small", + "expectedRunnerGroupId": null, + "minRunners": 0, + "maxRunners": 20, + "bootTimeoutMinutes": 10, + "githubApp": { + "appIdParameterName": "/github/app-id", + "privateKeyParameterName": "/github/private-key", + "installationIdParameterName": "/github/installation-id" + }, + "computeProvider": { + "type": "ec2", + "configuration": {} + } +} +``` + +The task receives only the group name, group path, and a SHA-256 revision. It loads the direct children with `GetParametersByPath`. Standard parameters are limited to 4096 encoded bytes and Advanced parameters to 8192 encoded bytes; Terraform validates every leaf against the selected tier and limits the decoded aggregate for one group to 4 MiB. The group revision changes the task definition whenever any reconciler configuration changes. + +Terraform always emits `sessionOwner`. An omitted value normally resolves to `.`; if that would exceed the runtime's 256-character limit, the module truncates both readable components and appends a deterministic hash. + +GitHub credential **values** never enter Terraform configuration, task definitions, or controller-config parameters. Each leaf carries only three Parameter Store names. The task role can read the exact credential parameter ARNs for its group and decrypt only explicitly declared KMS keys. + +## Container image + +The convenience default is: + +```text +ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service:latest +``` + +ECS `versionConsistency` is enabled so all tasks in a deployment resolve a tag consistently. Production callers should set `container.image` to the digest published with a release: + +```hcl +container = { + image = "ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service@sha256:" +} +``` + +Public registry images need no pull permission. For a private ECR override, set `container.ecr_repository.arn`; the execution role receives repository-scoped layer permissions plus the unavoidable resource-unscoped `ecr:GetAuthorizationToken` action. + +For the official GHCR default, verify an anonymous pull after the first package publish. Package visibility may inherit repository or organization settings and must not be inferred only from a successful authenticated workflow push. + +## ECS and security behavior + +- A managed ECS cluster is created by default. Set `ecs.cluster.mode = "external"` and pass `ecs.cluster.arn` to reuse a cluster. The mode must be known at plan time; the ARN may be computed. +- Every group gets a separate service, task definition, task role, execution role, log group, and security group. +- `desired_count` is fixed at one. Deployment percentages are `minimum = 0` and `maximum = 100`, preventing old and new tasks from overlapping while session leasing is unavailable. +- The ECS deployment circuit breaker and rollback are enabled. +- Tasks run in supplied private subnets with public IP assignment disabled. Managed security groups have no ingress and allow only TCP/443 egress. The IPv4 Internet default is intended for controlled NAT/firewall paths and can be narrowed. +- The application container runs with a numeric non-root UID/GID, a read-only root filesystem, init enabled, no privilege, and all Linux capabilities dropped. +- ECS probes `/healthz` for liveness, and `container.health_path` accepts only that endpoint. `/readyz` remains an application readiness signal; reconnecting to GitHub should not cause ECS to restart every reconciler in a group. +- CloudWatch encrypts logs at rest with an AWS-owned key by default. Set `logging.kms_key_arn` for a customer-managed key and ensure its key policy allows the regional CloudWatch Logs service. + +## Plan-shape requirements + +The following values control `for_each`, dynamic IAM statements, or resource ownership and must be known during planning: + +- runner-config map keys; +- compute-contract map keys and provider `type`; +- grouping strategy, custom group keys, and membership; +- IAM statement keys and optional KMS/ECR wrapper presence; +- optional ECS ephemeral-storage wrapper presence; +- managed versus external cluster mode. + +Inner values such as scale-set IDs, SSM/KMS ARNs, provider configuration values, IAM actions/resources, and an external cluster ARN may be computed. Nullable computed values should be placed inside a plan-known wrapper rather than used as the wrapper itself. + + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.4.0 | +| [aws](#requirement\_aws) | >= 6.33 | + +## Providers + +| Name | Version | +|------|---------| +| [aws](#provider\_aws) | >= 6.33 | +| [terraform](#provider\_terraform) | n/a | + +## Modules + +No modules. + +## Resources + +| Name | Type | +|------|------| +| [aws_cloudwatch_log_group.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource | +| [aws_ecs_cluster.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster) | resource | +| [aws_ecs_service.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) | resource | +| [aws_ecs_task_definition.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) | resource | +| [aws_iam_role.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [aws_iam_role.task](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [aws_iam_role_policy.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_iam_role_policy.task](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_security_group.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource | +| [aws_ssm_parameter.reconciler_config](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | +| [terraform_data.validate_config_store](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_contract](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_group_task_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_grouping](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_runtime](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | +| [aws_iam_policy_document.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_iam_policy_document.task](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_iam_policy_document.task_assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source | +| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [compute\_provider\_contracts](#input\_compute\_provider\_contracts) | Provider-neutral, scale-set capability fragments keyed exactly like `runner_configs`.

`type` is the plan-known provider discriminator used by the default grouping implementation and the runtime adapter registry. `configuration_json` is provider-owned, valid JSON and must contain no secrets. `environment_variables` contains non-secret provider process settings shared by every reconciler in the same controller group; conflicting values are rejected. IAM statement map keys and optional condition shapes must be known during planning; their action, resource, and condition values may be computed. | 
map(object({
    type = string
    capabilities = object({
      scale_set = object({
        configuration_json    = optional(string, "{}")
        environment_variables = optional(map(string), {})
        iam_statements = optional(map(object({
          actions   = set(string)
          resources = set(string)
          conditions = optional(list(object({
            test     = string
            variable = string
            values   = set(string)
          })), [])
        })), {})
      })
    })
  }))
| n/a | yes | +| [config\_store](#input\_config\_store) | Non-secret controller configuration storage. The module writes one SSM String parameter per reconciler below `path_prefix//`. The task receives only its group path and a SHA-256 revision, then loads the group with `GetParametersByPath`.

Standard parameters are limited to 4096 encoded bytes and Advanced parameters to 8192 encoded bytes. Null `path_prefix` resolves to `//scale-set-controller`. | 
object({
    path_prefix = optional(string, null)
    tier        = optional(string, "Standard")
    tags        = optional(map(string), {})
  })
| `{}` | no | +| [container](#input\_container) | Scale-set controller image and runtime settings. A null image uses the internal official convenience image; production callers should use the release digest. Filesystem and Linux capability hardening are enforced by the module; health\_path is fixed at /healthz, the ECS liveness endpoint. | 
object({
    image                             = optional(string, null)
    user                              = optional(string, "10001:10001")
    health_port                       = optional(number, 8080)
    health_path                       = optional(string, "/healthz")
    health_check_command              = optional(list(string), null)
    health_check_interval             = optional(number, 30)
    health_check_timeout              = optional(number, 5)
    health_check_retries              = optional(number, 3)
    health_check_start_period         = optional(number, 30)
    health_stale_after_seconds        = optional(number, 180)
    shutdown_timeout_seconds          = optional(number, 110)
    session_close_timeout_seconds     = optional(number, 10)
    reconnect_initial_backoff_seconds = optional(number, 1)
    reconnect_max_backoff_seconds     = optional(number, 30)
    stop_timeout_seconds              = optional(number, 120)
    ecr_repository = optional(object({
      arn = string
    }), null)
  })
| `{}` | no | +| [ecs](#input\_ecs) | ECS substrate configuration. A managed cluster is created by default. For an external cluster, set `cluster.mode = "external"` and pass its ARN; the mode must be plan-known while the ARN may be computed. | 
object({
    cluster = optional(object({
      mode               = optional(string, "managed")
      arn                = optional(string, null)
      name               = optional(string, null)
      container_insights = optional(bool, true)
    }), {})
    task = optional(object({
      cpu              = optional(number, 512)
      memory           = optional(number, 1024)
      cpu_architecture = optional(string, "X86_64")
      ephemeral_storage = optional(object({
        size_in_gib = number
      }), null)
    }), {})
    service = optional(object({
      platform_version = optional(string, "LATEST")
    }), {})
    iam = optional(object({
      path                 = optional(string, "/")
      permissions_boundary = optional(string, null)
    }), {})
  })
| `{}` | no | +| [grouping](#input\_grouping) | Packing strategy for scale-set reconcilers. `compute_provider` creates one controller group per compute-provider type and is the default. `runner_config` creates one group per runner config. `custom` uses `custom.groups`; custom membership must cover every runner config exactly once.

The strategy, custom group keys, and memberships select Terraform `for_each` instances and must be known during planning. | 
object({
    strategy = optional(string, "compute_provider")
    custom = optional(object({
      groups = map(object({
        runner_configs = set(string)
      }))
    }), null)
  })
| `{}` | no | +| [logging](#input\_logging) | CloudWatch Logs configuration. CloudWatch encrypts logs at rest with an AWS-owned key by default; set `kms_key_arn` to use a customer-managed key. | 
object({
    retention_in_days = optional(number, 30)
    kms_key_arn       = optional(string, null)
    log_group_class   = optional(string, "STANDARD")
    tags              = optional(map(string), {})
  })
| `{}` | no | +| [network](#input\_network) | Private Fargate networking. Tasks never receive public IP addresses and the managed security groups have no ingress. HTTPS egress defaults to IPv4 Internet access because GitHub endpoints cannot be represented as security-group destinations; route it through controlled NAT, firewall, or proxy infrastructure when required. | 
object({
    vpc_id     = string
    subnet_ids = set(string)
    https_egress = optional(object({
      ipv4_cidrs = optional(set(string), ["0.0.0.0/0"])
      ipv6_cidrs = optional(set(string), [])
    }), {})
  })
| n/a | yes | +| [prefix](#input\_prefix) | Stable prefix used for scale-set controller resources. | `string` | `"github-actions"` | no | +| [runner\_configs](#input\_runner\_configs) | Normalized scale-set runner configurations keyed by stable runner-config name.

Map keys must be known during planning. Credential values are never accepted: `github.app` contains only the exact GitHub App Parameter Store references used by the runtime. `github.ssl_verify` applies TLS verification per reconciler without changing process-global TLS behavior. Parameter and optional KMS ARNs, scale-set IDs, and other inner values may remain unknown until apply. | 
map(object({
    github = object({
      config_url = string
      app = object({
        app_id = object({
          name        = string
          arn         = string
          kms_key_arn = optional(string, null)
        })
        private_key = object({
          name        = string
          arn         = string
          kms_key_arn = optional(string, null)
        })
        installation_id = object({
          name        = string
          arn         = string
          kms_key_arn = optional(string, null)
        })
      })
      force_ghes = optional(bool, null)
      ssl_verify = optional(bool, true)
      user_agent = optional(string, null)
    })
    scale_set = object({
      name                 = string
      id                   = number
      runner_group_id      = optional(number, null)
      min_runners          = optional(number, 0)
      max_runners          = optional(number, 10)
      boot_time_in_minutes = optional(number, 10)
      session_owner        = optional(string, null)
    })
    work_folder = optional(string, null)
  }))
| n/a | yes | +| [tags](#input\_tags) | Tags applied to scale-set orchestration resources. | `map(string)` | `{}` | no | + +## Outputs + +| Name | Description | +|------|-------------| +| [cluster](#output\_cluster) | Managed or external ECS cluster selected for all controller groups. | +| [controller\_groups](#output\_controller\_groups) | Controller-group resources keyed by stable resolved group name. | +| [reconciler\_config\_parameters](#output\_reconciler\_config\_parameters) | Non-secret SSM controller configuration parameters keyed by `/`. Values are intentionally not exposed. | +| [resolved\_container\_image](#output\_resolved\_container\_image) | Container image reference selected for the controller task definitions. | + diff --git a/modules/orchestration-providers/scale-set/cluster.tf b/modules/orchestration-providers/scale-set/cluster.tf new file mode 100644 index 0000000000..6c9a586da8 --- /dev/null +++ b/modules/orchestration-providers/scale-set/cluster.tf @@ -0,0 +1,14 @@ +resource "aws_ecs_cluster" "controller" { + count = var.ecs.cluster.mode == "managed" ? 1 : 0 + + name = coalesce(var.ecs.cluster.name, "${var.prefix}-scale-set") + + setting { + name = "containerInsights" + value = var.ecs.cluster.container_insights ? "enabled" : "disabled" + } + + tags = local.common_tags + + depends_on = [terraform_data.validate_runtime] +} diff --git a/modules/orchestration-providers/scale-set/config-store.tf b/modules/orchestration-providers/scale-set/config-store.tf new file mode 100644 index 0000000000..86cb42a73a --- /dev/null +++ b/modules/orchestration-providers/scale-set/config-store.tf @@ -0,0 +1,27 @@ +resource "aws_ssm_parameter" "reconciler_config" { + for_each = local.reconciler_configs + + name = "${local.config_store_path_prefix}/${each.value.group_name}/${each.value.runner_name}" + description = "Non-secret scale-set reconciler configuration for ${each.value.runner_name}" + type = "String" + tier = var.config_store.tier + value = local.reconciler_config_json[each.key] + + tags = merge( + local.group_tags[each.value.group_name], + var.config_store.tags, + ) + + lifecycle { + precondition { + condition = local.reconciler_config_bytes[each.key] <= local.config_store_max_bytes + error_message = "The encoded reconciler configuration exceeds the selected Parameter Store tier limit." + } + } + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_config_store, + ] +} diff --git a/modules/orchestration-providers/scale-set/data.tf b/modules/orchestration-providers/scale-set/data.tf new file mode 100644 index 0000000000..99b50de05a --- /dev/null +++ b/modules/orchestration-providers/scale-set/data.tf @@ -0,0 +1,5 @@ +data "aws_caller_identity" "current" {} + +data "aws_partition" "current" {} + +data "aws_region" "current" {} diff --git a/modules/orchestration-providers/scale-set/iam.tf b/modules/orchestration-providers/scale-set/iam.tf new file mode 100644 index 0000000000..1c23772f00 --- /dev/null +++ b/modules/orchestration-providers/scale-set/iam.tf @@ -0,0 +1,163 @@ +data "aws_iam_policy_document" "task_assume_role" { + statement { + sid = "AllowEcsTasks" + effect = "Allow" + actions = ["sts:AssumeRole"] + + principals { + type = "Service" + identifiers = ["ecs-tasks.amazonaws.com"] + } + + condition { + test = "StringEquals" + variable = "aws:SourceAccount" + values = [data.aws_caller_identity.current.account_id] + } + + condition { + test = "ArnLike" + variable = "aws:SourceArn" + values = [format( + "arn:%s:ecs:%s:%s:*", + data.aws_partition.current.partition, + data.aws_region.current.region, + data.aws_caller_identity.current.account_id, + )] + } + } +} + +resource "aws_iam_role" "task" { + for_each = local.controller_groups + + name = "${local.group_resource_names[each.key]}-task" + path = var.ecs.iam.path + permissions_boundary = var.ecs.iam.permissions_boundary + assume_role_policy = data.aws_iam_policy_document.task_assume_role.json + tags = local.group_tags[each.key] + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} + +data "aws_iam_policy_document" "task" { + for_each = local.controller_groups + + source_policy_documents = [local.group_github_kms_policy_json[each.key]] + + statement { + sid = "ReadControllerGroupConfig" + effect = "Allow" + actions = ["ssm:GetParametersByPath"] + resources = [local.group_config_path_arns[each.key]] + } + + statement { + sid = "ReadGitHubAppParameters" + effect = "Allow" + actions = [ + "ssm:GetParameter", + "ssm:GetParameters", + ] + resources = local.group_ssm_parameter_arns[each.key] + } + + dynamic "statement" { + for_each = local.group_compute_iam_statements[each.key] + + content { + effect = "Allow" + actions = statement.value.actions + resources = statement.value.resources + + dynamic "condition" { + for_each = statement.value.conditions + + content { + test = condition.value.test + variable = condition.value.variable + values = condition.value.values + } + } + } + } +} + +resource "aws_iam_role_policy" "task" { + for_each = local.controller_groups + + name = "scale-set-controller" + role = aws_iam_role.task[each.key].name + policy = data.aws_iam_policy_document.task[each.key].json + + depends_on = [terraform_data.validate_group_task_policy] +} + +resource "aws_iam_role" "execution" { + for_each = local.controller_groups + + name = "${local.group_resource_names[each.key]}-exec" + path = var.ecs.iam.path + permissions_boundary = var.ecs.iam.permissions_boundary + assume_role_policy = data.aws_iam_policy_document.task_assume_role.json + tags = local.group_tags[each.key] + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} + +data "aws_iam_policy_document" "execution" { + for_each = local.controller_groups + + statement { + sid = "WriteControllerLogs" + effect = "Allow" + actions = [ + "logs:CreateLogStream", + "logs:PutLogEvents", + ] + resources = ["${aws_cloudwatch_log_group.controller[each.key].arn}:*"] + } + + dynamic "statement" { + for_each = var.container.ecr_repository == null ? [] : [var.container.ecr_repository] + + content { + sid = "PullPrivateEcrImage" + effect = "Allow" + actions = [ + "ecr:BatchCheckLayerAvailability", + "ecr:BatchGetImage", + "ecr:GetDownloadUrlForLayer", + ] + resources = [statement.value.arn] + } + } + + dynamic "statement" { + for_each = var.container.ecr_repository == null ? [] : [1] + + content { + # ECR does not support resource-level permissions for authorization tokens. + sid = "AuthorizePrivateEcrPull" + effect = "Allow" + actions = ["ecr:GetAuthorizationToken"] + resources = ["*"] + } + } +} + +resource "aws_iam_role_policy" "execution" { + for_each = local.controller_groups + + name = "scale-set-controller-execution" + role = aws_iam_role.execution[each.key].name + policy = data.aws_iam_policy_document.execution[each.key].json +} diff --git a/modules/orchestration-providers/scale-set/locals.tf b/modules/orchestration-providers/scale-set/locals.tf new file mode 100644 index 0000000000..cd74e967a9 --- /dev/null +++ b/modules/orchestration-providers/scale-set/locals.tf @@ -0,0 +1,268 @@ +locals { + configured_runner_names = toset(keys(var.runner_configs)) + contract_runner_names = toset(keys(var.compute_provider_contracts)) + routable_runner_names = sort(tolist(setintersection(local.configured_runner_names, local.contract_runner_names))) + + normalized_github_config_urls = { + for runner_name, runner_config in var.runner_configs : runner_name => replace( + trimsuffix(lower(runner_config.github.config_url), "/"), + ":443/", + "/", + ) + } + github_config_url_ports = { + for runner_name, runner_config in var.runner_configs : runner_name => try( + tonumber(regex("^https://[A-Za-z0-9.-]+:([0-9]+)/", runner_config.github.config_url)[0]), + 443, + ) + } + scale_set_ownership_keys = [ + for runner_name, runner_config in var.runner_configs : + "${local.normalized_github_config_urls[runner_name]}#${runner_config.scale_set.id}" + ] + + declared_custom_groups = var.grouping.strategy == "custom" && var.grouping.custom != null ? { + for group_name, group in var.grouping.custom.groups : group_name => sort(tolist(group.runner_configs)) + } : {} + + custom_members = flatten(values(local.declared_custom_groups)) + + compute_provider_types = distinct([ + for runner_name in local.routable_runner_names : var.compute_provider_contracts[runner_name].type + ]) + + compute_provider_groups = { + for provider_type in local.compute_provider_types : provider_type => [ + for runner_name in local.routable_runner_names : runner_name + if var.compute_provider_contracts[runner_name].type == provider_type + ] + } + + runner_config_groups = { + for runner_name in local.routable_runner_names : runner_name => [runner_name] + } + + custom_groups = { + for group_name, runner_names in local.declared_custom_groups : group_name => [ + for runner_name in runner_names : runner_name + if contains(local.routable_runner_names, runner_name) + ] + } + + controller_groups = ( + var.grouping.strategy == "compute_provider" ? local.compute_provider_groups : + var.grouping.strategy == "runner_config" ? local.runner_config_groups : + var.grouping.strategy == "custom" ? local.custom_groups : + {} + ) + + group_resource_names = { + for group_name in keys(local.controller_groups) : group_name => format( + "%s-ss-%s-%s", + var.prefix, + substr(replace(lower(group_name), "/[^a-z0-9_-]/", "-"), 0, 20), + substr(sha256(group_name), 0, 8), + ) + } + + official_container_image = "ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service:latest" + resolved_container_image = coalesce(var.container.image, local.official_container_image) + + resolved_health_check_command = var.container.health_check_command != null ? var.container.health_check_command : [ + "CMD", + "node", + "-e", + "fetch('http://127.0.0.1:${var.container.health_port}${var.container.health_path}').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))", + ] + + reconciler_configs = merge([ + for group_name, runner_names in local.controller_groups : { + for runner_name in runner_names : "${group_name}/${runner_name}" => { + group_name = group_name + runner_name = runner_name + value = merge({ + schemaVersion = 1 + runnerConfigName = runner_name + githubConfigUrl = var.runner_configs[runner_name].github.config_url + scaleSetId = var.runner_configs[runner_name].scale_set.id + expectedScaleSetName = var.runner_configs[runner_name].scale_set.name + expectedRunnerGroupId = var.runner_configs[runner_name].scale_set.runner_group_id + minRunners = var.runner_configs[runner_name].scale_set.min_runners + maxRunners = var.runner_configs[runner_name].scale_set.max_runners + bootTimeoutMinutes = var.runner_configs[runner_name].scale_set.boot_time_in_minutes + sslVerify = var.runner_configs[runner_name].github.ssl_verify + sessionOwner = ( + var.runner_configs[runner_name].scale_set.session_owner != null + ? var.runner_configs[runner_name].scale_set.session_owner + : length("${group_name}.${runner_name}") <= 256 + ? "${group_name}.${runner_name}" + : "${substr(group_name, 0, 119)}.${substr(runner_name, 0, 119)}.${substr(sha256(format("%s.%s", group_name, runner_name)), 0, 16)}" + ) + githubApp = { + appIdParameterName = var.runner_configs[runner_name].github.app.app_id.name + privateKeyParameterName = var.runner_configs[runner_name].github.app.private_key.name + installationIdParameterName = var.runner_configs[runner_name].github.app.installation_id.name + } + computeProvider = { + type = var.compute_provider_contracts[runner_name].type + configuration = jsondecode(var.compute_provider_contracts[runner_name].capabilities.scale_set.configuration_json) + } + }, var.runner_configs[runner_name].work_folder == null ? {} : { + workFolder = var.runner_configs[runner_name].work_folder + }, var.runner_configs[runner_name].github.force_ghes == null ? {} : { + forceGhes = var.runner_configs[runner_name].github.force_ghes + }, var.runner_configs[runner_name].github.user_agent == null ? {} : { + userAgent = var.runner_configs[runner_name].github.user_agent + }) + } + } + ]...) + + config_store_path_prefix = coalesce(var.config_store.path_prefix, "/${var.prefix}/scale-set-controller") + group_config_paths = { + for group_name in keys(local.controller_groups) : group_name => "${local.config_store_path_prefix}/${group_name}" + } + group_config_revisions = { + for group_name, runner_names in local.controller_groups : group_name => sha256(jsonencode({ + for runner_name in runner_names : runner_name => local.reconciler_configs["${group_name}/${runner_name}"].value + })) + } + + group_ssm_parameter_arns = { + for group_name, runner_names in local.controller_groups : group_name => flatten([ + for runner_name in runner_names : [ + var.runner_configs[runner_name].github.app.app_id.arn, + var.runner_configs[runner_name].github.app.private_key.arn, + var.runner_configs[runner_name].github.app.installation_id.arn, + ] + ]) + } + + group_ssm_kms_key_arns = { + for group_name, runner_names in local.controller_groups : group_name => flatten([ + for runner_name in runner_names : [ + for parameter in [ + var.runner_configs[runner_name].github.app.app_id, + var.runner_configs[runner_name].github.app.private_key, + var.runner_configs[runner_name].github.app.installation_id, + ] : parameter.kms_key_arn + ] + ]) + } + + group_github_kms_policy_json = { + for group_name, kms_key_arns in local.group_ssm_kms_key_arns : group_name => jsonencode({ + Version = "2012-10-17" + Statement = length(compact(kms_key_arns)) == 0 ? [] : [{ + Sid = "DecryptGitHubAppParameters" + Effect = "Allow" + Action = ["kms:Decrypt"] + Resource = distinct(compact(kms_key_arns)) + }] + }) + } + + group_compute_iam_statements = { + for group_name, runner_names in local.controller_groups : group_name => merge([ + for runner_name in runner_names : { + for statement_name, statement in var.compute_provider_contracts[runner_name].capabilities.scale_set.iam_statements : + "${runner_name}/${statement_name}" => statement + } + ]...) + } + + group_compute_environment_entries = { + for group_name, runner_names in local.controller_groups : group_name => flatten([ + for runner_name in runner_names : [ + for name, value in var.compute_provider_contracts[runner_name].capabilities.scale_set.environment_variables : { + runner_name = runner_name + name = name + value = value + } + ] + ]) + } + + group_compute_environment_variables = { + for group_name, entries in local.group_compute_environment_entries : group_name => merge([ + for entry in entries : { (entry.name) = entry.value } + ]...) + } + + reserved_environment_variable_names = toset([ + "PATH", + "HOME", + "HOSTNAME", + "PWD", + "SHLVL", + ]) + + config_store_max_bytes = var.config_store.tier == "Advanced" ? 8192 : 4096 + + reconciler_config_json = { + for config_key, config in local.reconciler_configs : config_key => jsonencode(config.value) + } + reconciler_config_base64 = { + for config_key, config_json in local.reconciler_config_json : config_key => base64encode(config_json) + } + reconciler_config_bytes = { + for config_key, encoded in local.reconciler_config_base64 : config_key => ( + floor(length(encoded) * 3 / 4) - + (endswith(encoded, "==") ? 2 : endswith(encoded, "=") ? 1 : 0) + ) + } + group_reconciler_config_bytes = { + for group_name, runner_names in local.controller_groups : group_name => sum([ + for runner_name in runner_names : local.reconciler_config_bytes["${group_name}/${runner_name}"] + ]) + } + + group_task_policy_base64 = { + for group_name, policy in data.aws_iam_policy_document.task : group_name => base64encode(policy.json) + } + group_task_policy_bytes = { + for group_name, encoded in local.group_task_policy_base64 : group_name => ( + floor(length(encoded) * 3 / 4) - + (endswith(encoded, "==") ? 2 : endswith(encoded, "=") ? 1 : 0) + ) + } + + cluster_arn = var.ecs.cluster.mode == "managed" ? aws_ecs_cluster.controller[0].arn : var.ecs.cluster.arn + + group_config_path_arns = { + for group_name, config_path in local.group_config_paths : group_name => format( + "arn:%s:ssm:%s:%s:parameter%s/*", + data.aws_partition.current.partition, + data.aws_region.current.region, + data.aws_caller_identity.current.account_id, + config_path, + ) + } + + fargate_memory_by_cpu = { + 256 = [512, 1024, 2048] + 512 = [1024, 2048, 3072, 4096] + 1024 = range(2048, 9216, 1024) + 2048 = range(4096, 17408, 1024) + 4096 = range(8192, 31744, 1024) + 8192 = range(16384, 65536, 4096) + 16384 = range(32768, 131072, 8192) + } + + common_tags = merge( + { + "ghr:component" = "scale-set-controller" + }, + var.tags, + ) + + group_tags = { + for group_name, resource_name in local.group_resource_names : group_name => merge( + local.common_tags, + { + Name = resource_name + "ghr:controller-group" = group_name + }, + ) + } +} diff --git a/modules/orchestration-providers/scale-set/logging.tf b/modules/orchestration-providers/scale-set/logging.tf new file mode 100644 index 0000000000..974106316a --- /dev/null +++ b/modules/orchestration-providers/scale-set/logging.tf @@ -0,0 +1,19 @@ +resource "aws_cloudwatch_log_group" "controller" { + for_each = local.controller_groups + + name = "/aws/ecs/${local.group_resource_names[each.key]}" + retention_in_days = var.logging.retention_in_days + kms_key_id = var.logging.kms_key_arn + log_group_class = var.logging.log_group_class + + tags = merge( + local.group_tags[each.key], + var.logging.tags, + ) + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} diff --git a/modules/orchestration-providers/scale-set/networking.tf b/modules/orchestration-providers/scale-set/networking.tf new file mode 100644 index 0000000000..5cd9d11203 --- /dev/null +++ b/modules/orchestration-providers/scale-set/networking.tf @@ -0,0 +1,27 @@ +resource "aws_security_group" "controller" { + for_each = local.controller_groups + + name = local.group_resource_names[each.key] + description = "Private scale-set controller ${each.key}; no ingress and HTTPS-only egress" + vpc_id = var.network.vpc_id + + ingress = [] + + egress { + description = "HTTPS to GitHub and AWS APIs" + from_port = 443 + to_port = 443 + protocol = "tcp" + cidr_blocks = sort(tolist(var.network.https_egress.ipv4_cidrs)) + ipv6_cidr_blocks = sort(tolist(var.network.https_egress.ipv6_cidrs)) + } + + revoke_rules_on_delete = true + tags = local.group_tags[each.key] + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} diff --git a/modules/orchestration-providers/scale-set/outputs.tf b/modules/orchestration-providers/scale-set/outputs.tf new file mode 100644 index 0000000000..fd9d1b8c57 --- /dev/null +++ b/modules/orchestration-providers/scale-set/outputs.tf @@ -0,0 +1,58 @@ +output "cluster" { + description = "Managed or external ECS cluster selected for all controller groups." + value = { + arn = local.cluster_arn + managed = var.ecs.cluster.mode == "managed" + } +} + +output "controller_groups" { + description = "Controller-group resources keyed by stable resolved group name." + value = { + for group_name, runner_names in local.controller_groups : group_name => { + runner_configs = runner_names + config_path = local.group_config_paths[group_name] + config_revision = local.group_config_revisions[group_name] + service = { + id = aws_ecs_service.controller[group_name].id + name = aws_ecs_service.controller[group_name].name + } + task_definition = { + arn = aws_ecs_task_definition.controller[group_name].arn + family = aws_ecs_task_definition.controller[group_name].family + } + task_role = { + arn = aws_iam_role.task[group_name].arn + name = aws_iam_role.task[group_name].name + } + execution_role = { + arn = aws_iam_role.execution[group_name].arn + name = aws_iam_role.execution[group_name].name + } + log_group = { + arn = aws_cloudwatch_log_group.controller[group_name].arn + name = aws_cloudwatch_log_group.controller[group_name].name + } + security_group = { + arn = aws_security_group.controller[group_name].arn + id = aws_security_group.controller[group_name].id + } + } + } +} + +output "reconciler_config_parameters" { + description = "Non-secret SSM controller configuration parameters keyed by `/`. Values are intentionally not exposed." + value = { + for config_key, parameter in aws_ssm_parameter.reconciler_config : config_key => { + arn = parameter.arn + name = parameter.name + tier = parameter.tier + } + } +} + +output "resolved_container_image" { + description = "Container image reference selected for the controller task definitions." + value = local.resolved_container_image +} diff --git a/modules/orchestration-providers/scale-set/service.tf b/modules/orchestration-providers/scale-set/service.tf new file mode 100644 index 0000000000..964849f4e7 --- /dev/null +++ b/modules/orchestration-providers/scale-set/service.tf @@ -0,0 +1,40 @@ +resource "aws_ecs_service" "controller" { + for_each = local.controller_groups + + name = local.group_resource_names[each.key] + cluster = local.cluster_arn + task_definition = aws_ecs_task_definition.controller[each.key].arn + desired_count = 1 + launch_type = "FARGATE" + platform_version = var.ecs.service.platform_version + + scheduling_strategy = "REPLICA" + deployment_minimum_healthy_percent = 0 + deployment_maximum_percent = 100 + enable_ecs_managed_tags = true + enable_execute_command = false + propagate_tags = "SERVICE" + + deployment_circuit_breaker { + enable = true + rollback = true + } + + deployment_controller { + type = "ECS" + } + + network_configuration { + assign_public_ip = false + security_groups = [aws_security_group.controller[each.key].id] + subnets = sort(tolist(var.network.subnet_ids)) + } + + tags = local.group_tags[each.key] + + depends_on = [ + aws_iam_role_policy.execution, + aws_iam_role_policy.task, + aws_ssm_parameter.reconciler_config, + ] +} diff --git a/modules/orchestration-providers/scale-set/task.tf b/modules/orchestration-providers/scale-set/task.tf new file mode 100644 index 0000000000..a4d0b89392 --- /dev/null +++ b/modules/orchestration-providers/scale-set/task.tf @@ -0,0 +1,112 @@ +resource "aws_ecs_task_definition" "controller" { + for_each = local.controller_groups + + family = local.group_resource_names[each.key] + requires_compatibilities = ["FARGATE"] + network_mode = "awsvpc" + cpu = tostring(var.ecs.task.cpu) + memory = tostring(var.ecs.task.memory) + task_role_arn = aws_iam_role.task[each.key].arn + execution_role_arn = aws_iam_role.execution[each.key].arn + + runtime_platform { + cpu_architecture = var.ecs.task.cpu_architecture + operating_system_family = "LINUX" + } + + dynamic "ephemeral_storage" { + for_each = var.ecs.task.ephemeral_storage == null ? [] : [var.ecs.task.ephemeral_storage] + + content { + size_in_gib = ephemeral_storage.value.size_in_gib + } + } + + container_definitions = jsonencode([ + { + name = "scale-set-controller" + image = local.resolved_container_image + essential = true + user = var.container.user + privileged = false + readonlyRootFilesystem = true + stopTimeout = var.container.stop_timeout_seconds + versionConsistency = "enabled" + linuxParameters = { + initProcessEnabled = true + capabilities = { + drop = ["ALL"] + } + } + environment = concat( + [ + { + name = "SCALE_SET_CONTROLLER_GROUP_NAME" + value = each.key + }, + { + name = "SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH" + value = local.group_config_paths[each.key] + }, + { + name = "SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION" + value = local.group_config_revisions[each.key] + }, + { + name = "SCALE_SET_HEALTH_PORT" + value = tostring(var.container.health_port) + }, + { + name = "SCALE_SET_HEALTH_STALE_AFTER_SECONDS" + value = tostring(var.container.health_stale_after_seconds) + }, + { + name = "SCALE_SET_SHUTDOWN_TIMEOUT_SECONDS" + value = tostring(var.container.shutdown_timeout_seconds) + }, + { + name = "SCALE_SET_SESSION_CLOSE_TIMEOUT_SECONDS" + value = tostring(var.container.session_close_timeout_seconds) + }, + { + name = "SCALE_SET_RECONNECT_INITIAL_BACKOFF_SECONDS" + value = tostring(var.container.reconnect_initial_backoff_seconds) + }, + { + name = "SCALE_SET_RECONNECT_MAX_BACKOFF_SECONDS" + value = tostring(var.container.reconnect_max_backoff_seconds) + }, + ], + [ + for name in sort(keys(local.group_compute_environment_variables[each.key])) : { + name = name + value = local.group_compute_environment_variables[each.key][name] + } + ], + ) + healthCheck = { + command = local.resolved_health_check_command + interval = var.container.health_check_interval + timeout = var.container.health_check_timeout + retries = var.container.health_check_retries + startPeriod = var.container.health_check_start_period + } + logConfiguration = { + logDriver = "awslogs" + options = { + "awslogs-group" = aws_cloudwatch_log_group.controller[each.key].name + "awslogs-region" = data.aws_region.current.region + "awslogs-stream-prefix" = "controller" + } + } + } + ]) + + tags = local.group_tags[each.key] + + depends_on = [ + aws_iam_role_policy.execution, + aws_iam_role_policy.task, + aws_ssm_parameter.reconciler_config, + ] +} diff --git a/modules/orchestration-providers/scale-set/tests/computed-inputs.tftest.hcl b/modules/orchestration-providers/scale-set/tests/computed-inputs.tftest.hcl new file mode 100644 index 0000000000..22640ccd7f --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/computed-inputs.tftest.hcl @@ -0,0 +1,49 @@ +mock_provider "aws" { + mock_data "aws_caller_identity" { + defaults = { + account_id = "123456789012" + } + } + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_region" { + defaults = { + region = "eu-west-1" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/computed-test" + } + } +} + +run "plans_with_computed_values_inside_known_wrappers" { + command = plan + + module { + source = "./tests/fixtures/computed-inputs" + } + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["ec2"]) && + toset(output.controller_groups.ec2.runner_configs) == toset(["computed"]) && + !output.cluster.managed && + toset(keys(output.reconciler_config_parameters)) == toset(["ec2/computed"]) + ) + error_message = "Computed inner values and explicit nulls must not affect group, ownership, IAM-wrapper, or cluster resource shape." + } +} diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/README.md b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/README.md new file mode 100644 index 0000000000..8ee616b61b --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/README.md @@ -0,0 +1,38 @@ + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.4.0 | +| [aws](#requirement\_aws) | >= 6.33 | + +## Providers + +| Name | Version | +|------|---------| +| [terraform](#provider\_terraform) | n/a | + +## Modules + +| Name | Source | Version | +|------|--------|---------| +| [subject](#module\_subject) | ../../.. | n/a | + +## Resources + +| Name | Type | +|------|------| +| [terraform_data.computed](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | + +## Inputs + +No inputs. + +## Outputs + +| Name | Description | +|------|-------------| +| [cluster](#output\_cluster) | n/a | +| [controller\_groups](#output\_controller\_groups) | n/a | +| [reconciler\_config\_parameters](#output\_reconciler\_config\_parameters) | n/a | + \ No newline at end of file diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/main.tf b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/main.tf new file mode 100644 index 0000000000..1bbb74f924 --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/main.tf @@ -0,0 +1,104 @@ +resource "terraform_data" "computed" { + input = { + external_cluster_arn = "arn:aws:ecs:eu-west-1:123456789012:cluster/external" + github_config_url = "https://github.com/example" + scale_set_id = 901 + app_id_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/computed/app-id" + private_key_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/computed/private-key" + installation_id_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/computed/installation-id" + kms_key_arn = "arn:aws:kms:eu-west-1:123456789012:key/33333333-3333-3333-3333-333333333333" + launch_template_name = "lt-computed" + action = "ec2:RunInstances" + resource = "arn:aws:ec2:eu-west-1:123456789012:launch-template/lt-computed" + } +} + +module "subject" { + source = "../../.." + + prefix = "computed-test" + + runner_configs = { + computed = { + github = { + config_url = terraform_data.computed.output.github_config_url + app = { + app_id = { + name = "/github/computed/app-id" + arn = terraform_data.computed.output.app_id_arn + } + private_key = { + name = "/github/computed/private-key" + arn = terraform_data.computed.output.private_key_arn + kms_key_arn = terraform_data.computed.output.kms_key_arn + } + installation_id = { + name = "/github/computed/installation-id" + arn = terraform_data.computed.output.installation_id_arn + } + } + } + scale_set = { + id = terraform_data.computed.output.scale_set_id + name = "computed" + runner_group_id = null + } + } + } + + compute_provider_contracts = { + computed = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = jsonencode({ + region = "eu-west-1" + environment = "computed-test" + runnerOwner = "example" + runnerType = "Org" + runnerNamePrefix = "computed-" + jitConfigParameterPath = "/computed-test/runners/tokens" + subnets = ["subnet-12345678"] + launchTemplateName = terraform_data.computed.output.launch_template_name + ec2instanceCriteria = { + instanceTypes = ["m7i.large"] + targetCapacityType = "on-demand" + instanceAllocationStrategy = "lowest-price" + } + scaleErrors = [] + }) + iam_statements = { + run_instances = { + actions = [terraform_data.computed.output.action] + resources = [terraform_data.computed.output.resource] + } + } + } + } + } + } + + ecs = { + cluster = { + mode = "external" + arn = terraform_data.computed.output.external_cluster_arn + } + } + + network = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + } +} + +output "controller_groups" { + value = module.subject.controller_groups +} + +output "cluster" { + value = module.subject.cluster +} + +output "reconciler_config_parameters" { + value = module.subject.reconciler_config_parameters +} diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/versions.tf b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/versions.tf new file mode 100644 index 0000000000..3ef011ea0a --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/versions.tf @@ -0,0 +1,10 @@ +terraform { + required_version = ">= 1.4.0" + + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 6.33" + } + } +} diff --git a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl new file mode 100644 index 0000000000..f90142015f --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl @@ -0,0 +1,981 @@ +mock_provider "aws" { + mock_data "aws_caller_identity" { + defaults = { + account_id = "123456789012" + } + } + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_region" { + defaults = { + region = "eu-west-1" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/scale-set-test" + } + } + + mock_resource "aws_ecs_cluster" { + defaults = { + arn = "arn:aws:ecs:eu-west-1:123456789012:cluster/scale-set-test" + } + } +} + +variables { + prefix = "scale-set-test" + + runner_configs = { + linux-small = { + github = { + config_url = "https://github.com/example" + app = { + app_id = { + name = "/github/linux-small/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/app-id" + } + private_key = { + name = "/github/linux-small/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/private-key" + kms_key_arn = "arn:aws:kms:eu-west-1:123456789012:key/11111111-1111-1111-1111-111111111111" + } + installation_id = { + name = "/github/linux-small/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/installation-id" + } + } + user_agent = "scale-set-test" + ssl_verify = false + } + scale_set = { + id = 101 + name = "linux-small" + runner_group_id = 1 + min_runners = 1 + max_runners = 10 + } + } + linux-large = { + github = { + config_url = "https://github.com/example" + app = { + app_id = { + name = "/github/linux-large/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-large/app-id" + } + private_key = { + name = "/github/linux-large/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-large/private-key" + } + installation_id = { + name = "/github/linux-large/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-large/installation-id" + } + } + } + scale_set = { + id = 102 + name = "linux-large" + min_runners = 0 + max_runners = 20 + } + work_folder = "_work/linux-large" + } + microvm = { + github = { + config_url = "https://github.com/example/repository" + app = { + app_id = { + name = "/github/microvm/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/app-id" + } + private_key = { + name = "/github/microvm/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/private-key" + } + installation_id = { + name = "/github/microvm/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/installation-id" + } + } + force_ghes = false + } + scale_set = { + id = 201 + name = "microvm" + min_runners = 0 + max_runners = 5 + session_owner = "test.microvm" + } + } + } + + compute_provider_contracts = { + linux-small = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = jsonencode({ + region = "eu-west-1" + environment = "scale-set-test" + runnerOwner = "example" + runnerType = "Org" + runnerNamePrefix = "small-" + jitConfigParameterPath = "/scale-set-test/runners/tokens" + subnets = ["subnet-11111111"] + launchTemplateName = "lt-small" + ec2instanceCriteria = { + instanceTypes = ["m7i.large"] + targetCapacityType = "on-demand" + instanceAllocationStrategy = "lowest-price" + } + scaleErrors = [] + }) + environment_variables = { + EC2_CONTROLLER_MODE = "grouped" + } + iam_statements = { + run_instances = { + actions = ["ec2:RunInstances"] + resources = ["arn:aws:ec2:eu-west-1:123456789012:launch-template/lt-small"] + } + } + } + } + } + linux-large = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = jsonencode({ + region = "eu-west-1" + environment = "scale-set-test" + runnerOwner = "example" + runnerType = "Org" + runnerNamePrefix = "large-" + jitConfigParameterPath = "/scale-set-test/runners/tokens" + subnets = ["subnet-22222222"] + launchTemplateName = "lt-large" + ec2instanceCriteria = { + instanceTypes = ["m7i.xlarge"] + targetCapacityType = "on-demand" + instanceAllocationStrategy = "lowest-price" + } + scaleErrors = [] + }) + environment_variables = { + EC2_CONTROLLER_MODE = "grouped" + } + iam_statements = { + run_instances = { + actions = ["ec2:RunInstances"] + resources = ["arn:aws:ec2:eu-west-1:123456789012:launch-template/lt-large"] + } + } + } + } + } + microvm = { + # Future provider used only to prove grouping remains provider-neutral. + type = "microvm" + capabilities = { + scale_set = { + configuration_json = jsonencode({ image_arn = "arn:aws:lambda:eu-west-1:123456789012:runtime-management-config:microvm" }) + iam_statements = { + run_microvm = { + actions = ["lambda:InvokeFunction"] + resources = ["arn:aws:lambda:eu-west-1:123456789012:function:microvm"] + } + } + } + } + } + } + + network = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-11111111", "subnet-22222222"] + } + + logging = { + kms_key_arn = "arn:aws:kms:eu-west-1:123456789012:key/22222222-2222-2222-2222-222222222222" + } + + tags = { + Test = "scale-set" + } +} + +run "groups_by_compute_provider_and_hardens_each_task" { + command = plan + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["ec2", "microvm"]) && + toset(output.controller_groups["ec2"].runner_configs) == toset(["linux-small", "linux-large"]) && + toset(output.controller_groups["microvm"].runner_configs) == toset(["microvm"]) + ) + error_message = "The default strategy must create one controller group per compute-provider type." + } + + assert { + condition = ( + length(aws_ecs_service.controller) == 2 && + length(aws_ecs_task_definition.controller) == 2 && + length(aws_iam_role.task) == 2 && + length(aws_cloudwatch_log_group.controller) == 2 && + length(aws_security_group.controller) == 2 && + length(aws_ssm_parameter.reconciler_config) == 3 + ) + error_message = "Every group must own one service, task definition, task role, log group, and security group while every reconciler gets one config parameter." + } + + assert { + condition = alltrue([ + for service in values(aws_ecs_service.controller) : ( + service.desired_count == 1 && + service.deployment_minimum_healthy_percent == 0 && + service.deployment_maximum_percent == 100 && + service.deployment_circuit_breaker[0].enable && + service.deployment_circuit_breaker[0].rollback && + !service.network_configuration[0].assign_public_ip && + length(service.network_configuration[0].security_groups) == 1 + ) + ]) + error_message = "Services must run one private task and use stop-first deployment with circuit-breaker rollback." + } + + assert { + condition = alltrue([ + for task in values(aws_ecs_task_definition.controller) : ( + length(jsondecode(task.container_definitions)) == 1 && + jsondecode(task.container_definitions)[0].image == "ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service:latest" && + jsondecode(task.container_definitions)[0].versionConsistency == "enabled" && + jsondecode(task.container_definitions)[0].readonlyRootFilesystem && + !jsondecode(task.container_definitions)[0].privileged && + jsondecode(task.container_definitions)[0].user == "10001:10001" && + jsondecode(task.container_definitions)[0].linuxParameters.capabilities.drop == ["ALL"] && + jsondecode(task.container_definitions)[0].healthCheck.command[3] == "fetch('http://127.0.0.1:8080/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))" && + !contains([for entry in jsondecode(task.container_definitions)[0].environment : entry.name], "SCALE_SET_CONTROLLER_MANIFEST") + ) + ]) + error_message = "Each task definition must contain one hardened controller container using group-path configuration and /healthz liveness." + } + + assert { + condition = one([ + for entry in jsondecode(aws_ecs_task_definition.controller["ec2"].container_definitions)[0].environment : + entry.value if entry.name == "EC2_CONTROLLER_MODE" + ]) == "grouped" + error_message = "Provider-owned non-secret environment variables must be merged into their controller group task." + } + + assert { + condition = ( + length(aws_security_group.controller["ec2"].ingress) == 0 && + length(aws_security_group.controller["ec2"].egress) == 1 && + one(aws_security_group.controller["ec2"].egress).from_port == 443 && + one(aws_security_group.controller["ec2"].egress).to_port == 443 && + aws_cloudwatch_log_group.controller["ec2"].kms_key_id == var.logging.kms_key_arn + ) + error_message = "Controller networking must have no ingress and only HTTPS egress, and logs must honor customer-managed encryption." + } + + assert { + condition = ( + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).schemaVersion == 1 && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).runnerConfigName == "linux-small" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).scaleSetId == 101 && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).expectedScaleSetName == "linux-small" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).bootTimeoutMinutes == 10 && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).sslVerify == false && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).githubApp.privateKeyParameterName == "/github/linux-small/private-key" && + !contains(keys(jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value))), "runnerConfig") + ) + error_message = "Each SSM leaf must use the frozen flat reconciler schema and contain references instead of GitHub credential values." + } + + assert { + condition = ( + contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/private-key") && + !contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/private-key") && + contains(jsondecode(local.group_github_kms_policy_json["ec2"]).Statement[0].Resource, "arn:aws:kms:eu-west-1:123456789012:key/11111111-1111-1111-1111-111111111111") && + length(jsondecode(local.group_github_kms_policy_json["microvm"]).Statement) == 0 && + contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set-test/scale-set-controller/ec2/*") + ) + error_message = "Task IAM must be scoped to its group config prefix, credential parameters, KMS keys, and compute resources." + } +} + +run "supports_one_group_per_runner_config" { + command = plan + + variables { + grouping = { + strategy = "runner_config" + } + container = { + image = "ghcr.io/example/scale-set-controller@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + } + } + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["linux-small", "linux-large", "microvm"]) && + length(aws_ecs_service.controller) == 3 && + output.resolved_container_image == "ghcr.io/example/scale-set-controller@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + ) + error_message = "runner_config grouping must create one independently deployable controller task per runner config and honor an image override." + } +} + +run "supports_exact_custom_groups" { + command = plan + + variables { + grouping = { + strategy = "custom" + custom = { + groups = { + general = { + runner_configs = ["linux-small", "microvm"] + } + isolated = { + runner_configs = ["linux-large"] + } + } + } + } + } + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["general", "isolated"]) && + toset(output.controller_groups.general.runner_configs) == toset(["linux-small", "microvm"]) && + toset(output.controller_groups.isolated.runner_configs) == toset(["linux-large"]) + ) + error_message = "Custom grouping must preserve the exact declared assignment." + } +} + +run "rejects_duplicate_custom_membership" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + grouping = { + strategy = "custom" + custom = { + groups = { + first = { + runner_configs = ["linux-small", "linux-large"] + } + second = { + runner_configs = ["linux-small", "microvm"] + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_incomplete_custom_membership" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + grouping = { + strategy = "custom" + custom = { + groups = { + partial = { + runner_configs = ["linux-small", "linux-large"] + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_contract_key_mismatch" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + compute_provider_contracts = { + linux-small = var.compute_provider_contracts.linux-small + linux-large = var.compute_provider_contracts.linux-large + } + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_readiness_path_as_ecs_liveness" { + command = plan + + plan_options { + target = [terraform_data.validate_runtime] + } + + variables { + container = { + health_path = "/readyz" + } + } + + expect_failures = [terraform_data.validate_runtime] +} + +run "rejects_oversized_standard_parameter" { + command = plan + + plan_options { + target = [terraform_data.validate_config_store] + } + + variables { + compute_provider_contracts = merge(var.compute_provider_contracts, { + linux-small = merge(var.compute_provider_contracts.linux-small, { + capabilities = { + scale_set = merge(var.compute_provider_contracts.linux-small.capabilities.scale_set, { + configuration_json = jsonencode({ payload = join("", [for index in range(1000) : "xxxxxx"]) }) + }) + } + }) + }) + } + + expect_failures = [terraform_data.validate_config_store] +} + +run "accepts_advanced_parameter_within_eight_kib" { + command = plan + + plan_options { + target = [terraform_data.validate_config_store] + } + + variables { + config_store = { + tier = "Advanced" + } + compute_provider_contracts = merge(var.compute_provider_contracts, { + linux-small = merge(var.compute_provider_contracts.linux-small, { + capabilities = { + scale_set = merge(var.compute_provider_contracts.linux-small.capabilities.scale_set, { + configuration_json = jsonencode({ payload = join("", [for index in range(800) : "xxxxxx"]) }) + }) + } + }) + }) + } + + assert { + condition = local.reconciler_config_bytes["ec2/linux-small"] > 4096 && local.reconciler_config_bytes["ec2/linux-small"] <= 8192 + error_message = "Advanced Parameter Store tier must accept reconciler JSON between four and eight KiB." + } +} + +run "rejects_duplicate_scale_set_ownership_across_groups" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + config_url = "https://GITHUB.COM:443/example/" + }) + scale_set = merge(var.runner_configs.microvm.scale_set, { + id = 101 + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_leading_zero_default_port_spelling" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + config_url = "https://github.com:0443/example/" + }) + scale_set = merge(var.runner_configs.microvm.scale_set, { + id = 101 + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_port_above_url_maximum" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + config_url = "https://github.com:65536/example" + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_non_ascii_scale_set_name" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + scale_set = merge(var.runner_configs.microvm.scale_set, { + name = "microvm-☃" + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_invalid_compute_provider_type_identifier" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + compute_provider_contracts = merge(var.compute_provider_contracts, { + microvm = merge(var.compute_provider_contracts.microvm, { + type = "AWS.MicroVM" + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_scale_set_id_above_runtime_integer_maximum" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + scale_set = merge(var.runner_configs.microvm.scale_set, { + id = 2147483648 + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_runner_group_id_above_runtime_integer_maximum" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + scale_set = merge(var.runner_configs.microvm.scale_set, { + runner_group_id = 2147483648 + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_credential_arn_name_mismatch" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + app = merge(var.runner_configs.microvm.github.app, { + app_id = merge(var.runner_configs.microvm.github.app.app_id, { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/another/app-id" + }) + }) + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_cross_account_credential_parameter" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + app = merge(var.runner_configs.microvm.github.app, { + app_id = merge(var.runner_configs.microvm.github.app.app_id, { + arn = "arn:aws:ssm:eu-west-1:210987654321:parameter/github/microvm/app-id" + }) + }) + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "allows_same_numeric_scale_set_id_in_another_github_scope" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + scale_set = merge(var.runner_configs.microvm.scale_set, { + id = 101 + }) + }) + }) + } + + assert { + condition = length(local.scale_set_ownership_keys) == length(distinct(local.scale_set_ownership_keys)) + error_message = "Scale-set IDs are scoped to their normalized GitHub configuration URL." + } +} + +run "bounds_default_session_owner_for_maximum_names" { + command = plan + + variables { + grouping = { + strategy = "runner_config" + } + runner_configs = { + (join("", [for index in range(128) : "a"])) = { + github = { + config_url = "https://github.com/example" + app = { + app_id = { + name = "/github/max/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/max/app-id" + } + private_key = { + name = "/github/max/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/max/private-key" + } + installation_id = { + name = "/github/max/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/max/installation-id" + } + } + } + scale_set = { + id = 301 + name = "maximum-name" + } + } + } + compute_provider_contracts = { + (join("", [for index in range(128) : "a"])) = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = "{}" + } + } + } + } + } + + assert { + condition = ( + length(one(values(local.reconciler_configs)).value.sessionOwner) == 256 && + can(regex("^[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}$", one(values(local.reconciler_configs)).value.sessionOwner)) + ) + error_message = "A generated session owner must remain deterministic and within the runtime's 256-character limit." + } +} + +run "rejects_controller_group_policy_above_inline_quota" { + command = plan + + plan_options { + target = [terraform_data.validate_group_task_policy["ec2"]] + } + + override_data { + target = data.aws_iam_policy_document.task["ec2"] + values = { + json = <<-JSON + {"payload":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"} + JSON + } + } + + expect_failures = [terraform_data.validate_group_task_policy["ec2"]] +} + +run "rejects_conflicting_group_environment_variables" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + compute_provider_contracts = merge(var.compute_provider_contracts, { + linux-large = merge(var.compute_provider_contracts.linux-large, { + capabilities = { + scale_set = merge(var.compute_provider_contracts.linux-large.capabilities.scale_set, { + environment_variables = { + EC2_CONTROLLER_MODE = "isolated" + } + }) + } + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_controller_group_environment_above_task_definition_budget" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + compute_provider_contracts = merge(var.compute_provider_contracts, { + linux-small = merge(var.compute_provider_contracts.linux-small, { + capabilities = { + scale_set = merge(var.compute_provider_contracts.linux-small.capabilities.scale_set, { + environment_variables = merge( + var.compute_provider_contracts.linux-small.capabilities.scale_set.environment_variables, + { + for index in range(16) : format("EC2_QUOTA_%02d", index) => join("", [for part in range(1024) : "xxxx"]) + }, + ) + }) + } + }) + }) + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_reserved_provider_environment_variables" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + compute_provider_contracts = merge(var.compute_provider_contracts, { + linux-small = merge(var.compute_provider_contracts.linux-small, { + capabilities = { + scale_set = merge(var.compute_provider_contracts.linux-small.capabilities.scale_set, { + environment_variables = { + SCALE_SET_OVERRIDE = "unsafe" + } + }) + } + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_invalid_boot_timeout" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + scale_set = merge(var.runner_configs.linux-small.scale_set, { + boot_time_in_minutes = 0 + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_controller_group_above_runtime_reconciler_limit" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + runner_configs = { + for index in range(1001) : format("runner-%04d", index) => var.runner_configs.linux-small + } + compute_provider_contracts = { + for index in range(1001) : format("runner-%04d", index) => var.compute_provider_contracts.linux-small + } + grouping = { + strategy = "custom" + custom = { + groups = { + oversized = { + runner_configs = toset([for index in range(1001) : format("runner-%04d", index)]) + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_controller_group_above_runtime_config_bytes" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + config_store = { + tier = "Advanced" + } + runner_configs = { + for index in range(900) : format("runner-%04d", index) => var.runner_configs.linux-small + } + compute_provider_contracts = { + for index in range(900) : format("runner-%04d", index) => merge(var.compute_provider_contracts.linux-small, { + capabilities = { + scale_set = merge(var.compute_provider_contracts.linux-small.capabilities.scale_set, { + configuration_json = jsonencode({ + payload = join("", [for part in range(1000) : "xxxxx"]) + }) + }) + } + }) + } + grouping = { + strategy = "custom" + custom = { + groups = { + oversized = { + runner_configs = toset([for index in range(900) : format("runner-%04d", index)]) + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_runtime_invalid_credential_parameter_name" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + app = merge(var.runner_configs.microvm.github.app, { + app_id = { + name = "/github/microvm/bad app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/bad app-id" + } + }) + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} diff --git a/modules/orchestration-providers/scale-set/validations.tf b/modules/orchestration-providers/scale-set/validations.tf new file mode 100644 index 0000000000..0d29a31e5d --- /dev/null +++ b/modules/orchestration-providers/scale-set/validations.tf @@ -0,0 +1,387 @@ +locals { + group_compute_environment_base64 = { + for group_name, environment_variables in local.group_compute_environment_variables : group_name => base64encode(jsonencode([ + for name in sort(keys(environment_variables)) : { + name = name + value = environment_variables[name] + } + ])) + } + group_compute_environment_bytes = { + for group_name, encoded in local.group_compute_environment_base64 : group_name => ( + floor(length(encoded) * 3 / 4) - + (endswith(encoded, "==") ? 2 : endswith(encoded, "=") ? 1 : 0) + ) + } +} + +resource "terraform_data" "validate_contract" { + lifecycle { + precondition { + condition = ( + length(var.prefix) >= 1 && + length(var.prefix) <= 20 && + can(regex("^[a-z0-9][a-z0-9-]*$", var.prefix)) + ) + error_message = "prefix must contain 1 to 20 lowercase ASCII letters, digits, or hyphens and start with a letter or digit." + } + + precondition { + condition = ( + length(setsubtract(local.configured_runner_names, local.contract_runner_names)) == 0 && + length(setsubtract(local.contract_runner_names, local.configured_runner_names)) == 0 + ) + error_message = "runner_configs and compute_provider_contracts must have exactly the same keys." + } + + precondition { + condition = alltrue([ + for runner_name in keys(var.runner_configs) : ( + length(runner_name) >= 1 && + length(runner_name) <= 128 && + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]*$", runner_name)) + ) + ]) + error_message = "runner-config keys must contain 1 to 128 ASCII letters, digits, dots, underscores, or hyphens and start with a letter or digit." + } + + precondition { + condition = alltrue([ + for runner_name, runner_config in var.runner_configs : ( + can(regex("^https://[A-Za-z0-9.-]+(:[1-9][0-9]{0,4})?/[A-Za-z0-9_.-]+(/[A-Za-z0-9_.-]+)?/?$", runner_config.github.config_url)) && + local.github_config_url_ports[runner_name] <= 65535 + ) + ]) + error_message = "Each github.config_url must be an HTTPS GitHub organization, repository, or enterprise URL without credentials, query, fragment, or whitespace." + } + + precondition { + condition = length(local.scale_set_ownership_keys) == length(distinct(local.scale_set_ownership_keys)) + error_message = "Each normalized github.config_url and scale_set.id tuple must be unique across runner_configs so two controller services cannot own the same message session. URL matching ignores case, one trailing slash, and the default HTTPS port." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.runner_configs) : ( + alltrue([ + for parameter in [ + runner_config.github.app.app_id, + runner_config.github.app.private_key, + runner_config.github.app.installation_id, + ] : ( + length(parameter.name) <= 2048 && + can(regex("^/[A-Za-z0-9_./-]+$", parameter.name)) && + !endswith(parameter.name, "/") && + !strcontains(parameter.name, "//") && + parameter.arn == format( + "arn:%s:ssm:%s:%s:parameter%s", + data.aws_partition.current.partition, + data.aws_region.current.region, + data.aws_caller_identity.current.account_id, + parameter.name, + ) && + (parameter.kms_key_arn == null ? true : can(regex("^arn:[^:]+:kms:[^:]+:[0-9]{12}:key/.+$", parameter.kms_key_arn))) + ) + ]) + ) + ]) + error_message = "GitHub App credentials must use valid absolute SSM parameter names and exact same-account, same-region parameter ARNs; optional KMS references must be key ARNs." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.runner_configs) : ( + can(regex("^[ -~]{1,128}$", runner_config.scale_set.name)) && + runner_config.scale_set.id >= 1 && + runner_config.scale_set.id <= 2147483647 && + floor(runner_config.scale_set.id) == runner_config.scale_set.id && + (runner_config.scale_set.runner_group_id == null ? true : ( + runner_config.scale_set.runner_group_id >= 1 && + runner_config.scale_set.runner_group_id <= 2147483647 && + floor(runner_config.scale_set.runner_group_id) == runner_config.scale_set.runner_group_id + )) && + runner_config.scale_set.min_runners >= 0 && + floor(runner_config.scale_set.min_runners) == runner_config.scale_set.min_runners && + runner_config.scale_set.max_runners >= 1 && + runner_config.scale_set.max_runners <= 10000 && + floor(runner_config.scale_set.max_runners) == runner_config.scale_set.max_runners && + runner_config.scale_set.min_runners <= runner_config.scale_set.max_runners && + runner_config.scale_set.boot_time_in_minutes >= 1 && + runner_config.scale_set.boot_time_in_minutes <= 120 && + floor(runner_config.scale_set.boot_time_in_minutes) == runner_config.scale_set.boot_time_in_minutes && + (runner_config.scale_set.session_owner == null ? true : can(regex("^[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}$", runner_config.scale_set.session_owner))) && + (runner_config.work_folder == null ? true : ( + length(runner_config.work_folder) <= 128 && + !startswith(runner_config.work_folder, "/") && + !strcontains(runner_config.work_folder, "\\") && + can(regex("^[A-Za-z0-9._/-]+$", runner_config.work_folder)) && + alltrue([for part in split("/", runner_config.work_folder) : !contains(["", ".", ".."], part)]) + )) && + (runner_config.github.user_agent == null ? true : ( + length(runner_config.github.user_agent) <= 256 && + can(regex("^[ -~]+$", runner_config.github.user_agent)) + )) + ) + ]) + error_message = "Scale-set names and IDs must be valid, boot_time_in_minutes must be an integer from 1 through 120, optional session/work-folder/user-agent values must match runtime constraints, and min_runners must be between zero and max_runners (maximum 10000)." + } + + precondition { + condition = alltrue([ + for contract in values(var.compute_provider_contracts) : ( + can(regex("^[a-z][a-z0-9_-]{0,63}$", contract.type)) && + can(keys(jsondecode(contract.capabilities.scale_set.configuration_json))) && + length(contract.capabilities.scale_set.environment_variables) <= 64 && + alltrue([ + for name, value in contract.capabilities.scale_set.environment_variables : ( + can(regex("^[A-Z][A-Z0-9_]{0,127}$", name)) && + !contains(local.reserved_environment_variable_names, name) && + alltrue([ + for prefix in ["AWS_", "ECS_", "GITHUB_", "SCALE_SET_", "NODE_"] : + !startswith(name, prefix) + ]) && + length(regexall("[\\x00-\\x1F\\x7F]", value)) == 0 && + ( + floor(length(base64encode(value)) * 3 / 4) - + (endswith(base64encode(value), "==") ? 2 : endswith(base64encode(value), "=") ? 1 : 0) + ) <= 4096 + ) + ]) && + alltrue([ + for statement_name, statement in contract.capabilities.scale_set.iam_statements : ( + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$", statement_name)) && + length(statement.actions) > 0 && + length(statement.resources) > 0 && + alltrue([for action in statement.actions : !strcontains(action, "*")]) && + alltrue([ + for condition in statement.conditions : ( + length(condition.test) > 0 && + length(condition.variable) > 0 && + length(condition.values) > 0 + ) + ]) + ) + ]) + ) + ]) + error_message = "Each compute-provider scale-set capability must have safe identifiers, object-shaped configuration JSON, non-secret environment variables with safe unreserved names and bounded values, and non-empty least-privilege IAM statements without wildcard actions." + } + + precondition { + condition = alltrue([ + for group_name, entries in local.group_compute_environment_entries : alltrue([ + for name in distinct([for entry in entries : entry.name]) : + length(distinct([for entry in entries : entry.value if entry.name == name])) <= 1 + ]) + ]) + error_message = "Compute-provider environment variables grouped into the same controller task must use identical values for duplicate names. Use a different grouping strategy when providers require conflicting process settings." + } + } +} + +resource "terraform_data" "validate_grouping" { + lifecycle { + precondition { + condition = contains(["compute_provider", "runner_config", "custom"], var.grouping.strategy) + error_message = "grouping.strategy must be compute_provider, runner_config, or custom." + } + + precondition { + condition = ( + var.grouping.strategy == "custom" + ? var.grouping.custom != null && length(var.grouping.custom.groups) > 0 + : var.grouping.custom == null + ) + error_message = "grouping.custom must be non-null and non-empty only when grouping.strategy is custom." + } + + precondition { + condition = var.grouping.strategy != "custom" ? true : alltrue([ + for group_name, group in var.grouping.custom.groups : ( + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$", group_name)) && + length(group.runner_configs) > 0 + ) + ]) + error_message = "Custom group names must be stable, safe identifiers of at most 64 characters, and every group must contain at least one runner config." + } + + precondition { + condition = var.grouping.strategy != "custom" ? true : ( + length(local.custom_members) == length(distinct(local.custom_members)) && + length(setsubtract(toset(local.custom_members), local.configured_runner_names)) == 0 && + length(setsubtract(local.configured_runner_names, toset(local.custom_members))) == 0 + ) + error_message = "Custom groups must contain every runner config exactly once and cannot contain unknown runner configs." + } + + precondition { + condition = alltrue([ + for group_name, runner_names in local.controller_groups : ( + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$", group_name)) && + length(runner_names) > 0 && + length(runner_names) <= 1000 + ) + ]) + error_message = "Resolved controller groups must have safe, non-empty, plan-known names and contain at most 1000 runner configs." + } + + precondition { + condition = alltrue([ + for group_name, decoded_bytes in local.group_reconciler_config_bytes : decoded_bytes <= 4 * 1024 * 1024 + ]) + error_message = "A controller group's decoded reconciler configuration must not exceed the runtime's 4 MiB aggregate limit. Split the group or reduce provider configuration size." + } + + precondition { + condition = alltrue([ + for group_name, environment_bytes in local.group_compute_environment_bytes : environment_bytes <= 48 * 1024 + ]) + error_message = "A controller group's compute-provider environment JSON must not exceed 49152 bytes. This reserves 16 KiB of AWS's 64 KiB ECS task-definition quota for the fixed task definition; split the group or reduce provider environment settings." + } + } +} + +resource "terraform_data" "validate_runtime" { + lifecycle { + precondition { + condition = ( + var.container.image == null ? true : ( + length(trimspace(var.container.image)) > 0 && + length(regexall("[[:space:]]", var.container.image)) == 0 + )) + error_message = "Container image references must be non-empty and cannot contain whitespace." + } + + precondition { + condition = ( + can(regex("^[1-9][0-9]{0,9}(:[1-9][0-9]{0,9})?$", var.container.user)) && + var.container.health_port >= 1 && var.container.health_port <= 65535 && + (var.container.health_check_command == null ? true : ( + length(var.container.health_check_command) >= 2 && + contains(["CMD", "CMD-SHELL"], var.container.health_check_command[0]) + )) + ) + error_message = "The container must use a numeric non-root UID (and optional GID), a valid health port, and a valid ECS health-check command." + } + + precondition { + condition = var.container.health_path == "/healthz" + error_message = "container.health_path must be /healthz, the scale-set service liveness endpoint." + } + + precondition { + condition = ( + var.container.health_check_interval >= 5 && var.container.health_check_interval <= 300 && + var.container.health_check_timeout >= 2 && var.container.health_check_timeout <= 60 && + var.container.health_check_timeout < var.container.health_check_interval && + var.container.health_check_retries >= 1 && var.container.health_check_retries <= 10 && + var.container.health_check_start_period >= 0 && var.container.health_check_start_period <= 300 && + var.container.health_stale_after_seconds >= 30 && var.container.health_stale_after_seconds <= 3600 && + var.container.shutdown_timeout_seconds >= 1 && var.container.shutdown_timeout_seconds <= 119 && + var.container.session_close_timeout_seconds >= 1 && var.container.session_close_timeout_seconds <= 60 && + var.container.reconnect_initial_backoff_seconds >= 1 && var.container.reconnect_initial_backoff_seconds <= 300 && + var.container.reconnect_max_backoff_seconds >= 1 && var.container.reconnect_max_backoff_seconds <= 3600 && + var.container.reconnect_initial_backoff_seconds <= var.container.reconnect_max_backoff_seconds && + var.container.stop_timeout_seconds >= 2 && var.container.stop_timeout_seconds <= 120 && + var.container.shutdown_timeout_seconds < var.container.stop_timeout_seconds + ) + error_message = "Container health and shutdown timings must be within ECS limits, with health timeout below interval and application shutdown below task stop timeout." + } + + precondition { + condition = ( + contains(keys(local.fargate_memory_by_cpu), tostring(var.ecs.task.cpu)) && + contains(lookup(local.fargate_memory_by_cpu, tostring(var.ecs.task.cpu), []), var.ecs.task.memory) + ) + error_message = "ecs.task.cpu and ecs.task.memory must be a supported Fargate CPU/memory combination." + } + + precondition { + condition = ( + contains(["X86_64", "ARM64"], var.ecs.task.cpu_architecture) && + (var.ecs.task.ephemeral_storage == null ? true : ( + var.ecs.task.ephemeral_storage.size_in_gib >= 21 && var.ecs.task.ephemeral_storage.size_in_gib <= 200 + )) + ) + error_message = "ecs.task.cpu_architecture must be X86_64 or ARM64, and optional ephemeral storage must be between 21 and 200 GiB." + } + + precondition { + condition = ( + contains(["managed", "external"], var.ecs.cluster.mode) && + (var.ecs.cluster.mode == "external" ? ( + var.ecs.cluster.arn != null && can(regex("^arn:[^:]+:ecs:[^:]+:[0-9]{12}:cluster/.+$", var.ecs.cluster.arn)) + ) : ( + var.ecs.cluster.arn == null && + (var.ecs.cluster.name == null ? true : can(regex("^[A-Za-z0-9_-]{1,255}$", var.ecs.cluster.name))) + )) + ) + error_message = "Use a valid external ECS cluster ARN only with cluster.mode external; managed cluster names may contain letters, digits, underscores, and hyphens." + } + + precondition { + condition = ( + startswith(var.ecs.iam.path, "/") && + endswith(var.ecs.iam.path, "/") && + length(var.ecs.iam.path) <= 512 + ) + error_message = "ecs.iam.path must start and end with a slash and be at most 512 characters." + } + + precondition { + condition = ( + length(var.network.vpc_id) > 0 && + length(var.network.subnet_ids) > 0 && + length(var.network.https_egress.ipv4_cidrs) + length(var.network.https_egress.ipv6_cidrs) > 0 && + alltrue([for cidr in var.network.https_egress.ipv4_cidrs : can(cidrnetmask(cidr))]) && + alltrue([for cidr in var.network.https_egress.ipv6_cidrs : can(cidrhost(cidr, 0)) && strcontains(cidr, ":")]) + ) + error_message = "network must select a VPC and at least one subnet, and HTTPS egress must contain valid IPv4 or IPv6 CIDRs." + } + + precondition { + condition = ( + contains(["STANDARD", "INFREQUENT_ACCESS"], var.logging.log_group_class) && + contains([1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, 3653], var.logging.retention_in_days) && + (var.logging.kms_key_arn == null ? true : can(regex("^arn:[^:]+:kms:[^:]+:[0-9]{12}:key/.+$", var.logging.kms_key_arn))) + ) + error_message = "logging must use a supported class and retention period; kms_key_arn must be a KMS key ARN when set." + } + } +} + +resource "terraform_data" "validate_config_store" { + lifecycle { + precondition { + condition = ( + contains(["Standard", "Advanced"], var.config_store.tier) && + startswith(local.config_store_path_prefix, "/") && + !endswith(local.config_store_path_prefix, "/") && + length(local.config_store_path_prefix) >= 2 && + can(regex("^/[A-Za-z0-9_.\\/-]+$", local.config_store_path_prefix)) + ) + error_message = "config_store must use Standard or Advanced tier and a valid absolute SSM path prefix without a trailing slash." + } + + precondition { + condition = alltrue([ + for config_key, config in local.reconciler_configs : ( + length("${local.config_store_path_prefix}/${config.group_name}/${config.runner_name}") <= 1011 && + local.reconciler_config_bytes[config_key] <= local.config_store_max_bytes + ) + ]) + error_message = "Each reconciler SSM parameter name and encoded JSON value must fit the selected Parameter Store tier. Split large controller groups or reduce provider configuration when necessary." + } + } +} + +resource "terraform_data" "validate_group_task_policy" { + for_each = local.controller_groups + + lifecycle { + precondition { + condition = local.group_task_policy_bytes[each.key] <= 10240 + error_message = "Controller group ${each.key} produces a ${local.group_task_policy_bytes[each.key]}-byte task-role policy, exceeding AWS's 10240-byte inline role-policy quota. Split the group or reduce provider IAM statements." + } + } +} diff --git a/modules/orchestration-providers/scale-set/variables.tf b/modules/orchestration-providers/scale-set/variables.tf new file mode 100644 index 0000000000..c1b854d5f2 --- /dev/null +++ b/modules/orchestration-providers/scale-set/variables.tf @@ -0,0 +1,201 @@ +variable "prefix" { + description = "Stable prefix used for scale-set controller resources." + type = string + default = "github-actions" + nullable = false +} + +variable "runner_configs" { + description = <<-EOT + Normalized scale-set runner configurations keyed by stable runner-config name. + + Map keys must be known during planning. Credential values are never accepted: `github.app` contains only the exact GitHub App Parameter Store references used by the runtime. `github.ssl_verify` applies TLS verification per reconciler without changing process-global TLS behavior. Parameter and optional KMS ARNs, scale-set IDs, and other inner values may remain unknown until apply. + EOT + type = map(object({ + github = object({ + config_url = string + app = object({ + app_id = object({ + name = string + arn = string + kms_key_arn = optional(string, null) + }) + private_key = object({ + name = string + arn = string + kms_key_arn = optional(string, null) + }) + installation_id = object({ + name = string + arn = string + kms_key_arn = optional(string, null) + }) + }) + force_ghes = optional(bool, null) + ssl_verify = optional(bool, true) + user_agent = optional(string, null) + }) + scale_set = object({ + name = string + id = number + runner_group_id = optional(number, null) + min_runners = optional(number, 0) + max_runners = optional(number, 10) + boot_time_in_minutes = optional(number, 10) + session_owner = optional(string, null) + }) + work_folder = optional(string, null) + })) + nullable = false +} + +variable "compute_provider_contracts" { + description = <<-EOT + Provider-neutral, scale-set capability fragments keyed exactly like `runner_configs`. + + `type` is the plan-known provider discriminator used by the default grouping implementation and the runtime adapter registry. `configuration_json` is provider-owned, valid JSON and must contain no secrets. `environment_variables` contains non-secret provider process settings shared by every reconciler in the same controller group; conflicting values are rejected. IAM statement map keys and optional condition shapes must be known during planning; their action, resource, and condition values may be computed. + EOT + type = map(object({ + type = string + capabilities = object({ + scale_set = object({ + configuration_json = optional(string, "{}") + environment_variables = optional(map(string), {}) + iam_statements = optional(map(object({ + actions = set(string) + resources = set(string) + conditions = optional(list(object({ + test = string + variable = string + values = set(string) + })), []) + })), {}) + }) + }) + })) + nullable = false +} + +variable "grouping" { + description = <<-EOT + Packing strategy for scale-set reconcilers. `compute_provider` creates one controller group per compute-provider type and is the default. `runner_config` creates one group per runner config. `custom` uses `custom.groups`; custom membership must cover every runner config exactly once. + + The strategy, custom group keys, and memberships select Terraform `for_each` instances and must be known during planning. + EOT + type = object({ + strategy = optional(string, "compute_provider") + custom = optional(object({ + groups = map(object({ + runner_configs = set(string) + })) + }), null) + }) + default = {} + nullable = false +} + +variable "container" { + description = "Scale-set controller image and runtime settings. A null image uses the internal official convenience image; production callers should use the release digest. Filesystem and Linux capability hardening are enforced by the module; health_path is fixed at /healthz, the ECS liveness endpoint." + type = object({ + image = optional(string, null) + user = optional(string, "10001:10001") + health_port = optional(number, 8080) + health_path = optional(string, "/healthz") + health_check_command = optional(list(string), null) + health_check_interval = optional(number, 30) + health_check_timeout = optional(number, 5) + health_check_retries = optional(number, 3) + health_check_start_period = optional(number, 30) + health_stale_after_seconds = optional(number, 180) + shutdown_timeout_seconds = optional(number, 110) + session_close_timeout_seconds = optional(number, 10) + reconnect_initial_backoff_seconds = optional(number, 1) + reconnect_max_backoff_seconds = optional(number, 30) + stop_timeout_seconds = optional(number, 120) + ecr_repository = optional(object({ + arn = string + }), null) + }) + default = {} + nullable = false +} + +variable "config_store" { + description = <<-EOT + Non-secret controller configuration storage. The module writes one SSM String parameter per reconciler below `path_prefix//`. The task receives only its group path and a SHA-256 revision, then loads the group with `GetParametersByPath`. + + Standard parameters are limited to 4096 encoded bytes and Advanced parameters to 8192 encoded bytes. Null `path_prefix` resolves to `//scale-set-controller`. + EOT + type = object({ + path_prefix = optional(string, null) + tier = optional(string, "Standard") + tags = optional(map(string), {}) + }) + default = {} + nullable = false +} + +variable "ecs" { + description = <<-EOT + ECS substrate configuration. A managed cluster is created by default. For an external cluster, set `cluster.mode = "external"` and pass its ARN; the mode must be plan-known while the ARN may be computed. + EOT + type = object({ + cluster = optional(object({ + mode = optional(string, "managed") + arn = optional(string, null) + name = optional(string, null) + container_insights = optional(bool, true) + }), {}) + task = optional(object({ + cpu = optional(number, 512) + memory = optional(number, 1024) + cpu_architecture = optional(string, "X86_64") + ephemeral_storage = optional(object({ + size_in_gib = number + }), null) + }), {}) + service = optional(object({ + platform_version = optional(string, "LATEST") + }), {}) + iam = optional(object({ + path = optional(string, "/") + permissions_boundary = optional(string, null) + }), {}) + }) + default = {} + nullable = false +} + +variable "network" { + description = <<-EOT + Private Fargate networking. Tasks never receive public IP addresses and the managed security groups have no ingress. HTTPS egress defaults to IPv4 Internet access because GitHub endpoints cannot be represented as security-group destinations; route it through controlled NAT, firewall, or proxy infrastructure when required. + EOT + type = object({ + vpc_id = string + subnet_ids = set(string) + https_egress = optional(object({ + ipv4_cidrs = optional(set(string), ["0.0.0.0/0"]) + ipv6_cidrs = optional(set(string), []) + }), {}) + }) + nullable = false +} + +variable "logging" { + description = "CloudWatch Logs configuration. CloudWatch encrypts logs at rest with an AWS-owned key by default; set `kms_key_arn` to use a customer-managed key." + type = object({ + retention_in_days = optional(number, 30) + kms_key_arn = optional(string, null) + log_group_class = optional(string, "STANDARD") + tags = optional(map(string), {}) + }) + default = {} + nullable = false +} + +variable "tags" { + description = "Tags applied to scale-set orchestration resources." + type = map(string) + default = {} + nullable = false +} diff --git a/modules/orchestration-providers/scale-set/versions.tf b/modules/orchestration-providers/scale-set/versions.tf new file mode 100644 index 0000000000..3ef011ea0a --- /dev/null +++ b/modules/orchestration-providers/scale-set/versions.tf @@ -0,0 +1,10 @@ +terraform { + required_version = ">= 1.4.0" + + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 6.33" + } + } +} diff --git a/modules/runner-config/README.md b/modules/runner-config/README.md index cff1ce73c7..6aff1e8b7b 100644 --- a/modules/runner-config/README.md +++ b/modules/runner-config/README.md @@ -109,21 +109,22 @@ yarn run dist |------|-------------|------|---------|:--------:| | [aws\_partition](#input\_aws\_partition) | AWS partition used to construct ARNs. | `string` | `"aws"` | no | | [aws\_region](#input\_aws\_region) | AWS region. | `string` | n/a | yes | -| [compute\_provider](#input\_compute\_provider) | Typed compute-provider configuration. Provider-owned settings remain inside the selected compute-provider block.

Exactly one compute-provider block must be non-null. The populated block selects the provider, and its presence must be known during planning. Values inside the selected block may remain unknown until apply.

- `aws`: AWS compute-provider configurations.
- `aws.ec2`: EC2 compute-provider configuration.
- `aws.ec2.ami`: Optional AMI discovery or external AMI-parameter configuration. Null uses the operating-system and architecture defaults.
- `aws.ec2.ami.filter`: EC2 AMI filters combined with the provider's default AMI-name filter.
- `aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Null creates a provider-managed AMI-ID parameter. The wrapper's presence is the plan-time ownership discriminator, so keep the object literal even when its ARN comes from another resource.
- `aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply.
- `aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence is the plan-time policy discriminator.
- `aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply.
- `aws.ec2.vpc_id`: VPC in which runner networking resources are created.
- `aws.ec2.subnet_ids`: Subnets from which scale-up may launch runner instances.
- `aws.ec2.overrides`: Optional resource-name overrides.
- `aws.ec2.overrides.name_runner`: Name tag used for runner compute resources. An empty value uses the generated provider name.
- `aws.ec2.overrides.name_sg`: Name tag used for the managed runner security group. An empty value uses the generated provider name.
- `aws.ec2.instance_profile`: Optional externally managed instance profile used by the launch template.
- `aws.ec2.instance_profile.name`: Name of the externally managed instance profile.
- `aws.ec2.instance_profile_path`: IAM path for the provider-managed instance profile. Null uses a path derived from the runner-configuration prefix.
- `aws.ec2.binaries_syncer`: Runner-distribution synchronization configuration.
- `aws.ec2.binaries_syncer.enabled`: Enables use of a synchronized runner distribution from S3.
- `aws.ec2.binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `aws.ec2.binaries_syncer.s3.arn`: ARN of the runner-distribution bucket, used by IAM policies.
- `aws.ec2.binaries_syncer.s3.id`: Bucket name used to construct the runner-distribution S3 URI.
- `aws.ec2.binaries_syncer.s3.key`: Object key of the runner distribution.
- `aws.ec2.block_device_mappings`: EBS mappings added to the runner launch template.
- `aws.ec2.block_device_mappings[].delete_on_termination`: Deletes the volume when its runner instance terminates.
- `aws.ec2.block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `aws.ec2.block_device_mappings[].encrypted`: Enables EBS encryption.
- `aws.ec2.block_device_mappings[].iops`: Provisioned IOPS for volume types that support it.
- `aws.ec2.block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `aws.ec2.block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `aws.ec2.block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `aws.ec2.block_device_mappings[].volume_initialization_rate`: Fixed initialization rate in MiB/s for supported snapshot-backed volumes.
- `aws.ec2.block_device_mappings[].volume_size`: Volume size in GiB.
- `aws.ec2.block_device_mappings[].volume_type`: EBS volume type.
- `aws.ec2.ebs_optimized`: Requests EBS-optimized runner instances.
- `aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select instance capacity.
- `aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price.
- `aws.ec2.instance_types`: EC2 instance types available to the scale-up and pool functions.
- `aws.ec2.user_data`: Runner bootstrap user-data configuration.
- `aws.ec2.user_data.enabled`: Enables launch-template user data.
- `aws.ec2.user_data.template`: Optional path to a custom user-data template.
- `aws.ec2.user_data.content`: Optional complete user-data content. When set, it is used instead of rendering a template.
- `aws.ec2.user_data.pre_install`: Script content inserted before runner installation in the default template.
- `aws.ec2.user_data.post_install`: Script content inserted after runner installation in the default template.
- `aws.ec2.user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets in logs.
- `aws.ec2.ssm_enabled`: Attaches runner permissions and policies required for AWS Systems Manager access.
- `aws.ec2.create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `aws.ec2.cloudwatch_agent`: CloudWatch agent configuration for runner instances.
- `aws.ec2.cloudwatch_agent.enabled`: Installs and configures the CloudWatch agent through the default bootstrap flow.
- `aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration. Null renders the provider default from `log_files`.
- `aws.ec2.managed_security_group_enabled`: Creates and attaches the provider-managed runner security group.
- `aws.ec2.log_files`: Optional log files collected by the CloudWatch agent. Null uses the provider defaults.
- `aws.ec2.log_files[].log_group_name`: CloudWatch log-group name, before optional prefixing.
- `aws.ec2.log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path when true.
- `aws.ec2.log_files[].file_path`: File or glob read by the CloudWatch agent.
- `aws.ec2.log_files[].log_stream_name`: CloudWatch log-stream name template.
- `aws.ec2.log_files[].log_class`: CloudWatch log-group class for the collected file.
- `aws.ec2.key_name`: Optional EC2 key-pair name added to the launch template.
- `aws.ec2.additional_security_group_ids`: Existing security groups attached in addition to the managed security group.
- `aws.ec2.detailed_monitoring_enabled`: Enables detailed EC2 monitoring for runner instances.
- `aws.ec2.egress_rules`: Egress rules created on the managed runner security group.
- `aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `aws.ec2.egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `aws.ec2.egress_rules[].from_port`: First destination port in the permitted range.
- `aws.ec2.egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `aws.ec2.egress_rules[].security_groups`: Destination security-group IDs.
- `aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `aws.ec2.egress_rules[].to_port`: Last destination port in the permitted range.
- `aws.ec2.egress_rules[].description`: Optional rule description.
- `aws.ec2.tags`: Additional tags for runner instances, EBS volumes, network interfaces, and eligible Spot instance requests created from the launch template. They override module-level tags and the generated runner `Name`; the provider-managed `ghr:environment`, `ghr:ssm_config_path`, and `ghr:runner_name_prefix` bootstrap tags take final precedence. These tags do not apply to static provider resources such as the launch template, security group, IAM resources, SSM parameters, or log groups.
- `aws.ec2.metadata_options`: Instance Metadata Service configuration in the launch template.
- `aws.ec2.metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when `enabled`.
- `aws.ec2.metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `aws.ec2.metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `aws.ec2.metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `aws.ec2.credit_specification`: CPU credit mode for burstable instance types, either `standard` or `unlimited`.
- `aws.ec2.cpu_options`: CPU topology and processor-feature configuration.
- `aws.ec2.cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `aws.ec2.cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `aws.ec2.cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `aws.ec2.cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `aws.ec2.placement`: EC2 placement configuration for runner instances.
- `aws.ec2.placement.affinity`: Host affinity setting.
- `aws.ec2.placement.availability_zone`: Availability Zone in which the instance is placed.
- `aws.ec2.placement.group_id`: Placement-group ID.
- `aws.ec2.placement.group_name`: Placement-group name.
- `aws.ec2.placement.host_id`: Dedicated Host ID.
- `aws.ec2.placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `aws.ec2.placement.spread_domain`: Spread-domain placement value.
- `aws.ec2.placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `aws.ec2.placement.partition_number`: Placement-group partition number.
- `aws.ec2.license_specifications`: License Manager configurations added to the launch template.
- `aws.ec2.license_specifications[].license_configuration_arn`: ARN of a License Manager license configuration.
- `aws.ec2.associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `aws.ec2.enable_on_demand_failover_for_errors`: EC2 error codes that trigger an on-demand fallback after a Spot launch failure.
- `aws.ec2.scale_errors`: EC2 error codes treated as retryable scale-up failures.
- `aws.ec2.use_dedicated_host`: Enables the dedicated-host launch path, required for macOS runners. | 
object({
    aws = optional(object({
      ec2 = optional(object({
        ami = optional(object({
          filter = optional(map(list(string)), { state = ["available"] })
          owners = optional(list(string), ["amazon"])
          id_ssm_parameter = optional(object({
            arn = string
          }), null)
          kms_key = optional(object({
            arn = string
          }), null)
        }), null)
        vpc_id     = string
        subnet_ids = list(string)
        overrides = optional(object({
          name_runner = optional(string, "")
          name_sg     = optional(string, "")
        }), {})
        instance_profile = optional(object({
          name = string
        }), null)
        instance_profile_path = optional(string, null)
        binaries_syncer = optional(object({
          enabled = optional(bool, true)
          s3 = optional(object({
            arn = string
            id  = string
            key = string
          }), null)
        }), {})
        block_device_mappings = optional(list(object({
          delete_on_termination      = optional(bool, true)
          device_name                = optional(string, "/dev/xvda")
          encrypted                  = optional(bool, true)
          iops                       = optional(number)
          kms_key_id                 = optional(string)
          snapshot_id                = optional(string)
          throughput                 = optional(number)
          volume_initialization_rate = optional(number)
          volume_size                = number
          volume_type                = optional(string, "gp3")
        })), [{ volume_size = 30 }])
        ebs_optimized                 = optional(bool, false)
        instance_target_capacity_type = optional(string, "spot")
        instance_allocation_strategy  = optional(string, "lowest-price")
        instance_type_priorities      = optional(map(number), null)
        instance_max_spot_price       = optional(string, null)
        instance_types                = list(string)
        user_data = optional(object({
          enabled               = optional(bool, true)
          template              = optional(string, null)
          content               = optional(string, null)
          pre_install           = optional(string, "")
          post_install          = optional(string, "")
          debug_logging_enabled = optional(bool, false)
        }), {})
        ssm_enabled                     = optional(bool, false)
        create_service_linked_role_spot = optional(bool, false)
        cloudwatch_agent = optional(object({
          enabled = optional(bool, true)
          config  = optional(string, null)
        }), {})
        managed_security_group_enabled = optional(bool, true)
        log_files = optional(list(object({
          log_group_name   = string
          prefix_log_group = bool
          file_path        = string
          log_stream_name  = string
          log_class        = optional(string, "STANDARD")
        })), null)
        key_name                      = optional(string, null)
        additional_security_group_ids = optional(list(string), [])
        detailed_monitoring_enabled   = optional(bool, false)
        egress_rules = optional(list(object({
          cidr_blocks      = list(string)
          ipv6_cidr_blocks = list(string)
          prefix_list_ids  = list(string)
          from_port        = number
          protocol         = string
          security_groups  = list(string)
          self             = bool
          to_port          = number
          description      = string
          })), [{
          cidr_blocks      = ["0.0.0.0/0"]
          ipv6_cidr_blocks = ["::/0"]
          prefix_list_ids  = null
          from_port        = 0
          protocol         = "-1"
          security_groups  = null
          self             = null
          to_port          = 0
          description      = null
        }])
        tags = optional(map(string), {})
        metadata_options = optional(object({
          instance_metadata_tags      = optional(string, "enabled")
          http_endpoint               = optional(string, "enabled")
          http_tokens                 = optional(string, "required")
          http_put_response_hop_limit = optional(number, 1)
        }), {})
        credit_specification = optional(string, null)
        cpu_options = optional(object({
          core_count            = optional(number)
          threads_per_core      = optional(number)
          amd_sev_snp           = optional(string)
          nested_virtualization = optional(string)
        }), null)
        placement = optional(object({
          affinity                = optional(string)
          availability_zone       = optional(string)
          group_id                = optional(string)
          group_name              = optional(string)
          host_id                 = optional(string)
          host_resource_group_arn = optional(string)
          spread_domain           = optional(string)
          tenancy                 = optional(string)
          partition_number        = optional(number)
        }), null)
        license_specifications = optional(list(object({
          license_configuration_arn = string
        })), [])
        associate_public_ipv4_address        = optional(bool, false)
        enable_on_demand_failover_for_errors = optional(list(string), [])
        scale_errors = optional(list(string), [
          "UnfulfillableCapacity",
          "MaxSpotInstanceCountExceeded",
          "TargetCapacityLimitExceededException",
          "RequestLimitExceeded",
          "ResourceLimitExceeded",
          "MaxSpotInstanceCountExceeded",
          "MaxSpotFleetRequestCountExceeded",
          "InsufficientInstanceCapacity",
          "InsufficientCapacityOnHost",
        ])
        use_dedicated_host = optional(bool, false)
      }), null)
    }), {})
  })
| n/a | yes | +| [compute\_provider](#input\_compute\_provider) | Typed compute-provider configuration. Provider-owned settings remain inside the selected compute-provider block.

Exactly one compute-provider block must be non-null. The populated block selects the provider, and its presence must be known during planning. Values inside the selected block may remain unknown until apply.

- `aws`: AWS compute-provider configurations.
- `aws.ec2`: EC2 compute-provider configuration.
- `aws.ec2.ami`: Optional AMI discovery or external AMI-parameter configuration. Null uses the operating-system and architecture defaults.
- `aws.ec2.ami.filter`: EC2 AMI filters combined with the provider's default AMI-name filter.
- `aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Null creates a provider-managed AMI-ID parameter. The wrapper's presence is the plan-time ownership discriminator, so keep the object literal even when its ARN comes from another resource.
- `aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply. EC2 scale-set orchestration requires an exact same-account, same-region ARN whose extracted absolute parameter name matches the runtime grammar.
- `aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence is the plan-time policy discriminator.
- `aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply.
- `aws.ec2.vpc_id`: VPC in which runner networking resources are created.
- `aws.ec2.subnet_ids`: Subnets from which scale-up may launch runner instances.
- `aws.ec2.overrides`: Optional resource-name overrides.
- `aws.ec2.overrides.name_runner`: Name tag used for runner compute resources. An empty value uses the generated provider name.
- `aws.ec2.overrides.name_sg`: Name tag used for the managed runner security group. An empty value uses the generated provider name.
- `aws.ec2.instance_profile`: Optional externally managed instance profile used by the launch template.
- `aws.ec2.instance_profile.name`: Name of the externally managed instance profile.
- `aws.ec2.instance_profile_path`: IAM path for the provider-managed instance profile. Null uses a path derived from the runner-configuration prefix.
- `aws.ec2.binaries_syncer`: Runner-distribution synchronization configuration.
- `aws.ec2.binaries_syncer.enabled`: Enables use of a synchronized runner distribution from S3.
- `aws.ec2.binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `aws.ec2.binaries_syncer.s3.arn`: ARN of the runner-distribution bucket, used by IAM policies.
- `aws.ec2.binaries_syncer.s3.id`: Bucket name used to construct the runner-distribution S3 URI.
- `aws.ec2.binaries_syncer.s3.key`: Object key of the runner distribution.
- `aws.ec2.block_device_mappings`: EBS mappings added to the runner launch template.
- `aws.ec2.block_device_mappings[].delete_on_termination`: Deletes the volume when its runner instance terminates.
- `aws.ec2.block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `aws.ec2.block_device_mappings[].encrypted`: Enables EBS encryption.
- `aws.ec2.block_device_mappings[].iops`: Provisioned IOPS for volume types that support it.
- `aws.ec2.block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `aws.ec2.block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `aws.ec2.block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `aws.ec2.block_device_mappings[].volume_initialization_rate`: Fixed initialization rate in MiB/s for supported snapshot-backed volumes.
- `aws.ec2.block_device_mappings[].volume_size`: Volume size in GiB.
- `aws.ec2.block_device_mappings[].volume_type`: EBS volume type.
- `aws.ec2.ebs_optimized`: Requests EBS-optimized runner instances.
- `aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select instance capacity. EC2 scale-set orchestration allows only `lowest-price` or `prioritized` with `on-demand`; Spot supports the provider's complete strategy set.
- `aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price.
- `aws.ec2.instance_types`: EC2 instance types available to the scale-up and pool functions.
- `aws.ec2.user_data`: Runner bootstrap user-data configuration.
- `aws.ec2.user_data.enabled`: Enables launch-template user data.
- `aws.ec2.user_data.template`: Optional path to a custom user-data template.
- `aws.ec2.user_data.content`: Optional complete user-data content. When set, it is used instead of rendering a template.
- `aws.ec2.user_data.pre_install`: Script content inserted before runner installation in the default template.
- `aws.ec2.user_data.post_install`: Script content inserted after runner installation in the default template.
- `aws.ec2.user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets in logs.
- `aws.ec2.ssm_enabled`: Attaches runner permissions and policies required for AWS Systems Manager access.
- `aws.ec2.create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `aws.ec2.cloudwatch_agent`: CloudWatch agent configuration for runner instances.
- `aws.ec2.cloudwatch_agent.enabled`: Installs and configures the CloudWatch agent through the default bootstrap flow.
- `aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration. Null renders the provider default from `log_files`.
- `aws.ec2.managed_security_group_enabled`: Creates and attaches the provider-managed runner security group.
- `aws.ec2.log_files`: Optional log files collected by the CloudWatch agent. Null uses the provider defaults.
- `aws.ec2.log_files[].log_group_name`: CloudWatch log-group name, before optional prefixing.
- `aws.ec2.log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path when true.
- `aws.ec2.log_files[].file_path`: File or glob read by the CloudWatch agent.
- `aws.ec2.log_files[].log_stream_name`: CloudWatch log-stream name template.
- `aws.ec2.log_files[].log_class`: CloudWatch log-group class for the collected file.
- `aws.ec2.key_name`: Optional EC2 key-pair name added to the launch template.
- `aws.ec2.additional_security_group_ids`: Existing security groups attached in addition to the managed security group.
- `aws.ec2.detailed_monitoring_enabled`: Enables detailed EC2 monitoring for runner instances.
- `aws.ec2.egress_rules`: Egress rules created on the managed runner security group.
- `aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `aws.ec2.egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `aws.ec2.egress_rules[].from_port`: First destination port in the permitted range.
- `aws.ec2.egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `aws.ec2.egress_rules[].security_groups`: Destination security-group IDs.
- `aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `aws.ec2.egress_rules[].to_port`: Last destination port in the permitted range.
- `aws.ec2.egress_rules[].description`: Optional rule description.
- `aws.ec2.tags`: Additional tags for runner instances, EBS volumes, network interfaces, and eligible Spot instance requests created from the launch template. They override module-level tags and the generated runner `Name`; the provider-managed `ghr:environment`, `ghr:ssm_config_path`, and `ghr:runner_name_prefix` bootstrap tags take final precedence. These tags do not apply to static provider resources such as the launch template, security group, IAM resources, SSM parameters, or log groups. EC2 scale-set orchestration rejects caller values for its ownership and lifecycle tag keys.
- `aws.ec2.metadata_options`: Instance Metadata Service configuration in the launch template.
- `aws.ec2.metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when `enabled`.
- `aws.ec2.metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `aws.ec2.metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `aws.ec2.metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `aws.ec2.credit_specification`: CPU credit mode for burstable instance types, either `standard` or `unlimited`.
- `aws.ec2.cpu_options`: CPU topology and processor-feature configuration.
- `aws.ec2.cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `aws.ec2.cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `aws.ec2.cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `aws.ec2.cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `aws.ec2.placement`: EC2 placement configuration for runner instances.
- `aws.ec2.placement.affinity`: Host affinity setting.
- `aws.ec2.placement.availability_zone`: Availability Zone in which the instance is placed.
- `aws.ec2.placement.group_id`: Placement-group ID.
- `aws.ec2.placement.group_name`: Placement-group name.
- `aws.ec2.placement.host_id`: Dedicated Host ID.
- `aws.ec2.placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `aws.ec2.placement.spread_domain`: Spread-domain placement value.
- `aws.ec2.placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `aws.ec2.placement.partition_number`: Placement-group partition number.
- `aws.ec2.license_specifications`: License Manager configurations added to the launch template.
- `aws.ec2.license_specifications[].license_configuration_arn`: ARN of a License Manager license configuration.
- `aws.ec2.associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `aws.ec2.enable_on_demand_failover_for_errors`: EC2 error codes that trigger an on-demand fallback after a Spot launch failure.
- `aws.ec2.scale_errors`: EC2 error codes treated as retryable scale-up failures.
- `aws.ec2.use_dedicated_host`: Enables the dedicated-host launch path, required for macOS runners. | 
object({
    aws = optional(object({
      ec2 = optional(object({
        ami = optional(object({
          filter = optional(map(list(string)), { state = ["available"] })
          owners = optional(list(string), ["amazon"])
          id_ssm_parameter = optional(object({
            arn = string
          }), null)
          kms_key = optional(object({
            arn = string
          }), null)
        }), null)
        vpc_id     = string
        subnet_ids = list(string)
        overrides = optional(object({
          name_runner = optional(string, "")
          name_sg     = optional(string, "")
        }), {})
        instance_profile = optional(object({
          name = string
        }), null)
        instance_profile_path = optional(string, null)
        binaries_syncer = optional(object({
          enabled = optional(bool, true)
          s3 = optional(object({
            arn = string
            id  = string
            key = string
          }), null)
        }), {})
        block_device_mappings = optional(list(object({
          delete_on_termination      = optional(bool, true)
          device_name                = optional(string, "/dev/xvda")
          encrypted                  = optional(bool, true)
          iops                       = optional(number)
          kms_key_id                 = optional(string)
          snapshot_id                = optional(string)
          throughput                 = optional(number)
          volume_initialization_rate = optional(number)
          volume_size                = number
          volume_type                = optional(string, "gp3")
        })), [{ volume_size = 30 }])
        ebs_optimized                 = optional(bool, false)
        instance_target_capacity_type = optional(string, "spot")
        instance_allocation_strategy  = optional(string, "lowest-price")
        instance_type_priorities      = optional(map(number), null)
        instance_max_spot_price       = optional(string, null)
        instance_types                = list(string)
        user_data = optional(object({
          enabled               = optional(bool, true)
          template              = optional(string, null)
          content               = optional(string, null)
          pre_install           = optional(string, "")
          post_install          = optional(string, "")
          debug_logging_enabled = optional(bool, false)
        }), {})
        ssm_enabled                     = optional(bool, false)
        create_service_linked_role_spot = optional(bool, false)
        cloudwatch_agent = optional(object({
          enabled = optional(bool, true)
          config  = optional(string, null)
        }), {})
        managed_security_group_enabled = optional(bool, true)
        log_files = optional(list(object({
          log_group_name   = string
          prefix_log_group = bool
          file_path        = string
          log_stream_name  = string
          log_class        = optional(string, "STANDARD")
        })), null)
        key_name                      = optional(string, null)
        additional_security_group_ids = optional(list(string), [])
        detailed_monitoring_enabled   = optional(bool, false)
        egress_rules = optional(list(object({
          cidr_blocks      = list(string)
          ipv6_cidr_blocks = list(string)
          prefix_list_ids  = list(string)
          from_port        = number
          protocol         = string
          security_groups  = list(string)
          self             = bool
          to_port          = number
          description      = string
          })), [{
          cidr_blocks      = ["0.0.0.0/0"]
          ipv6_cidr_blocks = ["::/0"]
          prefix_list_ids  = null
          from_port        = 0
          protocol         = "-1"
          security_groups  = null
          self             = null
          to_port          = 0
          description      = null
        }])
        tags = optional(map(string), {})
        metadata_options = optional(object({
          instance_metadata_tags      = optional(string, "enabled")
          http_endpoint               = optional(string, "enabled")
          http_tokens                 = optional(string, "required")
          http_put_response_hop_limit = optional(number, 1)
        }), {})
        credit_specification = optional(string, null)
        cpu_options = optional(object({
          core_count            = optional(number)
          threads_per_core      = optional(number)
          amd_sev_snp           = optional(string)
          nested_virtualization = optional(string)
        }), null)
        placement = optional(object({
          affinity                = optional(string)
          availability_zone       = optional(string)
          group_id                = optional(string)
          group_name              = optional(string)
          host_id                 = optional(string)
          host_resource_group_arn = optional(string)
          spread_domain           = optional(string)
          tenancy                 = optional(string)
          partition_number        = optional(number)
        }), null)
        license_specifications = optional(list(object({
          license_configuration_arn = string
        })), [])
        associate_public_ipv4_address        = optional(bool, false)
        enable_on_demand_failover_for_errors = optional(list(string), [])
        scale_errors = optional(list(string), [
          "UnfulfillableCapacity",
          "MaxSpotInstanceCountExceeded",
          "TargetCapacityLimitExceededException",
          "RequestLimitExceeded",
          "ResourceLimitExceeded",
          "MaxSpotFleetRequestCountExceeded",
          "InsufficientInstanceCapacity",
          "InsufficientCapacityOnHost",
        ])
        use_dedicated_host = optional(bool, false)
      }), null)
    }), {})
  })
| n/a | yes | | [compute\_provider\_key](#input\_compute\_provider\_key) | Optional plan-known compute-provider dispatch key. Null discovers the key from the exactly one populated compute\_provider block. | `string` | `null` | no | | [github](#input\_github) | GitHub API and runner-registration configuration.

- `app_parameters.key_base64`: Ordered Parameter Store references for GitHub App private keys.
- `app_parameters.id`: Ordered Parameter Store references for GitHub App IDs.
- `app_parameters.installation_id`: Ordered optional Parameter Store references for GitHub App installation IDs.
- `enterprise_server.url`: Optional GitHub Enterprise Server base URL. Null selects GitHub.com.
- `enterprise_server.ssl_verify`: Enables TLS certificate verification for GitHub Enterprise Server requests.
- `user_agent`: Optional User-Agent value added to GitHub API requests. | 
object({
    app_parameters = object({
      key_base64      = list(map(string))
      id              = list(map(string))
      installation_id = list(object({ name = string, arn = string }))
    })
    enterprise_server = optional(object({
      url        = optional(string, null)
      ssl_verify = optional(bool, true)
    }), {})
    user_agent = optional(string, null)
  })
| n/a | yes | | [lambda](#input\_lambda) | Common Lambda substrate independent of the selected runner orchestration provider.

- `artifact.s3.bucket`: Optional shared S3 bucket containing component-owned Lambda artifacts. An orchestration provider selects its own object key and version; the bucket alone selects no artifact.
- `runtime`: Runtime used by the control-plane Lambda functions.
- `architecture`: Instruction-set architecture used by the control-plane Lambda functions. Supported values are `arm64` and `x86_64`.
- `subnet_ids`: Subnets used for Lambda VPC configuration.
- `security_group_ids`: Security groups used for Lambda VPC configuration.
- `tags`: Shared tags applied to Lambda function resources only. These override module-level `tags`; component `tags` override this map when keys conflict.
- `principals`: Additional principals allowed to assume the control-plane Lambda roles.
- `role.path`: IAM path for module-managed Lambda execution roles. Defaults to a path derived from `prefix`.
- `role.permissions_boundary`: Permissions-boundary ARN applied to module-managed Lambda execution roles. | 
object({
    artifact = optional(object({
      s3 = optional(object({
        bucket = optional(string, null)
      }), {})
    }), {})
    runtime            = optional(string, "nodejs24.x")
    architecture       = optional(string, "arm64")
    subnet_ids         = optional(list(string), [])
    security_group_ids = optional(list(string), [])
    tags               = optional(map(string), {})
    principals = optional(list(object({
      type        = string
      identifiers = list(string)
    })), [])
    role = optional(object({
      path                 = optional(string, null)
      permissions_boundary = optional(string, null)
    }), {})
  })
| `{}` | no | | [observability](#input\_observability) | Logging, tracing, and metrics configuration for control-plane and provider resources.

- `logs.level`: Application log level supplied to the control-plane functions.
- `logs.retention_in_days`: CloudWatch Logs retention period.
- `logs.kms_key_id`: Optional KMS key ID or ARN used to encrypt CloudWatch log groups.
- `logs.class`: CloudWatch log-group class. Supported values are `STANDARD` and `INFREQUENT_ACCESS`.
- `logs.tags`: Shared tags for CloudWatch log groups. These override module-level `tags`; component `tags` override this map when keys conflict.
- `tracing.mode`: Optional Lambda active-tracing mode. Null disables X-Ray tracing configuration.
- `tracing.capture_http_requests`: Enables HTTP request capture in the tracing helper.
- `tracing.capture_error`: Enables error capture in the tracing helper.
- `metrics.enable`: Enables module-emitted metrics.
- `metrics.namespace`: CloudWatch namespace used for emitted metrics.
- `metrics.metric.enable_github_app_rate_limit`: Emits GitHub App rate-limit metrics.
- `metrics.metric.enable_job_retry`: Emits job-retry metrics.
- `metrics.metric.enable_spot_termination_warning`: Emits spot-termination warning metrics where supported. | 
object({
    logs = optional(object({
      level             = optional(string, "info")
      retention_in_days = optional(number, 180)
      kms_key_id        = optional(string, null)
      class             = optional(string, "STANDARD")
      tags              = optional(map(string), {})
    }), {})
    tracing = optional(object({
      mode                  = optional(string, null)
      capture_http_requests = optional(bool, false)
      capture_error         = optional(bool, false)
    }), {})
    metrics = optional(object({
      enable    = optional(bool, false)
      namespace = optional(string, "GitHub Runners")
      metric = optional(object({
        enable_github_app_rate_limit    = optional(bool, true)
        enable_job_retry                = optional(bool, true)
        enable_spot_termination_warning = optional(bool, true)
      }), {})
    }), {})
  })
| `{}` | no | -| [orchestration\_provider](#input\_orchestration\_provider) | Runner demand-orchestration provider configuration. Exactly one provider block must be non-null. Wrapper presence selects the provider and must therefore be known during planning; values inside the selected provider may remain unknown until apply.

- `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls. Future providers can be added as sibling blocks without moving this contract.
- `webhook.runner`: Runner lifecycle, boot timeout, and capacity settings owned by webhook orchestration.
- `webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by scale-down and pool controls. The default is `5`.
- `webhook.runner.ephemeral`: Registers runners in ephemeral mode. The default is `false`.
- `webhook.runner.jit_config_enabled`: Explicitly enables or disables just-in-time configuration. The default is null, which follows `runner.ephemeral`.
- `webhook.runner.maximum_count`: Maximum number of runners managed for this runner configuration. The default is `3`.
- `webhook.github.organization_runners`: Registers runners at organization scope when true; otherwise registration is repository-scoped.
- `webhook.queue.build.arn`: ARN of the runner configuration's build queue.
- `webhook.queue.build.url`: URL of the runner configuration's build queue.
- `webhook.queue.kms_key_id`: Optional KMS key ARN encrypting the build queue. The default is null and is independent from the Parameter Store KMS key.
- `webhook.queue.tags`: Tags inherited by queue-related provider resources before component-specific overrides. The default is `{}`.
- `webhook.lambda.artifact`: Runner-control artifact shared by scale, pool, and job-retry components. Set at most one of `zip` or `s3`; no selection uses the packaged runner archive.
- `webhook.lambda.artifact.zip`: Optional local path to the runner-control Lambda archive. The default is null.
- `webhook.lambda.artifact.s3`: Optional S3 object selector in the common `lambda.artifact.s3.bucket`. Wrapper presence must be known during planning, selecting it requires a non-null common bucket, and the default is null.
- `webhook.lambda.artifact.s3.key`: Object key of the runner-control Lambda archive.
- `webhook.lambda.artifact.s3.object_version`: Optional object version of the runner-control Lambda archive. The default is null.
- `webhook.lambda.scale.up.memory_size`: Memory allocated to the scale-up Lambda in MB. The default is `512`.
- `webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for scale-up. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.scale.up.job_queued_check_enabled`: Enables queued-job verification before scaling. The default is null, which follows the resolved runner mode.
- `webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered per scale-up invocation. The default is `10`.
- `webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum batching window for build-queue records. The default is `0`.
- `webhook.lambda.scale.up.tags`: Tags applied within scale-up resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.scale.down.memory_size`: Memory allocated to the scale-down Lambda in MB. The default is `512`.
- `webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.down.schedule_expression`: EventBridge schedule expression that invokes scale-down. The default is `cron(*/5 * * * ? *)`.
- `webhook.lambda.scale.down.minimum_running_time_in_minutes`: Optional minimum runner age before scale-down may terminate it. The default is null, which selects the operating-system default.
- `webhook.lambda.scale.down.idle_config`: Time-based desired idle-runner configurations. The default is `[]`.
- `webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate the cron expression.
- `webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners retained during the matching period.
- `webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed. The default is `oldest_first`.
- `webhook.lambda.scale.down.tags`: Tags applied within scale-down resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.pool.memory_size`: Memory allocated to the pool Lambda in MB. The default is `512`.
- `webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.pool.config`: Scheduled target pool sizes. The default is `[]`, which disables the pool component.
- `webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `webhook.lambda.pool.config[].size`: Desired number of runners for the schedule.
- `webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity. The default is `false`.
- `webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used for pooled runners. The default is null.
- `webhook.lambda.pool.tags`: Tags applied within pool resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.enabled`: Creates the retry queue, Lambda function, event-source mapping, and related IAM resources. The default is `false`.
- `webhook.job_retry.delay_in_seconds`: Initial delay before a queued-job retry check. The default is `300`.
- `webhook.job_retry.delay_backoff`: Multiplier applied to the delay after each unsuccessful check. The default is `2`.
- `webhook.job_retry.max_attempts`: Maximum retry-check attempts before the message is no longer republished. The default is `1`.
- `webhook.job_retry.tags`: Tags applied within job-retry resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.lambda.memory_size`: Memory allocated to the job-retry Lambda in MB. The default is `256`.
- `webhook.job_retry.lambda.reserved_concurrent_executions`: Reserved concurrency for job retry. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.job_retry.lambda.timeout`: Job-retry Lambda timeout in seconds and visibility timeout for its retry queue. The default is `30`. | 
object({
    webhook = optional(object({
      runner = optional(object({
        boot_time_in_minutes = optional(number, 5)
        ephemeral            = optional(bool, false)
        jit_config_enabled   = optional(bool, null)
        maximum_count        = optional(number, 3)
      }), {})
      github = object({
        organization_runners = bool
      })
      queue = object({
        build = object({
          arn = string
          url = string
        })
        kms_key_id = optional(string, null)
        tags       = optional(map(string), {})
      })
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        scale = optional(object({
          up = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 60)
            reserved_concurrent_executions = optional(number, 1)
            job_queued_check_enabled       = optional(bool, null)
            event_source_mapping = optional(object({
              batch_size                         = optional(number, 10)
              maximum_batching_window_in_seconds = optional(number, 0)
            }), {})
            tags = optional(map(string), {})
          }), {})
          down = optional(object({
            memory_size                     = optional(number, 512)
            timeout                         = optional(number, 60)
            schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
            minimum_running_time_in_minutes = optional(number, null)
            idle_config = optional(list(object({
              cron             = string
              timeZone         = string
              idleCount        = number
              evictionStrategy = optional(string, "oldest_first")
            })), [])
            tags = optional(map(string), {})
          }), {})
        }), {})
        pool = optional(object({
          memory_size                    = optional(number, 512)
          timeout                        = optional(number, 60)
          reserved_concurrent_executions = optional(number, 1)
          config = optional(list(object({
            schedule_expression          = string
            schedule_expression_timezone = optional(string)
            size                         = number
          })), [])
          include_busy_runners = optional(bool, false)
          runner_owner         = optional(string, null)
          tags                 = optional(map(string), {})
        }), {})
      }), {})
      job_retry = optional(object({
        enabled          = optional(bool, false)
        delay_in_seconds = optional(number, 300)
        delay_backoff    = optional(number, 2)
        max_attempts     = optional(number, 1)
        tags             = optional(map(string), {})
        lambda = optional(object({
          memory_size                    = optional(number, 256)
          reserved_concurrent_executions = optional(number, 1)
          timeout                        = optional(number, 30)
        }), {})
      }), {})
    }), null)
  })
| n/a | yes | +| [orchestration\_provider](#input\_orchestration\_provider) | Runner demand-orchestration provider configuration. Exactly one provider block must be non-null. Wrapper presence selects the provider and must therefore be known during planning; values inside the selected provider may remain unknown until apply.

- `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls.
- `scale_set`: Selects scale-set orchestration for this runner config. The multi-runner topology owns the shared controller service and passes only this plan-known selection marker to runner-config. Scale-set runners always use ephemeral JIT registration.
- `webhook.runner`: Runner lifecycle, boot timeout, and capacity settings owned by webhook orchestration.
- `webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by scale-down and pool controls. The default is `5`.
- `webhook.runner.ephemeral`: Registers runners in ephemeral mode. The default is `false`.
- `webhook.runner.jit_config_enabled`: Explicitly enables or disables just-in-time configuration. The default is null, which follows `runner.ephemeral`.
- `webhook.runner.maximum_count`: Maximum number of runners managed for this runner configuration. The default is `3`.
- `webhook.github.organization_runners`: Registers runners at organization scope when true; otherwise registration is repository-scoped.
- `webhook.queue.build.arn`: ARN of the runner configuration's build queue.
- `webhook.queue.build.url`: URL of the runner configuration's build queue.
- `webhook.queue.kms_key_id`: Optional KMS key ARN encrypting the build queue. The default is null and is independent from the Parameter Store KMS key.
- `webhook.queue.tags`: Tags inherited by queue-related provider resources before component-specific overrides. The default is `{}`.
- `webhook.lambda.artifact`: Runner-control artifact shared by scale, pool, and job-retry components. Set at most one of `zip` or `s3`; no selection uses the packaged runner archive.
- `webhook.lambda.artifact.zip`: Optional local path to the runner-control Lambda archive. The default is null.
- `webhook.lambda.artifact.s3`: Optional S3 object selector in the common `lambda.artifact.s3.bucket`. Wrapper presence must be known during planning, selecting it requires a non-null common bucket, and the default is null.
- `webhook.lambda.artifact.s3.key`: Object key of the runner-control Lambda archive.
- `webhook.lambda.artifact.s3.object_version`: Optional object version of the runner-control Lambda archive. The default is null.
- `webhook.lambda.scale.up.memory_size`: Memory allocated to the scale-up Lambda in MB. The default is `512`.
- `webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for scale-up. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.scale.up.job_queued_check_enabled`: Enables queued-job verification before scaling. The default is null, which follows the resolved runner mode.
- `webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered per scale-up invocation. The default is `10`.
- `webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum batching window for build-queue records. The default is `0`.
- `webhook.lambda.scale.up.tags`: Tags applied within scale-up resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.scale.down.memory_size`: Memory allocated to the scale-down Lambda in MB. The default is `512`.
- `webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.down.schedule_expression`: EventBridge schedule expression that invokes scale-down. The default is `cron(*/5 * * * ? *)`.
- `webhook.lambda.scale.down.minimum_running_time_in_minutes`: Optional minimum runner age before scale-down may terminate it. The default is null, which selects the operating-system default.
- `webhook.lambda.scale.down.idle_config`: Time-based desired idle-runner configurations. The default is `[]`.
- `webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate the cron expression.
- `webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners retained during the matching period.
- `webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed. The default is `oldest_first`.
- `webhook.lambda.scale.down.tags`: Tags applied within scale-down resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.pool.memory_size`: Memory allocated to the pool Lambda in MB. The default is `512`.
- `webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.pool.config`: Scheduled target pool sizes. The default is `[]`, which disables the pool component.
- `webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `webhook.lambda.pool.config[].size`: Desired number of runners for the schedule.
- `webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity. The default is `false`.
- `webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used for pooled runners. The default is null.
- `webhook.lambda.pool.tags`: Tags applied within pool resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.enabled`: Creates the retry queue, Lambda function, event-source mapping, and related IAM resources. The default is `false`.
- `webhook.job_retry.delay_in_seconds`: Initial delay before a queued-job retry check. The default is `300`.
- `webhook.job_retry.delay_backoff`: Multiplier applied to the delay after each unsuccessful check. The default is `2`.
- `webhook.job_retry.max_attempts`: Maximum retry-check attempts before the message is no longer republished. The default is `1`.
- `webhook.job_retry.tags`: Tags applied within job-retry resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.lambda.memory_size`: Memory allocated to the job-retry Lambda in MB. The default is `256`.
- `webhook.job_retry.lambda.reserved_concurrent_executions`: Reserved concurrency for job retry. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.job_retry.lambda.timeout`: Job-retry Lambda timeout in seconds and visibility timeout for its retry queue. The default is `30`. | 
object({
    webhook = optional(object({
      runner = optional(object({
        boot_time_in_minutes = optional(number, 5)
        ephemeral            = optional(bool, false)
        jit_config_enabled   = optional(bool, null)
        maximum_count        = optional(number, 3)
      }), {})
      github = object({
        organization_runners = bool
      })
      queue = object({
        build = object({
          arn = string
          url = string
        })
        kms_key_id = optional(string, null)
        tags       = optional(map(string), {})
      })
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        scale = optional(object({
          up = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 60)
            reserved_concurrent_executions = optional(number, 1)
            job_queued_check_enabled       = optional(bool, null)
            event_source_mapping = optional(object({
              batch_size                         = optional(number, 10)
              maximum_batching_window_in_seconds = optional(number, 0)
            }), {})
            tags = optional(map(string), {})
          }), {})
          down = optional(object({
            memory_size                     = optional(number, 512)
            timeout                         = optional(number, 60)
            schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
            minimum_running_time_in_minutes = optional(number, null)
            idle_config = optional(list(object({
              cron             = string
              timeZone         = string
              idleCount        = number
              evictionStrategy = optional(string, "oldest_first")
            })), [])
            tags = optional(map(string), {})
          }), {})
        }), {})
        pool = optional(object({
          memory_size                    = optional(number, 512)
          timeout                        = optional(number, 60)
          reserved_concurrent_executions = optional(number, 1)
          config = optional(list(object({
            schedule_expression          = string
            schedule_expression_timezone = optional(string)
            size                         = number
          })), [])
          include_busy_runners = optional(bool, false)
          runner_owner         = optional(string, null)
          tags                 = optional(map(string), {})
        }), {})
      }), {})
      job_retry = optional(object({
        enabled          = optional(bool, false)
        delay_in_seconds = optional(number, 300)
        delay_backoff    = optional(number, 2)
        max_attempts     = optional(number, 1)
        tags             = optional(map(string), {})
        lambda = optional(object({
          memory_size                    = optional(number, 256)
          reserved_concurrent_executions = optional(number, 1)
          timeout                        = optional(number, 30)
        }), {})
      }), {})
    }), null)
    scale_set = optional(object({}), null)
  })
| n/a | yes | | [prefix](#input\_prefix) | The prefix used for naming resources. | `string` | `"github-actions"` | no | -| [runner](#input\_runner) | Provider-neutral GitHub runner configuration.

- `os`: Runner operating system. Supported values are `linux`, `osx`, and `windows`.
- `architecture`: Runner distribution architecture, such as `x64` or `arm64`.
- `disable_default_labels`: Prevents GitHub's default self-hosted, operating-system, and architecture labels from being registered.
- `labels`: Complete set of labels supplied to the control-plane functions.
- `group_name`: GitHub runner group used during registration.
- `name_prefix`: Prefix added to registered runner names.
- `run_as_root`: Runs the runner service as root when supported by the compute provider.
- `run_as`: Operating-system user used when `run_as_root` is false.
- `auto_update_disabled`: Disables the GitHub runner application's built-in updater.
- `tags`: Additional tags for common runner resources, currently the managed runner IAM role. These override module-level `tags` with the same key.
- `hooks.job_started`: Script content installed as the runner job-started hook.
- `hooks.job_completed`: Script content installed as the runner job-completed hook.
- `iam.role.arn`: ARN of an externally managed runner role. When set, this module does not create or modify that role.
- `iam.managed_policy_arns`: Named managed-policy ARNs attached to the module-managed runner role.
- `iam.additional_trust_policy_json`: Optional IAM policy document merged with the selected compute provider's default runner-role trust policy.
- `iam.path`: IAM path for the module-managed runner role. Defaults to a path derived from `prefix`.
- `iam.permissions_boundary`: Permissions-boundary ARN for the module-managed runner role. | 
object({
    os                     = optional(string, "linux")
    architecture           = optional(string, "x64")
    disable_default_labels = optional(bool, false)
    labels                 = list(string)
    group_name             = optional(string, "Default")
    name_prefix            = optional(string, "")
    run_as_root            = optional(bool, false)
    run_as                 = optional(string, "ec2-user")
    auto_update_disabled   = optional(bool, false)
    tags                   = optional(map(string), {})
    hooks = optional(object({
      job_started   = optional(string, "")
      job_completed = optional(string, "")
    }), {})
    iam = optional(object({
      role = optional(object({
        arn = string
      }), null)
      managed_policy_arns          = optional(map(string), {})
      additional_trust_policy_json = optional(string, null)
      path                         = optional(string, null)
      permissions_boundary         = optional(string, null)
    }), {})
  })
| n/a | yes | -| [ssm](#input\_ssm) | Parameter Store paths, encryption, tag scopes, and housekeeper configuration.

- `paths.root`: Root Parameter Store path for this runner configuration.
- `paths.tokens`: Path segment under `paths.root` used for registration tokens and just-in-time configuration.
- `paths.config`: Path segment under `paths.root` used for persistent runner configuration.
- `kms_key_id`: Optional customer-managed KMS key ARN used by control-plane IAM policies to decrypt shared GitHub App parameters. The ARN may be unknown until apply; null omits the provider-owned KMS statements. It does not select encryption for runtime-created runner parameters.
- `tags`: Shared tags for SSM-related resources. These override module-level `tags` and are inherited by parameter and housekeeper resources.
- `parameters.tags`: Tags for Terraform-managed runner configuration parameters and temporary parameters created by the scale-up and pool Lambdas. These override module-level and `ssm.tags` values with the same key.
- `housekeeper.schedule_expression`: EventBridge schedule expression that invokes the SSM housekeeper.
- `housekeeper.state`: EventBridge rule state, such as `ENABLED` or `DISABLED`.
- `housekeeper.tags`: Tags for housekeeper resources, including the Lambda function, log group, EventBridge rule, and IAM role. These override module-level, `ssm.tags`, shared Lambda, and shared log tags when keys conflict.
- `housekeeper.lambda.artifact`: Component-owned SSM-housekeeper artifact selection. Set at most one of `zip` or `s3`; when neither is selected, the module uses its packaged runner control-plane archive. This selector does not inherit an orchestration-provider artifact.
- `housekeeper.lambda.artifact.zip`: Optional local path to the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.artifact.s3`: Optional object key and version in the shared `lambda.artifact.s3.bucket`. Selecting S3 requires that common bucket.
- `housekeeper.lambda.artifact.s3.key`: Object key of the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.artifact.s3.object_version`: Optional object version of the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.memory_size`: Memory allocated to the SSM housekeeper Lambda in MB.
- `housekeeper.lambda.timeout`: SSM housekeeper Lambda timeout in seconds.
- `housekeeper.config.tokenPath`: Parameter Store token path cleaned by the housekeeper. When omitted, the configured runner token path is used.
- `housekeeper.config.minimumDaysOld`: Minimum parameter age in days before deletion is allowed.
- `housekeeper.config.dryRun`: Reports eligible parameters without deleting them when true. | 
object({
    paths = object({
      root   = string
      tokens = string
      config = string
    })
    kms_key_id = optional(string, null)
    tags       = optional(map(string), {})
    parameters = optional(object({
      tags = optional(map(string), {})
    }), {})
    housekeeper = optional(object({
      schedule_expression = optional(string, "rate(1 day)")
      state               = optional(string, "ENABLED")
      tags                = optional(map(string), {})
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        memory_size = optional(number, 512)
        timeout     = optional(number, 60)
      }), {})
      config = optional(object({
        tokenPath      = optional(string)
        minimumDaysOld = optional(number, 1)
        dryRun         = optional(bool, false)
      }), {})
    }), {})
  })
| n/a | yes | +| [runner](#input\_runner) | Provider-neutral GitHub runner configuration.

- `os`: Runner operating system. Supported values are `linux`, `osx`, and `windows`.
- `architecture`: Runner distribution architecture, such as `x64` or `arm64`.
- `disable_default_labels`: Prevents GitHub's default self-hosted, operating-system, and architecture labels from being registered.
- `labels`: Complete set of labels supplied to the control-plane functions.
- `group_name`: GitHub runner group used during registration.
- `name_prefix`: Prefix added to registered runner names. EC2 scale-set orchestration additionally requires at most 45 ASCII letters, digits, dots, underscores, or hyphens.
- `run_as_root`: Runs the runner service as root when supported by the compute provider.
- `run_as`: Operating-system user used when `run_as_root` is false.
- `auto_update_disabled`: Disables the GitHub runner application's built-in updater.
- `tags`: Additional tags for common runner resources, currently the managed runner IAM role. These override module-level `tags` with the same key.
- `hooks.job_started`: Script content installed as the runner job-started hook.
- `hooks.job_completed`: Script content installed as the runner job-completed hook.
- `iam.role.arn`: ARN of an externally managed runner role. When set, this module does not create or modify that role.
- `iam.managed_policy_arns`: Named managed-policy ARNs attached to the module-managed runner role.
- `iam.additional_trust_policy_json`: Optional IAM policy document merged with the selected compute provider's default runner-role trust policy.
- `iam.path`: IAM path for the module-managed runner role. Defaults to a path derived from `prefix`.
- `iam.permissions_boundary`: Permissions-boundary ARN for the module-managed runner role. | 
object({
    os                     = optional(string, "linux")
    architecture           = optional(string, "x64")
    disable_default_labels = optional(bool, false)
    labels                 = list(string)
    group_name             = optional(string, "Default")
    name_prefix            = optional(string, "")
    run_as_root            = optional(bool, false)
    run_as                 = optional(string, "ec2-user")
    auto_update_disabled   = optional(bool, false)
    tags                   = optional(map(string), {})
    hooks = optional(object({
      job_started   = optional(string, "")
      job_completed = optional(string, "")
    }), {})
    iam = optional(object({
      role = optional(object({
        arn = string
      }), null)
      managed_policy_arns          = optional(map(string), {})
      additional_trust_policy_json = optional(string, null)
      path                         = optional(string, null)
      permissions_boundary         = optional(string, null)
    }), {})
  })
| n/a | yes | +| [ssm](#input\_ssm) | Parameter Store paths, encryption, tag scopes, and housekeeper configuration.

- `paths.root`: Root Parameter Store path for this runner configuration.
- `paths.tokens`: Path segment under `paths.root` used for registration tokens and just-in-time configuration.
- `paths.config`: Path segment under `paths.root` used for persistent runner configuration.
- `kms_key_id`: Optional customer-managed KMS key ARN used by control-plane IAM policies to decrypt shared GitHub App parameters. The ARN may be unknown until apply; null omits the provider-owned KMS statements. It does not select encryption for runtime-created runner parameters.
- `tags`: Shared tags for SSM-related resources. These override module-level `tags` and are inherited by parameter and housekeeper resources. When EC2 scale-set orchestration is selected, the effective Parameter Store tag map must contain at most 45 runtime-compatible keys and values.
- `parameters.tags`: Tags for Terraform-managed runner configuration parameters and temporary parameters created by webhook controls or the scale-set reconciler. These override module-level and `ssm.tags` values with the same key and participate in the EC2 scale-set runtime tag limit.
- `housekeeper.schedule_expression`: EventBridge schedule expression that invokes the SSM housekeeper.
- `housekeeper.state`: EventBridge rule state, such as `ENABLED` or `DISABLED`.
- `housekeeper.tags`: Tags for housekeeper resources, including the Lambda function, log group, EventBridge rule, and IAM role. These override module-level, `ssm.tags`, shared Lambda, and shared log tags when keys conflict.
- `housekeeper.lambda.artifact`: Component-owned SSM-housekeeper artifact selection. Set at most one of `zip` or `s3`; when neither is selected, the module uses its packaged runner control-plane archive. This selector does not inherit an orchestration-provider artifact.
- `housekeeper.lambda.artifact.zip`: Optional local path to the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.artifact.s3`: Optional object key and version in the shared `lambda.artifact.s3.bucket`. Selecting S3 requires that common bucket.
- `housekeeper.lambda.artifact.s3.key`: Object key of the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.artifact.s3.object_version`: Optional object version of the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.memory_size`: Memory allocated to the SSM housekeeper Lambda in MB.
- `housekeeper.lambda.timeout`: SSM housekeeper Lambda timeout in seconds.
- `housekeeper.config.tokenPath`: Parameter Store token path cleaned by the housekeeper. When omitted, the configured runner token path is used.
- `housekeeper.config.minimumDaysOld`: Minimum parameter age in days before deletion is allowed.
- `housekeeper.config.dryRun`: Reports eligible parameters without deleting them when true. | 
object({
    paths = object({
      root   = string
      tokens = string
      config = string
    })
    kms_key_id = optional(string, null)
    tags       = optional(map(string), {})
    parameters = optional(object({
      tags = optional(map(string), {})
    }), {})
    housekeeper = optional(object({
      schedule_expression = optional(string, "rate(1 day)")
      state               = optional(string, "ENABLED")
      tags                = optional(map(string), {})
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        memory_size = optional(number, 512)
        timeout     = optional(number, 60)
      }), {})
      config = optional(object({
        tokenPath      = optional(string)
        minimumDaysOld = optional(number, 1)
        dryRun         = optional(bool, false)
      }), {})
    }), {})
  })
| n/a | yes | | [tags](#input\_tags) | Base tags added to taggable resources created by this runner configuration. Shared, component, and compute-provider tag maps override matching keys within their documented resource scopes. | `map(string)` | `{}` | no | ## Outputs | Name | Description | |------|-------------| +| [compute\_provider\_contract](#output\_compute\_provider\_contract) | Provider-neutral orchestration capabilities exposed by the selected compute provider for topology-level aggregation. | | [orchestration\_provider](#output\_orchestration\_provider) | Resources grouped under the selected runner orchestration provider. | | [pool](#output\_pool) | Scheduled pool resources. Null when no pool configuration is supplied. | | [provider](#output\_provider) | Provider-specific resources grouped under the selected provider namespace and type. | diff --git a/modules/runner-config/compute-provider.tf b/modules/runner-config/compute-provider.tf index bffc43b814..581adc97c2 100644 --- a/modules/runner-config/compute-provider.tf +++ b/modules/runner-config/compute-provider.tf @@ -13,17 +13,50 @@ locals { aws_ec2 = "ec2" } - provider_type = local.provider_types[local.provider_key] + provider_type = try(local.provider_types[local.provider_key], null) provider_assume_role_policies = { aws_ec2 = try(module.compute_aws_ec2_trust_policy[0].assume_role_policy, null) } - provider_assume_role_policy = local.provider_assume_role_policies[local.provider_key] + provider_assume_role_policy = try(local.provider_assume_role_policies[local.provider_key], null) + + empty_provider_contract = { + type = null + capabilities = { scale_set = null } + environment_variables = { + scale_up = {} + scale_down = {} + pool = {} + } + policies = { + runner = { + inline_policies = {} + managed_policy_arns = {} + } + scale_up = { + iam_policy_json = null + additional_iam_policy_json = null + managed_policy_enabled = false + managed_policy_arn = null + } + scale_down = { + iam_policy_json = null + } + pool = { + iam_policy_json = null + managed_policy_enabled = false + managed_policy_arn = null + } + } + resources = null + } provider_contracts = { aws_ec2 = one(module.compute_aws_ec2[*].provider) } - provider_contract = local.provider_contracts[local.provider_key] + # Keep invalid or empty selections evaluable long enough for validate_config + # to report the exact-one contract error. + provider_contract = local.provider_key == null ? local.empty_provider_contract : local.provider_contracts[local.provider_key] } diff --git a/modules/runner-config/orchestration-provider.tf b/modules/runner-config/orchestration-provider.tf index 25994beaba..1735e86dc2 100644 --- a/modules/runner-config/orchestration-provider.tf +++ b/modules/runner-config/orchestration-provider.tf @@ -7,12 +7,57 @@ locals { orchestration_provider_type = one(keys(local.orchestration_providers)) orchestration_provider_enabled = { - webhook = local.orchestration_provider_type == "webhook" + webhook = local.orchestration_provider_type == "webhook" + scale_set = local.orchestration_provider_type == "scale_set" } orchestration_provider_runner_lifecycle = { webhook = one(module.orchestration_webhook[*].runner_lifecycle) + scale_set = { + ephemeral = true + jit_config_enabled = true + } }[local.orchestration_provider_type] + + scale_set_ec2_reserved_runner_tag_keys = toset([ + "ghr:Application", + "ghr:created_by", + "ghr:Type", + "ghr:Owner", + "ghr:runner_config", + "ghr:scale_set_id", + "ghr:github_scope_hash", + "ghr:scale_set_state", + "ghr:runner_name", + "ghr:github_runner_id", + ]) + + scale_set_ec2_selected = ( + var.orchestration_provider.scale_set != null && + local.provider_key == "aws_ec2" + ) + + # The EC2 scale-set runtime serializes the same merged tags as the provider's + # Parameter Store resources, including the generated Name tag. + scale_set_ec2_ssm_parameter_tags = local.scale_set_ec2_selected ? merge( + { Name = format("%s-action-runner", var.prefix) }, + var.tags, + var.ssm.tags, + var.ssm.parameters.tags, + ) : {} + + scale_set_ec2_external_ami_parameter_arn = local.scale_set_ec2_selected ? try( + var.compute_provider.aws.ec2.ami.id_ssm_parameter.arn, + null, + ) : null + + scale_set_ec2_external_ami_parameter_name = local.scale_set_ec2_external_ami_parameter_arn == null ? null : try( + regex( + "^arn:${var.aws_partition}:ssm:${var.aws_region}:${data.aws_caller_identity.current.account_id}:parameter(/[A-Za-z0-9_.\\-/]+)$", + local.scale_set_ec2_external_ami_parameter_arn, + )[0], + null, + ) } module "orchestration_webhook" { diff --git a/modules/runner-config/outputs.tf b/modules/runner-config/outputs.tf index 486e3261eb..cbe8d1a638 100644 --- a/modules/runner-config/outputs.tf +++ b/modules/runner-config/outputs.tf @@ -29,6 +29,15 @@ output "orchestration_provider" { pool = one(module.orchestration_webhook[*].pool) job_retry = one(module.orchestration_webhook[*].job_retry) } : null + scale_set = local.orchestration_provider_enabled.scale_set ? {} : null + } +} + +output "compute_provider_contract" { + description = "Provider-neutral orchestration capabilities exposed by the selected compute provider for topology-level aggregation." + value = { + type = local.provider_contract.type + capabilities = local.provider_contract.capabilities } } diff --git a/modules/runner-config/runner-role.tf b/modules/runner-config/runner-role.tf index 6baa1e4206..0e6ad9de7d 100644 --- a/modules/runner-config/runner-role.tf +++ b/modules/runner-config/runner-role.tf @@ -2,12 +2,16 @@ locals { # Role ownership belongs to the common runner configuration. The selected trust-policy # submodule supplies the assume-role document, while the full compute provider # supplies permissions after the role has been resolved. - create_runner_role = var.runner.iam.role == null - - runner_role = { - arn = local.create_runner_role ? one(aws_iam_role.runner[*].arn) : var.runner.iam.role.arn - name = local.create_runner_role ? one(aws_iam_role.runner[*].name) : basename(var.runner.iam.role.arn) - managed = local.create_runner_role + create_runner_role = var.runner.iam.role == null && local.provider_key != null + + runner_role = var.runner.iam.role == null ? { + arn = one(aws_iam_role.runner[*].arn) + name = one(aws_iam_role.runner[*].name) + managed = true + } : { + arn = var.runner.iam.role.arn + name = basename(var.runner.iam.role.arn) + managed = false } common_runner_managed_policy_arns = merge( diff --git a/modules/runner-config/tests/pool.tftest.hcl b/modules/runner-config/tests/pool.tftest.hcl index 12a81854c3..a039bd674b 100644 --- a/modules/runner-config/tests/pool.tftest.hcl +++ b/modules/runner-config/tests/pool.tftest.hcl @@ -1,4 +1,10 @@ mock_provider "aws" { + mock_data "aws_caller_identity" { + defaults = { + account_id = "123456789012" + } + } + mock_data "aws_iam_policy_document" { defaults = { json = "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"lambda.amazonaws.com\"},\"Action\":\"sts:AssumeRole\"}]}" @@ -210,8 +216,9 @@ run "plan_with_pool_enabled" { assert { condition = ( - toset(keys(output.orchestration_provider)) == toset(["webhook"]) + toset(keys(output.orchestration_provider)) == toset(["webhook", "scale_set"]) && output.orchestration_provider.webhook != null + && output.orchestration_provider.scale_set == null && output.orchestration_provider.webhook.scale_up != null && output.orchestration_provider.webhook.scale_down != null && output.orchestration_provider.webhook.pool != null @@ -422,6 +429,454 @@ run "rejects_missing_orchestration_provider" { expect_failures = [terraform_data.validate_config] } +run "scale_set_selects_jit_lifecycle_without_per_runner_controller" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + } + + assert { + condition = ( + length(module.orchestration_webhook) == 0 && + aws_ssm_parameter.runner_agent_mode.value == "ephemeral" && + aws_ssm_parameter.jit_config_enabled.value == "true" && + output.scale_up == null && + output.scale_down == null && + output.pool == null && + output.orchestration_provider.webhook == null && + output.orchestration_provider.scale_set != null + ) + error_message = "Scale-set selection must use fixed ephemeral JIT lifecycle, keep webhook aliases null, and create no per-runner orchestration controller." + } + + assert { + condition = ( + output.compute_provider_contract.type == "ec2" && + output.compute_provider_contract.capabilities.scale_set != null + ) + error_message = "Runner-config must expose the selected compute provider's scale-set capability for topology-level aggregation." + } +} + +run "webhook_preserves_existing_runner_tag_contract" { + command = plan + + variables { + tags = { + "ghr:github_scope_hash" = "legacy-caller-value" + } + } + + assert { + condition = output.orchestration_provider.webhook != null + error_message = "Scale-set ownership-tag restrictions must not change the existing webhook runner tag contract." + } +} + +run "webhook_preserves_existing_ec2_runtime_input_contract" { + command = plan + + variables { + runner = { + labels = ["self-hosted", "linux", "x64"] + name_prefix = "legacy prefix" + } + ssm = { + paths = { + root = "/github-runner" + tokens = "tokens" + config = "config" + } + tags = { + "aws:legacy" = "allowed-for-webhook" + } + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = concat([for index in range(101) : format("subnet-%08x", index)], ["subnet-00000000"]) + instance_types = concat([for index in range(101) : "m5.${index}large"], ["m5.0large"]) + instance_type_priorities = { "m5.large" = 1001.5 } + instance_target_capacity_type = "on-demand" + instance_allocation_strategy = "diversified" + enable_on_demand_failover_for_errors = concat( + [for index in range(101) : "FailoverError${index}"], + ["FailoverError0"], + ) + scale_errors = concat( + [for index in range(101) : "ScaleError${index}"], + ["ScaleError0"], + ) + binaries_syncer = { + enabled = false + } + } + } + } + } + + assert { + condition = output.orchestration_provider.webhook != null + error_message = "Scale-set runtime parser restrictions must not change existing webhook-only EC2 inputs." + } +} + +run "scale_set_rejects_duplicate_ec2_runtime_lists" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678", "subnet-12345678"] + instance_types = ["m5.large", "m5.large"] + enable_on_demand_failover_for_errors = ["RequestLimitExceeded", "RequestLimitExceeded"] + scale_errors = ["InsufficientInstanceCapacity", "InsufficientInstanceCapacity"] + binaries_syncer = { + enabled = false + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_oversized_ec2_runtime_lists" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = [for index in range(101) : format("subnet-%08x", index)] + instance_types = [for index in range(101) : "m5.${index}large"] + enable_on_demand_failover_for_errors = [ + for index in range(101) : "FailoverError${index}" + ] + scale_errors = [for index in range(101) : "ScaleError${index}"] + binaries_syncer = { + enabled = false + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_fractional_ec2_instance_type_priority" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + instance_types = ["m5.large"] + instance_type_priorities = { + "m5.large" = 1.5 + } + binaries_syncer = { + enabled = false + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_out_of_range_ec2_instance_type_priorities" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + instance_types = ["m5.large", "c5.large"] + instance_type_priorities = { + "m5.large" = -1 + "c5.large" = 1001 + } + binaries_syncer = { + enabled = false + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_accepts_ec2_runtime_boundaries" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = [for index in range(100) : format("subnet-%08x", index)] + instance_types = [for index in range(100) : "m5.${index}large"] + instance_type_priorities = { + "m5.0large" = 0 + "m5.99large" = 1000 + } + enable_on_demand_failover_for_errors = [ + for index in range(100) : "FailoverError${index}" + ] + scale_errors = [for index in range(100) : "ScaleError${index}"] + binaries_syncer = { + enabled = false + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } +} + +run "scale_set_rejects_provider_owned_runner_tags" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + instance_types = ["m5.large"] + binaries_syncer = { + enabled = false + } + tags = { + "ghr:github_scope_hash" = "caller-controlled" + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_unsafe_ec2_runner_name_prefix" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + runner = { + labels = ["self-hosted", "linux", "x64"] + name_prefix = "unsafe prefix" + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_incompatible_ec2_capacity_strategy" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + instance_types = ["m5.large"] + instance_target_capacity_type = "on-demand" + instance_allocation_strategy = "diversified" + binaries_syncer = { + enabled = false + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_runtime_incompatible_ssm_tags" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + ssm = { + paths = { + root = "/github-runner" + tokens = "tokens" + config = "config" + } + tags = { + "aws:reserved" = "not-valid-for-runtime-created-parameters" + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_too_many_ssm_tags" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + ssm = { + paths = { + root = "/github-runner" + tokens = "tokens" + config = "config" + } + tags = { + for index in range(45) : "Tag${index}" => "value" + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "scale_set_rejects_invalid_external_ami_parameter_arn" { + command = plan + + variables { + orchestration_provider = { + scale_set = {} + } + compute_provider = { + aws = { + ec2 = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + instance_types = ["m5.large"] + ami = { + id_ssm_parameter = { + arn = "arn:aws:ssm:us-east-1:123456789012:parameter/github-runner/external-ami-id" + } + } + binaries_syncer = { + enabled = false + } + } + } + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + +run "rejects_multiple_orchestration_providers" { + command = plan + + variables { + orchestration_provider = { + webhook = { + github = { + organization_runners = true + } + queue = { + build = { + arn = "arn:aws:sqs:eu-west-1:123456789012:build-queue" + url = "https://sqs.eu-west-1.amazonaws.com/123456789012/build-queue" + } + } + } + scale_set = {} + } + } + + plan_options { + target = [terraform_data.validate_config] + } + + expect_failures = [terraform_data.validate_config] +} + run "external_runner_role_is_not_managed_by_common" { command = plan @@ -555,25 +1010,6 @@ run "external_role_rejects_trust_policy_extension" { expect_failures = [terraform_data.validate_config] } -run "rejects_invalid_trust_policy_extension" { - command = plan - - variables { - runner = { - labels = ["self-hosted", "linux", "x64"] - iam = { - additional_trust_policy_json = "{" - } - } - } - - plan_options { - target = [terraform_data.validate_config] - } - - expect_failures = [terraform_data.validate_config] -} - run "rejects_empty_compute_provider" { command = plan diff --git a/modules/runner-config/validations.tf b/modules/runner-config/validations.tf index f510bf0422..ba9a096749 100644 --- a/modules/runner-config/validations.tf +++ b/modules/runner-config/validations.tf @@ -90,7 +90,93 @@ resource "terraform_data" "validate_config" { for provider_name, provider_config in var.orchestration_provider : provider_name if provider_config != null ]) == 1 - error_message = "Exactly one orchestration provider must be configured. Supported providers: webhook." + error_message = "Exactly one orchestration provider must be configured. Supported providers: webhook and scale_set." + } + + precondition { + condition = var.orchestration_provider.scale_set == null ? true : local.provider_contract.capabilities.scale_set != null + error_message = "The selected compute provider must expose a scale_set capability when scale_set orchestration is selected." + } + + precondition { + condition = var.orchestration_provider.scale_set == null ? true : ( + local.provider_key == "aws_ec2" ? length(setintersection( + toset(keys(merge(var.tags, var.compute_provider.aws.ec2.tags))), + local.scale_set_ec2_reserved_runner_tag_keys, + )) == 0 : true + ) + error_message = "Scale-set runner tags must not set provider-owned ownership or lifecycle keys." + } + + precondition { + condition = !local.scale_set_ec2_selected ? true : ( + length(var.runner.name_prefix) <= 45 && + length(regexall("[^A-Za-z0-9._-]", var.runner.name_prefix)) == 0 + ) + error_message = "Scale-set EC2 runner.name_prefix must contain at most 45 ASCII letters, digits, dots, underscores, or hyphens." + } + + precondition { + condition = !local.scale_set_ec2_selected ? true : ( + var.compute_provider.aws.ec2.instance_target_capacity_type == "spot" || + contains( + ["lowest-price", "prioritized"], + var.compute_provider.aws.ec2.instance_allocation_strategy, + ) + ) + error_message = "Scale-set EC2 on-demand capacity supports only lowest-price or prioritized instance allocation strategies." + } + + precondition { + condition = !local.scale_set_ec2_selected ? true : alltrue([ + for values in [ + var.compute_provider.aws.ec2.subnet_ids, + var.compute_provider.aws.ec2.instance_types, + var.compute_provider.aws.ec2.enable_on_demand_failover_for_errors, + var.compute_provider.aws.ec2.scale_errors, + ] : length(values) <= 100 && length(values) == length(distinct(values)) + ]) + error_message = "Scale-set EC2 subnet_ids, instance_types, enable_on_demand_failover_for_errors, and scale_errors must each contain at most 100 unique values." + } + + precondition { + condition = !local.scale_set_ec2_selected ? true : ( + var.compute_provider.aws.ec2.instance_type_priorities == null ? true : alltrue([ + for priority in values(var.compute_provider.aws.ec2.instance_type_priorities) : ( + priority >= 0 && + priority <= 1000 && + floor(priority) == priority + ) + ]) + ) + error_message = "Scale-set EC2 instance_type_priorities values must be integers from 0 through 1000." + } + + precondition { + condition = !local.scale_set_ec2_selected ? true : ( + length(local.scale_set_ec2_ssm_parameter_tags) <= 45 && + alltrue([ + for key, value in local.scale_set_ec2_ssm_parameter_tags : ( + length(key) >= 1 && + length(key) <= 128 && + length(regexall("[^A-Za-z0-9_.:/=+@-]", key)) == 0 && + !startswith(lower(key), "aws:") && + length(value) <= 256 && + length(regexall("[[:cntrl:]]", value)) == 0 + ) + ]) + ) + error_message = "Scale-set EC2 Parameter Store tags must contain at most 45 entries with runtime-compatible keys and values." + } + + precondition { + condition = (!local.scale_set_ec2_selected || local.scale_set_ec2_external_ami_parameter_arn == null) ? true : ( + local.scale_set_ec2_external_ami_parameter_name == null ? false : ( + length(local.scale_set_ec2_external_ami_parameter_name) <= 900 && + !strcontains(local.scale_set_ec2_external_ami_parameter_name, "//") + ) + ) + error_message = "Scale-set EC2 external AMI parameters must use an exact same-account, same-region SSM parameter ARN whose extracted absolute name matches the runtime grammar." } precondition { diff --git a/modules/runner-config/variables.compute-provider.tf b/modules/runner-config/variables.compute-provider.tf index 2b63be5703..1a2012d493 100644 --- a/modules/runner-config/variables.compute-provider.tf +++ b/modules/runner-config/variables.compute-provider.tf @@ -23,7 +23,7 @@ variable "compute_provider" { - `aws.ec2.ami.filter`: EC2 AMI filters combined with the provider's default AMI-name filter. - `aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI. - `aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Null creates a provider-managed AMI-ID parameter. The wrapper's presence is the plan-time ownership discriminator, so keep the object literal even when its ARN comes from another resource. - - `aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply. + - `aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply. EC2 scale-set orchestration requires an exact same-account, same-region ARN whose extracted absolute parameter name matches the runtime grammar. - `aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence is the plan-time policy discriminator. - `aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply. - `aws.ec2.vpc_id`: VPC in which runner networking resources are created. @@ -53,7 +53,7 @@ variable "compute_provider" { - `aws.ec2.block_device_mappings[].volume_type`: EBS volume type. - `aws.ec2.ebs_optimized`: Requests EBS-optimized runner instances. - `aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`. - - `aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select instance capacity. + - `aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select instance capacity. EC2 scale-set orchestration allows only `lowest-price` or `prioritized` with `on-demand`; Spot supports the provider's complete strategy set. - `aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type. - `aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price. - `aws.ec2.instance_types`: EC2 instance types available to the scale-up and pool functions. @@ -89,7 +89,7 @@ variable "compute_provider" { - `aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true. - `aws.ec2.egress_rules[].to_port`: Last destination port in the permitted range. - `aws.ec2.egress_rules[].description`: Optional rule description. - - `aws.ec2.tags`: Additional tags for runner instances, EBS volumes, network interfaces, and eligible Spot instance requests created from the launch template. They override module-level tags and the generated runner `Name`; the provider-managed `ghr:environment`, `ghr:ssm_config_path`, and `ghr:runner_name_prefix` bootstrap tags take final precedence. These tags do not apply to static provider resources such as the launch template, security group, IAM resources, SSM parameters, or log groups. + - `aws.ec2.tags`: Additional tags for runner instances, EBS volumes, network interfaces, and eligible Spot instance requests created from the launch template. They override module-level tags and the generated runner `Name`; the provider-managed `ghr:environment`, `ghr:ssm_config_path`, and `ghr:runner_name_prefix` bootstrap tags take final precedence. These tags do not apply to static provider resources such as the launch template, security group, IAM resources, SSM parameters, or log groups. EC2 scale-set orchestration rejects caller values for its ownership and lifecycle tag keys. - `aws.ec2.metadata_options`: Instance Metadata Service configuration in the launch template. - `aws.ec2.metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when `enabled`. - `aws.ec2.metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint. @@ -250,7 +250,6 @@ variable "compute_provider" { "TargetCapacityLimitExceededException", "RequestLimitExceeded", "ResourceLimitExceeded", - "MaxSpotInstanceCountExceeded", "MaxSpotFleetRequestCountExceeded", "InsufficientInstanceCapacity", "InsufficientCapacityOnHost", diff --git a/modules/runner-config/variables.orchestration-provider.tf b/modules/runner-config/variables.orchestration-provider.tf index 6d176ad6d7..6fe2642ff1 100644 --- a/modules/runner-config/variables.orchestration-provider.tf +++ b/modules/runner-config/variables.orchestration-provider.tf @@ -3,7 +3,8 @@ variable "orchestration_provider" { description = <<-EOT Runner demand-orchestration provider configuration. Exactly one provider block must be non-null. Wrapper presence selects the provider and must therefore be known during planning; values inside the selected provider may remain unknown until apply. - - `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls. Future providers can be added as sibling blocks without moving this contract. + - `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls. + - `scale_set`: Selects scale-set orchestration for this runner config. The multi-runner topology owns the shared controller service and passes only this plan-known selection marker to runner-config. Scale-set runners always use ephemeral JIT registration. - `webhook.runner`: Runner lifecycle, boot timeout, and capacity settings owned by webhook orchestration. - `webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by scale-down and pool controls. The default is `5`. - `webhook.runner.ephemeral`: Registers runners in ephemeral mode. The default is `false`. @@ -135,6 +136,7 @@ variable "orchestration_provider" { }), {}) }), {}) }), null) + scale_set = optional(object({}), null) }) nullable = false diff --git a/modules/runner-config/variables.tf b/modules/runner-config/variables.tf index 9315eb48c7..3beed85004 100644 --- a/modules/runner-config/variables.tf +++ b/modules/runner-config/variables.tf @@ -30,7 +30,7 @@ variable "runner" { - `disable_default_labels`: Prevents GitHub's default self-hosted, operating-system, and architecture labels from being registered. - `labels`: Complete set of labels supplied to the control-plane functions. - `group_name`: GitHub runner group used during registration. - - `name_prefix`: Prefix added to registered runner names. + - `name_prefix`: Prefix added to registered runner names. EC2 scale-set orchestration additionally requires at most 45 ASCII letters, digits, dots, underscores, or hyphens. - `run_as_root`: Runs the runner service as root when supported by the compute provider. - `run_as`: Operating-system user used when `run_as_root` is false. - `auto_update_disabled`: Disables the GitHub runner application's built-in updater. @@ -142,8 +142,8 @@ variable "ssm" { - `paths.tokens`: Path segment under `paths.root` used for registration tokens and just-in-time configuration. - `paths.config`: Path segment under `paths.root` used for persistent runner configuration. - `kms_key_id`: Optional customer-managed KMS key ARN used by control-plane IAM policies to decrypt shared GitHub App parameters. The ARN may be unknown until apply; null omits the provider-owned KMS statements. It does not select encryption for runtime-created runner parameters. - - `tags`: Shared tags for SSM-related resources. These override module-level `tags` and are inherited by parameter and housekeeper resources. - - `parameters.tags`: Tags for Terraform-managed runner configuration parameters and temporary parameters created by the scale-up and pool Lambdas. These override module-level and `ssm.tags` values with the same key. + - `tags`: Shared tags for SSM-related resources. These override module-level `tags` and are inherited by parameter and housekeeper resources. When EC2 scale-set orchestration is selected, the effective Parameter Store tag map must contain at most 45 runtime-compatible keys and values. + - `parameters.tags`: Tags for Terraform-managed runner configuration parameters and temporary parameters created by webhook controls or the scale-set reconciler. These override module-level and `ssm.tags` values with the same key and participate in the EC2 scale-set runtime tag limit. - `housekeeper.schedule_expression`: EventBridge schedule expression that invokes the SSM housekeeper. - `housekeeper.state`: EventBridge rule state, such as `ENABLED` or `DISABLED`. - `housekeeper.tags`: Tags for housekeeper resources, including the Lambda function, log group, EventBridge rule, and IAM role. These override module-level, `ssm.tags`, shared Lambda, and shared log tags when keys conflict.