feat(multi-runner): add experimental EC2 provider stack - #5251
Open
edersonbrilhante wants to merge 37 commits into
Open
feat(multi-runner): add experimental EC2 provider stack#5251edersonbrilhante wants to merge 37 commits into
edersonbrilhante wants to merge 37 commits into
Conversation
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
edersonbrilhante
force-pushed
the
experimental-multi-runner-config-v2-20260805
branch
from
August 5, 2026 20:58
34b9f68 to
eafd4b5
Compare
Brend-Smits
reviewed
Aug 6, 2026
Brend-Smits
left a comment
Brend-Smits
left a comment
Contributor
There was a problem hiding this comment.
I like the general direction where this is going. Wondering if it makes sense to introduce some kind of schema for people to consume as well now that we are making these changes.
This will be a much needed change seeing as we are moving to a new architecture that supports multiple compute variants. Please do keep in mind though that the people that are reading this documentation (like in your variables.tf), might now have all the context that you have. So try to keep the easy to consume for those folks as well (explain jargon, etc).
Do I understand it correctly that this change will be opt-in for this version while we test it further and in the future we will make a breaking change version update to change the default to this new config?
Great work!
edersonbrilhante
changed the title
This was referenced Aug 6, 2026
edersonbrilhante
added a commit
that referenced
this pull request
Aug 7, 2026
…dules (#5257) ## Description - Keep `modules/runners` and stable `multi_runner_config` dispatch unchanged. Stable configurations retain their historical `module.runners["configuration"]` addresses and flat `runners_map` fields. - Add explicit opt-in through `experimental.multi_runner_config_v2`. Stable and experimental configurations can coexist when their keys do not overlap; duplicate keys are rejected. - Normalize stable v1 once for shared queues, webhook matching, and runner-binary discovery while routing only v2 configurations through `modules/runner-stack`. - Make `runner-stack` the provider-neutral control plane for scale-up, scale-down, pool, job retry, SSM housekeeping, common Lambda IAM, and runner-role ownership. - Keep EC2-specific launch templates, instance profiles, security groups, AMI/bootstrap resources, runner log groups, IAM fragments, and Lambda environment fragments under `modules/compute-providers/ec2`. - Define provider-owned runner-role requirements in EC2 and attach them to the common runner role in `runner-stack`, allowing future compute providers to supply different policies without duplicating the role lifecycle. - Replace flat runner-stack inputs with ownership-based nested objects. Logging configuration is grouped under `observability.logs`, including `level`, retention, encryption, class, and tags. - Pass the canonical `compute_provider.ec2` object and nested `runner`, `github`, `ssm`, and `observability` objects directly into the EC2 resource and runner-role policy modules instead of expanding them back into prefixed scalar inputs. - Layer module, shared-resource, component, subcomponent, and EC2 runtime tags with documented precedence; provider-required EC2 bootstrap tags retain final precedence. - Group experimental v2 outputs by ownership: `runner.role`, `scale_up.{lambda,log_group,role}`, `scale_down.{lambda,log_group,role}`, nullable `pool.{lambda,log_group,role}`, and provider-specific resources under `provider.<type>`. - Use caller-known optional wrappers for external AMI parameters and KMS keys. The wrapper determines Terraform graph shape while its `arn` leaf may remain unknown until apply. - Generate runner-stack, pool, job-retry, and EC2 IAM policies with `aws_iam_policy_document` and retain provider-policy merge behavior. - Document the experimental boundary, ownership model, plan-time wrapper pattern, phased migration, and nested output contract under the internal module documentation path. This draft is stacked on #5251 because the provider boundary consumes the experimental v2 normalization introduced there. Lambda/TypeScript terminology changes are tracked separately in #5258. ## Test Plan - `pre-commit run --all-files` — Terraform fmt, TFLint, validation, and merge-conflict checks passed. - `terraform test` in `modules/runner-stack` — 11 passed. - `terraform test` in `modules/multi-runner` — 7 passed. - `terraform test` in `modules/compute-providers/ec2` — 4 passed. - `terraform test` in `modules/compute-providers/ec2/runner-role` — 3 passed. - `terraform test` in `modules/runner-stack/pool` — 1 passed. - `terraform test` in `modules/runner-stack/job-retry` — 1 passed. - `terraform validate` in `modules/lambda` — passed. - Verified `modules/runners` has no diff from `origin/main`, stable v1 still dispatches only to `module.runners`, and only the experimental map dispatches to `module.runner_stacks`. - Verified computed external role, profile, AMI-parameter, managed-policy, and KMS ARN inputs plan successfully through the real wrapper fixture. No live AWS apply was performed. Terraform tests use mocked providers, and state migration is intentionally deferred to the later migration phase. ## Related Issues Closes #5252 Depends on #5251 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Contributor
Author
|
@Brend-Smits I changed my first implementation. I will not touch runners. Instead I created a whole cloned runners, refactoring it to add support for a decoupled ec2 module. |
edersonbrilhante
marked this pull request as ready for review
August 7, 2026 20:19
edersonbrilhante
force-pushed
the
experimental-multi-runner-config-v2-20260805
branch
from
August 7, 2026 21:42
4a9c778 to
0849752
Compare
edersonbrilhante
force-pushed
the
refactor-runner-label-resolution
branch
2 times, most recently
from
August 12, 2026 21:19
aaaaa64 to
26666e1
Compare
edersonbrilhante
added a commit
that referenced
this pull request
Aug 12, 2026
…dules (#5257) ## Description - Keep `modules/runners` and stable `multi_runner_config` dispatch unchanged. Stable configurations retain their historical `module.runners["configuration"]` addresses and flat `runners_map` fields. - Add explicit opt-in through `experimental.multi_runner_config_v2`. Stable and experimental configurations can coexist when their keys do not overlap; duplicate keys are rejected. - Normalize stable v1 once for shared queues, webhook matching, and runner-binary discovery while routing only v2 configurations through `modules/runner-stack`. - Make `runner-stack` the provider-neutral control plane for scale-up, scale-down, pool, job retry, SSM housekeeping, common Lambda IAM, and runner-role ownership. - Keep EC2-specific launch templates, instance profiles, security groups, AMI/bootstrap resources, runner log groups, IAM fragments, and Lambda environment fragments under `modules/compute-providers/ec2`. - Define provider-owned runner-role requirements in EC2 and attach them to the common runner role in `runner-stack`, allowing future compute providers to supply different policies without duplicating the role lifecycle. - Replace flat runner-stack inputs with ownership-based nested objects. Logging configuration is grouped under `observability.logs`, including `level`, retention, encryption, class, and tags. - Pass the canonical `compute_provider.ec2` object and nested `runner`, `github`, `ssm`, and `observability` objects directly into the EC2 resource and runner-role policy modules instead of expanding them back into prefixed scalar inputs. - Layer module, shared-resource, component, subcomponent, and EC2 runtime tags with documented precedence; provider-required EC2 bootstrap tags retain final precedence. - Group experimental v2 outputs by ownership: `runner.role`, `scale_up.{lambda,log_group,role}`, `scale_down.{lambda,log_group,role}`, nullable `pool.{lambda,log_group,role}`, and provider-specific resources under `provider.<type>`. - Use caller-known optional wrappers for external AMI parameters and KMS keys. The wrapper determines Terraform graph shape while its `arn` leaf may remain unknown until apply. - Generate runner-stack, pool, job-retry, and EC2 IAM policies with `aws_iam_policy_document` and retain provider-policy merge behavior. - Document the experimental boundary, ownership model, plan-time wrapper pattern, phased migration, and nested output contract under the internal module documentation path. This draft is stacked on #5251 because the provider boundary consumes the experimental v2 normalization introduced there. Lambda/TypeScript terminology changes are tracked separately in #5258. ## Test Plan - `pre-commit run --all-files` — Terraform fmt, TFLint, validation, and merge-conflict checks passed. - `terraform test` in `modules/runner-stack` — 11 passed. - `terraform test` in `modules/multi-runner` — 7 passed. - `terraform test` in `modules/compute-providers/ec2` — 4 passed. - `terraform test` in `modules/compute-providers/ec2/runner-role` — 3 passed. - `terraform test` in `modules/runner-stack/pool` — 1 passed. - `terraform test` in `modules/runner-stack/job-retry` — 1 passed. - `terraform validate` in `modules/lambda` — passed. - Verified `modules/runners` has no diff from `origin/main`, stable v1 still dispatches only to `module.runners`, and only the experimental map dispatches to `module.runner_stacks`. - Verified computed external role, profile, AMI-parameter, managed-policy, and KMS ARN inputs plan successfully through the real wrapper fixture. No live AWS apply was performed. Terraform tests use mocked providers, and state migration is intentionally deferred to the later migration phase. ## Related Issues Closes #5252 Depends on #5251 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
edersonbrilhante
force-pushed
the
experimental-multi-runner-config-v2-20260805
branch
from
August 12, 2026 21:58
0849752 to
2aa54f7
Compare
edersonbrilhante
added a commit
that referenced
this pull request
Aug 12, 2026
…dules (#5257) ## Description - Keep `modules/runners` and stable `multi_runner_config` dispatch unchanged. Stable configurations retain their historical `module.runners["configuration"]` addresses and flat `runners_map` fields. - Add explicit opt-in through `experimental.multi_runner_config_v2`. Stable and experimental configurations can coexist when their keys do not overlap; duplicate keys are rejected. - Normalize stable v1 once for shared queues, webhook matching, and runner-binary discovery while routing only v2 configurations through `modules/runner-stack`. - Make `runner-stack` the provider-neutral control plane for scale-up, scale-down, pool, job retry, SSM housekeeping, common Lambda IAM, and runner-role ownership. - Keep EC2-specific launch templates, instance profiles, security groups, AMI/bootstrap resources, runner log groups, IAM fragments, and Lambda environment fragments under `modules/compute-providers/ec2`. - Define provider-owned runner-role requirements in EC2 and attach them to the common runner role in `runner-stack`, allowing future compute providers to supply different policies without duplicating the role lifecycle. - Replace flat runner-stack inputs with ownership-based nested objects. Logging configuration is grouped under `observability.logs`, including `level`, retention, encryption, class, and tags. - Pass the canonical `compute_provider.ec2` object and nested `runner`, `github`, `ssm`, and `observability` objects directly into the EC2 resource and runner-role policy modules instead of expanding them back into prefixed scalar inputs. - Layer module, shared-resource, component, subcomponent, and EC2 runtime tags with documented precedence; provider-required EC2 bootstrap tags retain final precedence. - Group experimental v2 outputs by ownership: `runner.role`, `scale_up.{lambda,log_group,role}`, `scale_down.{lambda,log_group,role}`, nullable `pool.{lambda,log_group,role}`, and provider-specific resources under `provider.<type>`. - Use caller-known optional wrappers for external AMI parameters and KMS keys. The wrapper determines Terraform graph shape while its `arn` leaf may remain unknown until apply. - Generate runner-stack, pool, job-retry, and EC2 IAM policies with `aws_iam_policy_document` and retain provider-policy merge behavior. - Document the experimental boundary, ownership model, plan-time wrapper pattern, phased migration, and nested output contract under the internal module documentation path. This draft is stacked on #5251 because the provider boundary consumes the experimental v2 normalization introduced there. Lambda/TypeScript terminology changes are tracked separately in #5258. ## Test Plan - `pre-commit run --all-files` — Terraform fmt, TFLint, validation, and merge-conflict checks passed. - `terraform test` in `modules/runner-stack` — 11 passed. - `terraform test` in `modules/multi-runner` — 7 passed. - `terraform test` in `modules/compute-providers/ec2` — 4 passed. - `terraform test` in `modules/compute-providers/ec2/runner-role` — 3 passed. - `terraform test` in `modules/runner-stack/pool` — 1 passed. - `terraform test` in `modules/runner-stack/job-retry` — 1 passed. - `terraform validate` in `modules/lambda` — passed. - Verified `modules/runners` has no diff from `origin/main`, stable v1 still dispatches only to `module.runners`, and only the experimental map dispatches to `module.runner_stacks`. - Verified computed external role, profile, AMI-parameter, managed-policy, and KMS ARN inputs plan successfully through the real wrapper fixture. No live AWS apply was performed. Terraform tests use mocked providers, and state migration is intentionally deferred to the later migration phase. ## Related Issues Closes #5252 Depends on #5251 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
edersonbrilhante
added a commit
that referenced
this pull request
Aug 12, 2026
…dules (#5257) ## Description - Keep `modules/runners` and stable `multi_runner_config` dispatch unchanged. Stable configurations retain their historical `module.runners["configuration"]` addresses and flat `runners_map` fields. - Add explicit opt-in through `experimental.multi_runner_config_v2`. Stable and experimental configurations can coexist when their keys do not overlap; duplicate keys are rejected. - Normalize stable v1 once for shared queues, webhook matching, and runner-binary discovery while routing only v2 configurations through `modules/runner-stack`. - Make `runner-stack` the provider-neutral control plane for scale-up, scale-down, pool, job retry, SSM housekeeping, common Lambda IAM, and runner-role ownership. - Keep EC2-specific launch templates, instance profiles, security groups, AMI/bootstrap resources, runner log groups, IAM fragments, and Lambda environment fragments under `modules/compute-providers/ec2`. - Define provider-owned runner-role requirements in EC2 and attach them to the common runner role in `runner-stack`, allowing future compute providers to supply different policies without duplicating the role lifecycle. - Replace flat runner-stack inputs with ownership-based nested objects. Logging configuration is grouped under `observability.logs`, including `level`, retention, encryption, class, and tags. - Pass the canonical `compute_provider.ec2` object and nested `runner`, `github`, `ssm`, and `observability` objects directly into the EC2 resource and runner-role policy modules instead of expanding them back into prefixed scalar inputs. - Layer module, shared-resource, component, subcomponent, and EC2 runtime tags with documented precedence; provider-required EC2 bootstrap tags retain final precedence. - Group experimental v2 outputs by ownership: `runner.role`, `scale_up.{lambda,log_group,role}`, `scale_down.{lambda,log_group,role}`, nullable `pool.{lambda,log_group,role}`, and provider-specific resources under `provider.<type>`. - Use caller-known optional wrappers for external AMI parameters and KMS keys. The wrapper determines Terraform graph shape while its `arn` leaf may remain unknown until apply. - Generate runner-stack, pool, job-retry, and EC2 IAM policies with `aws_iam_policy_document` and retain provider-policy merge behavior. - Document the experimental boundary, ownership model, plan-time wrapper pattern, phased migration, and nested output contract under the internal module documentation path. This draft is stacked on #5251 because the provider boundary consumes the experimental v2 normalization introduced there. Lambda/TypeScript terminology changes are tracked separately in #5258. ## Test Plan - `pre-commit run --all-files` — Terraform fmt, TFLint, validation, and merge-conflict checks passed. - `terraform test` in `modules/runner-stack` — 11 passed. - `terraform test` in `modules/multi-runner` — 7 passed. - `terraform test` in `modules/compute-providers/ec2` — 4 passed. - `terraform test` in `modules/compute-providers/ec2/runner-role` — 3 passed. - `terraform test` in `modules/runner-stack/pool` — 1 passed. - `terraform test` in `modules/runner-stack/job-retry` — 1 passed. - `terraform validate` in `modules/lambda` — passed. - Verified `modules/runners` has no diff from `origin/main`, stable v1 still dispatches only to `module.runners`, and only the experimental map dispatches to `module.runner_stacks`. - Verified computed external role, profile, AMI-parameter, managed-policy, and KMS ARN inputs plan successfully through the real wrapper fixture. No live AWS apply was performed. Terraform tests use mocked providers, and state migration is intentionally deferred to the later migration phase. ## Related Issues Closes #5252 Depends on #5251 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
edersonbrilhante
marked this pull request as draft
August 12, 2026 23:15
edersonbrilhante
changed the base branch from
main
to
refactor-ec2-provider-isolation
August 12, 2026 23:21
edersonbrilhante
added a commit
that referenced
this pull request
Aug 13, 2026
…dules (#5257) ## Description - Keep `modules/runners` and stable `multi_runner_config` dispatch unchanged. Stable configurations retain their historical `module.runners["configuration"]` addresses and flat `runners_map` fields. - Add explicit opt-in through `experimental.multi_runner_config_v2`. Stable and experimental configurations can coexist when their keys do not overlap; duplicate keys are rejected. - Normalize stable v1 once for shared queues, webhook matching, and runner-binary discovery while routing only v2 configurations through `modules/runner-stack`. - Make `runner-stack` the provider-neutral control plane for scale-up, scale-down, pool, job retry, SSM housekeeping, common Lambda IAM, and runner-role ownership. - Keep EC2-specific launch templates, instance profiles, security groups, AMI/bootstrap resources, runner log groups, IAM fragments, and Lambda environment fragments under `modules/compute-providers/ec2`. - Define provider-owned runner-role requirements in EC2 and attach them to the common runner role in `runner-stack`, allowing future compute providers to supply different policies without duplicating the role lifecycle. - Replace flat runner-stack inputs with ownership-based nested objects. Logging configuration is grouped under `observability.logs`, including `level`, retention, encryption, class, and tags. - Pass the canonical `compute_provider.ec2` object and nested `runner`, `github`, `ssm`, and `observability` objects directly into the EC2 resource and runner-role policy modules instead of expanding them back into prefixed scalar inputs. - Layer module, shared-resource, component, subcomponent, and EC2 runtime tags with documented precedence; provider-required EC2 bootstrap tags retain final precedence. - Group experimental v2 outputs by ownership: `runner.role`, `scale_up.{lambda,log_group,role}`, `scale_down.{lambda,log_group,role}`, nullable `pool.{lambda,log_group,role}`, and provider-specific resources under `provider.<type>`. - Use caller-known optional wrappers for external AMI parameters and KMS keys. The wrapper determines Terraform graph shape while its `arn` leaf may remain unknown until apply. - Generate runner-stack, pool, job-retry, and EC2 IAM policies with `aws_iam_policy_document` and retain provider-policy merge behavior. - Document the experimental boundary, ownership model, plan-time wrapper pattern, phased migration, and nested output contract under the internal module documentation path. This draft is stacked on #5251 because the provider boundary consumes the experimental v2 normalization introduced there. Lambda/TypeScript terminology changes are tracked separately in #5258. ## Test Plan - `pre-commit run --all-files` — Terraform fmt, TFLint, validation, and merge-conflict checks passed. - `terraform test` in `modules/runner-stack` — 11 passed. - `terraform test` in `modules/multi-runner` — 7 passed. - `terraform test` in `modules/compute-providers/ec2` — 4 passed. - `terraform test` in `modules/compute-providers/ec2/runner-role` — 3 passed. - `terraform test` in `modules/runner-stack/pool` — 1 passed. - `terraform test` in `modules/runner-stack/job-retry` — 1 passed. - `terraform validate` in `modules/lambda` — passed. - Verified `modules/runners` has no diff from `origin/main`, stable v1 still dispatches only to `module.runners`, and only the experimental map dispatches to `module.runner_stacks`. - Verified computed external role, profile, AMI-parameter, managed-policy, and KMS ARN inputs plan successfully through the real wrapper fixture. No live AWS apply was performed. Terraform tests use mocked providers, and state migration is intentionally deferred to the later migration phase. ## Related Issues Closes #5252 Depends on #5251 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
edersonbrilhante
force-pushed
the
experimental-multi-runner-config-v2-20260805
branch
2 times, most recently
from
August 13, 2026 11:41
4dd211d to
ca56c4c
Compare
edersonbrilhante
changed the title
This was referenced Aug 13, 2026
edersonbrilhante
force-pushed
the
refactor-ec2-provider-isolation
branch
from
August 14, 2026 12:32
6a5922a to
fd78317
Compare
edersonbrilhante
added a commit
that referenced
this pull request
Aug 14, 2026
…dules (#5257) ## Description - Keep `modules/runners` and stable `multi_runner_config` dispatch unchanged. Stable configurations retain their historical `module.runners["configuration"]` addresses and flat `runners_map` fields. - Add explicit opt-in through `experimental.multi_runner_config_v2`. Stable and experimental configurations can coexist when their keys do not overlap; duplicate keys are rejected. - Normalize stable v1 once for shared queues, webhook matching, and runner-binary discovery while routing only v2 configurations through `modules/runner-stack`. - Make `runner-stack` the provider-neutral control plane for scale-up, scale-down, pool, job retry, SSM housekeeping, common Lambda IAM, and runner-role ownership. - Keep EC2-specific launch templates, instance profiles, security groups, AMI/bootstrap resources, runner log groups, IAM fragments, and Lambda environment fragments under `modules/compute-providers/ec2`. - Define provider-owned runner-role requirements in EC2 and attach them to the common runner role in `runner-stack`, allowing future compute providers to supply different policies without duplicating the role lifecycle. - Replace flat runner-stack inputs with ownership-based nested objects. Logging configuration is grouped under `observability.logs`, including `level`, retention, encryption, class, and tags. - Pass the canonical `compute_provider.ec2` object and nested `runner`, `github`, `ssm`, and `observability` objects directly into the EC2 resource and runner-role policy modules instead of expanding them back into prefixed scalar inputs. - Layer module, shared-resource, component, subcomponent, and EC2 runtime tags with documented precedence; provider-required EC2 bootstrap tags retain final precedence. - Group experimental v2 outputs by ownership: `runner.role`, `scale_up.{lambda,log_group,role}`, `scale_down.{lambda,log_group,role}`, nullable `pool.{lambda,log_group,role}`, and provider-specific resources under `provider.<type>`. - Use caller-known optional wrappers for external AMI parameters and KMS keys. The wrapper determines Terraform graph shape while its `arn` leaf may remain unknown until apply. - Generate runner-stack, pool, job-retry, and EC2 IAM policies with `aws_iam_policy_document` and retain provider-policy merge behavior. - Document the experimental boundary, ownership model, plan-time wrapper pattern, phased migration, and nested output contract under the internal module documentation path. This draft is stacked on #5251 because the provider boundary consumes the experimental v2 normalization introduced there. Lambda/TypeScript terminology changes are tracked separately in #5258. ## Test Plan - `pre-commit run --all-files` — Terraform fmt, TFLint, validation, and merge-conflict checks passed. - `terraform test` in `modules/runner-stack` — 11 passed. - `terraform test` in `modules/multi-runner` — 7 passed. - `terraform test` in `modules/compute-providers/ec2` — 4 passed. - `terraform test` in `modules/compute-providers/ec2/runner-role` — 3 passed. - `terraform test` in `modules/runner-stack/pool` — 1 passed. - `terraform test` in `modules/runner-stack/job-retry` — 1 passed. - `terraform validate` in `modules/lambda` — passed. - Verified `modules/runners` has no diff from `origin/main`, stable v1 still dispatches only to `module.runners`, and only the experimental map dispatches to `module.runner_stacks`. - Verified computed external role, profile, AMI-parameter, managed-policy, and KMS ARN inputs plan successfully through the real wrapper fixture. No live AWS apply was performed. Terraform tests use mocked providers, and state migration is intentionally deferred to the later migration phase. ## Related Issues Closes #5252 Depends on #5251 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Use orchestration_provider consistently with compute_provider, centralize nested-module validation with terraform_data preconditions, and split stable-v1 from experimental-v2 routing tests.
edersonbrilhante
force-pushed
the
experimental-multi-runner-config-v2-20260805
branch
from
August 19, 2026 14:18
127c8f8 to
eb845d9
Compare
edersonbrilhante
changed the base branch from
refactor-ec2-provider-isolation
to
main
August 19, 2026 14:19
edersonbrilhante
added a commit
that referenced
this pull request
Aug 19, 2026
…dules (#5257) ## Description - Keep `modules/runners` and stable `multi_runner_config` dispatch unchanged. Stable configurations retain their historical `module.runners["configuration"]` addresses and flat `runners_map` fields. - Add explicit opt-in through `experimental.multi_runner_config_v2`. Stable and experimental configurations can coexist when their keys do not overlap; duplicate keys are rejected. - Normalize stable v1 once for shared queues, webhook matching, and runner-binary discovery while routing only v2 configurations through `modules/runner-stack`. - Make `runner-stack` the provider-neutral control plane for scale-up, scale-down, pool, job retry, SSM housekeeping, common Lambda IAM, and runner-role ownership. - Keep EC2-specific launch templates, instance profiles, security groups, AMI/bootstrap resources, runner log groups, IAM fragments, and Lambda environment fragments under `modules/compute-providers/ec2`. - Define provider-owned runner-role requirements in EC2 and attach them to the common runner role in `runner-stack`, allowing future compute providers to supply different policies without duplicating the role lifecycle. - Replace flat runner-stack inputs with ownership-based nested objects. Logging configuration is grouped under `observability.logs`, including `level`, retention, encryption, class, and tags. - Pass the canonical `compute_provider.ec2` object and nested `runner`, `github`, `ssm`, and `observability` objects directly into the EC2 resource and runner-role policy modules instead of expanding them back into prefixed scalar inputs. - Layer module, shared-resource, component, subcomponent, and EC2 runtime tags with documented precedence; provider-required EC2 bootstrap tags retain final precedence. - Group experimental v2 outputs by ownership: `runner.role`, `scale_up.{lambda,log_group,role}`, `scale_down.{lambda,log_group,role}`, nullable `pool.{lambda,log_group,role}`, and provider-specific resources under `provider.<type>`. - Use caller-known optional wrappers for external AMI parameters and KMS keys. The wrapper determines Terraform graph shape while its `arn` leaf may remain unknown until apply. - Generate runner-stack, pool, job-retry, and EC2 IAM policies with `aws_iam_policy_document` and retain provider-policy merge behavior. - Document the experimental boundary, ownership model, plan-time wrapper pattern, phased migration, and nested output contract under the internal module documentation path. This draft is stacked on #5251 because the provider boundary consumes the experimental v2 normalization introduced there. Lambda/TypeScript terminology changes are tracked separately in #5258. ## Test Plan - `pre-commit run --all-files` — Terraform fmt, TFLint, validation, and merge-conflict checks passed. - `terraform test` in `modules/runner-stack` — 11 passed. - `terraform test` in `modules/multi-runner` — 7 passed. - `terraform test` in `modules/compute-providers/ec2` — 4 passed. - `terraform test` in `modules/compute-providers/ec2/runner-role` — 3 passed. - `terraform test` in `modules/runner-stack/pool` — 1 passed. - `terraform test` in `modules/runner-stack/job-retry` — 1 passed. - `terraform validate` in `modules/lambda` — passed. - Verified `modules/runners` has no diff from `origin/main`, stable v1 still dispatches only to `module.runners`, and only the experimental map dispatches to `module.runner_stacks`. - Verified computed external role, profile, AMI-parameter, managed-policy, and KMS ARN inputs plan successfully through the real wrapper fixture. No live AWS apply was performed. Terraform tests use mocked providers, and state migration is intentionally deferred to the later migration phase. ## Related Issues Closes #5252 Depends on #5251 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
edersonbrilhante
force-pushed
the
experimental-multi-runner-config-v2-20260805
branch
from
August 25, 2026 17:22
799b3af to
eb845d9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds the opt-in provider-oriented
experimental.multi_runner_configpath while preserving stablemulti_runner_configbehavior and existingmodule.runners["configuration"]addresses.runner-stackand typed EC2 provider module.EC2 is the only Terraform-managed compute provider enabled here. The generic termination-watcher environment/KMS fix is excluded:
modules/termination-watchermatchesmain, and #5274 owns that fix.Test Plan
git diff --checkpassed.modules/termination-watcheris identical tomain.Related Issues
maindirectly.