diff --git a/.dockerignore b/.dockerignore index ad18704..91d0519 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,6 +1,5 @@ .git .github -.cursor .direnv .envrc .env* diff --git a/.golangci.yaml b/.golangci.yaml index dc97e0b..fd79597 100644 --- a/.golangci.yaml +++ b/.golangci.yaml @@ -355,3 +355,8 @@ issues: run: # Timeout for total work timeout: 5m + # Without these, every //go:build e2e and //go:build integration file is + # invisible to the linter, so the e2e and scenario suites go unchecked. + build-tags: + - e2e + - integration diff --git a/README.md b/README.md index cdd682e..e6d8daa 100644 --- a/README.md +++ b/README.md @@ -88,18 +88,16 @@ For how Deployah compares to DevSpace, Werf, Score, Epinio, and Kubero, see Know these before you invest time: -- **`env` is not applied yet.** The `env` field on a component passes schema - validation but does not reach the running container. Put runtime values in - your image or your app's own config for now. See +- **`env` is not applied to Deployments yet.** The `env` field on a component + passes schema validation but does not reach the running container. Task + `env` is inlined onto Jobs. See [Two kinds of variables](docs/configuration.md#two-kinds-of-variables). -- **`role: job` is not deployable yet.** It exists in the schema; only - `service` and `worker` deploy today. - **Deployah does not build images.** Give it an image that already exists in a registry your cluster can pull from. - **Stateful with persistence needs Kubernetes 1.32 or newer.** Deployah checks the API version and fails fast on older clusters. Identity-only stateful components have no such floor. -- **The schemas are alpha.** App manifests are at `v1-alpha.4` and platform +- **The schemas are alpha.** App manifests are at `v1-alpha.5` and platform files at `platform/v1-alpha.3`; expect breaking changes between releases. ## Contents @@ -193,7 +191,7 @@ Save this as `deployah.yaml` in an empty folder. It runs the public `nginx` image, so you do not need to build anything. ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: my-first-app components: web: @@ -335,15 +333,16 @@ A few words you will see often. - **Role.** What a component is for: - `service`: it serves traffic and can be exposed (the default). - `worker`: a long-running background task, not exposed. - - `job`: a one-off task that runs and then stops. +- **Task.** Run-to-completion work (`preDeploy`, `postDeploy`, or `manual`). + See [Tasks](docs/tasks.md). - **Kind.** The component's `kind` field: `stateless` (the default, easy to scale) or `stateful` (StatefulSet with stable identity; optional per-pod volumes). This field has nothing to do with Kind, the tool that runs the optional local cluster. See [Stateful workloads](docs/workloads.md#stateful-workloads) and [Storage classes](docs/platform.md#storage-classes). -- **Workload matrix.** `role` and `kind` combine independently. `job` is in - the schema but not deployable yet. +- **Workload matrix.** `role` and `kind` combine independently. Run-to-completion + work is `tasks:`, not a component role. | Capability | service+stateless | service+stateful | worker+stateless | worker+stateful | |---------------------|:-----------------:|:----------------:|:----------------:|:---------------:| @@ -396,6 +395,7 @@ The README covers the shape of the tool. The details live in `docs/`: | [Spec reference](docs/spec-reference.md) | Every `deployah.yaml` field, value rules, resource presets, and full examples. | | [Platform file](docs/platform.md) | Contexts, domains, TLS modes, storage classes, and profiles. | | [Workloads](docs/workloads.md) | Stateful components and volumes, workers, health checks, metrics. | +| [Tasks](docs/tasks.md) | Migrations, smoke checks, `deployah run`, and fanout. | | [Configuration](docs/configuration.md) | Environment selection, variables, `.env` files, precedence rules. | | [Networking](docs/networking.md) | Reaching your app, and how the local cluster resolves hostnames. | | [Custom manifests and CRDs](docs/custom-manifests-and-crds.md) | Ship plain Kubernetes YAML alongside the release. | @@ -437,6 +437,7 @@ These work with every command: | `deployah resolve --environments` | List every environment from both files: where it is registered, its context (or the kubeconfig fallback), domains, and overrides. | | `deployah plan ` | Preview what a deploy would change, without applying anything. Extra manifests from `.deployah/manifests/` appear in the diff; pending CRDs are reported but not applied. Use `--offline` to render with no cluster access, `--raw` for raw Kubernetes field paths instead of the compact Deployah vocabulary, `--yaml` to show changed fields as YAML blocks, `--drift` to also compare against live cluster state, `--detailed-exitcode` to exit 2 when changes are pending, or `--output json` for CI. | | `deployah deploy ` | Deploy your project. Shows the plan and asks for confirmation before applying; use `-y`/`--yes` to skip the prompt, `--reapply` to upgrade even with no changes, `--crds` for [CRD install policy](docs/custom-manifests-and-crds.md#crd-policy) (`create` or `create-replace`), `--explain` to print the resolution report first, `--force-hostname-change` to bypass the hostname guard, or `--resize-volumes` to grow [persistence](docs/workloads.md#growing-volumes) sizes. | +| `deployah run ` | Run a spec task as a one-off Job. Wait is the default; `--detach` returns after create. `--count` / `--parallelism` override fanout for that run. | | `deployah status ` | Show the status of a deployed project. Use `--detailed` for pod details, `-e` for an environment. | | `deployah logs ` | Stream logs. Filter with `--component`, `-e`, `--container`, `--since`, `--tail`. Use `--no-follow` for a one-off read. | | `deployah shell ` | Open a shell in a running container. Choose with `--component` and `--container`. | @@ -456,9 +457,9 @@ These work with every command: Deployah validates your spec and platform file with JSON Schema. -- **Manifest schema version:** v1-alpha.4 -- **Manifest schema:** `internal/spec/schema/v1-alpha.4/manifest.json` -- **Manifest environments schema:** `internal/spec/schema/v1-alpha.4/environments.json` +- **Manifest schema version:** v1-alpha.5 +- **Manifest schema:** `internal/spec/schema/v1-alpha.5/manifest.json` +- **Manifest environments schema:** `internal/spec/schema/v1-alpha.5/environments.json` - **Platform schema version:** platform/v1-alpha.3 - **Platform schema:** `internal/spec/schema/platform/v1-alpha.3/platform.json` diff --git a/docs/cli/deployah.md b/docs/cli/deployah.md index c88968b..98c2b02 100644 --- a/docs/cli/deployah.md +++ b/docs/cli/deployah.md @@ -20,7 +20,7 @@ deployah [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO @@ -33,6 +33,7 @@ deployah [flags] * [deployah logs](deployah_logs.md) - View logs for a deployed project * [deployah plan](deployah_plan.md) - Preview the changes a deploy would make * [deployah resolve](deployah_resolve.md) - Show the fully resolved configuration for an environment +* [deployah run](deployah_run.md) - Run a spec task as a one-off Job * [deployah shell](deployah_shell.md) - Connect to a shell in a container * [deployah status](deployah_status.md) - Display the status of a project * [deployah validate](deployah_validate.md) - Validate a Deployah spec diff --git a/docs/cli/deployah_cluster.md b/docs/cli/deployah_cluster.md index 3be14eb..c976e86 100644 --- a/docs/cli/deployah_cluster.md +++ b/docs/cli/deployah_cluster.md @@ -21,7 +21,7 @@ deployah cluster [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_cluster_down.md b/docs/cli/deployah_cluster_down.md index cdb25ee..50f9a3c 100644 --- a/docs/cli/deployah_cluster_down.md +++ b/docs/cli/deployah_cluster_down.md @@ -26,7 +26,7 @@ deployah cluster down [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_cluster_kubeconfig.md b/docs/cli/deployah_cluster_kubeconfig.md index f9b983a..d21b471 100644 --- a/docs/cli/deployah_cluster_kubeconfig.md +++ b/docs/cli/deployah_cluster_kubeconfig.md @@ -26,7 +26,7 @@ deployah cluster kubeconfig [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_cluster_status.md b/docs/cli/deployah_cluster_status.md index f852ff9..60e8d03 100644 --- a/docs/cli/deployah_cluster_status.md +++ b/docs/cli/deployah_cluster_status.md @@ -26,7 +26,7 @@ deployah cluster status [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_cluster_up.md b/docs/cli/deployah_cluster_up.md index cc3d69c..00d0414 100644 --- a/docs/cli/deployah_cluster_up.md +++ b/docs/cli/deployah_cluster_up.md @@ -34,7 +34,7 @@ deployah cluster up [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_delete.md b/docs/cli/deployah_delete.md index e4f3424..5898f33 100644 --- a/docs/cli/deployah_delete.md +++ b/docs/cli/deployah_delete.md @@ -4,7 +4,7 @@ Delete a deployed project in an environment ### Synopsis -Delete (uninstall) a deployed project in an environment from the Kubernetes cluster. +Delete (uninstall) a deployed project in an environment from the Kubernetes cluster. Also deletes leftover Jobs labeled for the project and environment, including CLI runs. --dry-run lists those Jobs even when the Helm release is already gone. ```text deployah delete [flags] @@ -18,7 +18,7 @@ deployah delete [flags] -o, --output string Output format for dry-run preview (default "tree") --show-resources Show detailed resources that would be deleted (implies --dry-run) --wait Wait until all Kubernetes resources are fully deleted before returning (uses stable legacy polling; suitable for CI) - -y, --yes Skip confirmation prompt and continue even if the release is not found + -y, --yes Skip confirmation prompt ``` ### Options inherited from parent commands @@ -31,7 +31,7 @@ deployah delete [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_deploy.md b/docs/cli/deployah_deploy.md index 0c08aa1..f9a7445 100644 --- a/docs/cli/deployah_deploy.md +++ b/docs/cli/deployah_deploy.md @@ -31,7 +31,7 @@ deployah deploy [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_init.md b/docs/cli/deployah_init.md index 9eb819c..9bd03a8 100644 --- a/docs/cli/deployah_init.md +++ b/docs/cli/deployah_init.md @@ -32,7 +32,7 @@ deployah init [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_list.md b/docs/cli/deployah_list.md index b54cdb3..639f274 100644 --- a/docs/cli/deployah_list.md +++ b/docs/cli/deployah_list.md @@ -28,7 +28,7 @@ deployah list [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_logs.md b/docs/cli/deployah_logs.md index 5e08150..3811871 100644 --- a/docs/cli/deployah_logs.md +++ b/docs/cli/deployah_logs.md @@ -37,7 +37,7 @@ deployah logs [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_plan.md b/docs/cli/deployah_plan.md index 95e76c2..b07b626 100644 --- a/docs/cli/deployah_plan.md +++ b/docs/cli/deployah_plan.md @@ -32,7 +32,7 @@ deployah plan [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_resolve.md b/docs/cli/deployah_resolve.md index febefc3..871c05b 100644 --- a/docs/cli/deployah_resolve.md +++ b/docs/cli/deployah_resolve.md @@ -38,7 +38,7 @@ deployah resolve [environment] [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_run.md b/docs/cli/deployah_run.md new file mode 100644 index 0000000..9641972 --- /dev/null +++ b/docs/cli/deployah_run.md @@ -0,0 +1,37 @@ +## deployah run + +Run a spec task as a one-off Job + +### Synopsis + +Create a Kubernetes Job for a task from the spec. Works for preDeploy, postDeploy, and manual tasks. Runs only the named task; tasks listed in its after field are not run. Waits for completion unless --detach is set. + +```text +deployah run [flags] +``` + +### Options + +```text + --count int Override fanout count for this run + --detach Return after creating the Job without waiting for completion + --parallelism int Override how many copies may run at once + -y, --yes Run without an interactive confirmation prompt +``` + +### Options inherited from parent commands + +```text + --context string Kubernetes context to use (overrides the current context and any environment 'context' field) + -d, --debug Enable debug mode (verbose logging and keep temporary files) + -h, --help show help for this command + -k, --kubeconfig string Path to the kubeconfig file to use (defaults to standard kubeconfig resolution) + -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) + --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) + -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) +``` + +### SEE ALSO + +* [deployah](deployah.md) - Deployah turns a spec into a running release on Kubernetes (Spec-to-Release) diff --git a/docs/cli/deployah_shell.md b/docs/cli/deployah_shell.md index abd9a76..863dd94 100644 --- a/docs/cli/deployah_shell.md +++ b/docs/cli/deployah_shell.md @@ -31,7 +31,7 @@ deployah shell [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_status.md b/docs/cli/deployah_status.md index 820cc4d..5cf4e3e 100644 --- a/docs/cli/deployah_status.md +++ b/docs/cli/deployah_status.md @@ -28,7 +28,7 @@ deployah status [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_validate.md b/docs/cli/deployah_validate.md index 84e58af..4a5fe7c 100644 --- a/docs/cli/deployah_validate.md +++ b/docs/cli/deployah_validate.md @@ -20,7 +20,7 @@ deployah validate [environment] [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/cli/deployah_version.md b/docs/cli/deployah_version.md index 1a70bbd..8480d80 100644 --- a/docs/cli/deployah_version.md +++ b/docs/cli/deployah_version.md @@ -22,7 +22,7 @@ deployah version [flags] -n, --namespace string Kubernetes namespace to use for Deployah operations (defaults to current context namespace) --platform-file string Path to the platform config file (overrides DEPLOYAH_PLATFORM_FILE and the default same-directory lookup) -s, --spec string Path to the Deployah spec file (YAML or JSON) (default "deployah.yaml") - -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete) (default 10m0s) + -t, --timeout duration Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run) (default 10m0s) ``` ### SEE ALSO diff --git a/docs/comparison.md b/docs/comparison.md index ef7fb9f..8bb5114 100644 --- a/docs/comparison.md +++ b/docs/comparison.md @@ -95,11 +95,15 @@ must install a platform into a cluster first (Epinio, Kubero). | **Local cluster included?** | **Yes** (`deployah cluster up`, kind) | No | No | No | No | No | | **Builds your image?** | No (you bring it) | Yes (and a dev loop) | Yes (Dockerfile/Stapel) | No (you bring it) | Yes (buildpacks) | Yes (buildpacks) | | **Output** | **Helm release** | Helm release | Helm release (via Nelm) | Raw YAML (no install) | Helm release (hidden) | K8s objects via operator | -| **Multi-component** (service/worker/job, stateless/stateful) | **Yes, named** (`kind: stateful` with per-pod PVCs; see [stateful workloads](workloads.md#stateful-workloads)) | Partial | No (you template each) | No (one workload per file) | No (web apps) | web/worker/cron, plus DB add-ons | +| **Multi-component** (service/worker, plus `tasks`) | **Yes, named** (`kind: stateful` with per-pod PVCs; see [stateful workloads](workloads.md#stateful-workloads); [tasks](tasks.md)) | Partial | No (you template each) | No (one workload per file) | No (web apps) | web/worker/cron, plus DB add-ons | | **Multiple environments** | **Yes** (own context, config, env, vars) | Partial (profiles and vars) | Yes (env name; you template the diffs) | No (the platform decides) | Namespaces only | Pipelines (up to 4 stages) | | **Installs and day-2** | **Yes** | Yes (and dev mode) | Yes (converge/plan/dismiss/status/logs) | No (you run `kubectl apply`) | Yes (and UI) | Yes (and UI) | | **Maturity (mid-2026)** | Early, independent | Mature, CNCF, ~4.9k★ | Mature, CNCF, Flant, ~4.7k★ | Mature spec, CNCF | Active, ~585★ | Active, ~4.3k★ | +`tasks:` is the Deployah equivalent of a Heroku release phase, a Fly +`release_command`, or a Cloud Run Job: migrate and smoke on deploy, and +one-off work via `deployah run`. See [Tasks](tasks.md). + ## Tool by tool ### DevSpace, the closest in design diff --git a/docs/configuration.md b/docs/configuration.md index d23a19d..b71bb90 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -46,9 +46,10 @@ It helps to know there are two different things: image tag or the ingress host. This works today and is described below. 2. **Container environment variables.** These are the variables your app reads at runtime. You would set them with the `env` field on a component. Note: - that field is accepted by the schema but is **not applied to the running - container yet** (it is planned). For now, put runtime values into your image - or your app's own config. + that field is accepted by the schema but is **not applied to Deployments + yet**. Task `env` (inherited from `from` or set on the task) **is** inlined + onto the Job. For components, put runtime values into your image or your + app's own config for now. ### Substitution variables diff --git a/docs/spec-reference.md b/docs/spec-reference.md index 44d38ba..8bee77b 100644 --- a/docs/spec-reference.md +++ b/docs/spec-reference.md @@ -1,9 +1,10 @@ # Spec reference Your spec is a file named `deployah.yaml`. It has three required parts: -`apiVersion`, `project`, and `components`. This page is the full field -reference, the value rules the validator enforces, the resource presets, and a -set of complete examples. +`apiVersion`, `project`, and `components`. Optional `tasks` hold +run-to-completion work. This page is the full field reference, the value +rules the validator enforces, the resource presets, and a set of complete +examples. `deployah.yaml` is the developer's half of the configuration: what to run, never where it runs. The other half is @@ -14,13 +15,13 @@ Here is a full example that shows the common fields. You do not need all of them; most have defaults. ```yaml -apiVersion: v1-alpha.4 # required: the schema version +apiVersion: v1-alpha.5 # required: the schema version project: shop # required: your project name components: # required: one or more components api: image: ghcr.io/acme/shop-api:${TAG} # tag comes from the environment below - role: service # service | worker | job (default: service) + role: service # service | worker (default: service) kind: stateless # stateless | stateful (default: stateless) port: 8080 # the port your app listens on (default: 8080) environments: [staging, prod] # which environments deploy this component @@ -77,9 +78,10 @@ Top level: | Field | Required | Notes | |---|---|---| -| `apiVersion` | Yes | The schema version. Must be `v1-alpha.4`. | +| `apiVersion` | Yes | The schema version. Must be `v1-alpha.5`. | | `project` | Yes | Lowercase name (DNS-1123). Prefixes your Kubernetes resources. | | `components` | Yes | A map of component name to component settings. | +| `tasks` | No | A map of task name to run-to-completion work. Names must not collide with component names. See [Tasks](#tasks). | | `environments` | Yes in practice | Environment **overrides**: a map of environment name to per-environment settings (`variables`, `envFile`). Keys support prefix-based wildcard matching, e.g. a `review` key matches `--environment review/pr-123`. Which environments exist is owned by the platform file's registry. | Component: @@ -87,7 +89,7 @@ Component: | Field | Default | Notes | |---|---|---| | `image` | none | The container image to run. You provide this. | -| `role` | `service` | `service` or `worker` (`job` is accepted by the schema but not deployable yet). | +| `role` | `service` | `service` or `worker`. | | `kind` | `stateless` | `stateless` or `stateful`. | | `port` | `8080` (services) | App listen port (1 to 65535). Not allowed on workers. | | `command` / `args` | none | Override the image ENTRYPOINT and CMD. | @@ -105,10 +107,37 @@ Component: | `profiles` | none | List of platform profile names. Merged left to right. See [Profiles](platform.md#profiles). | > [!IMPORTANT] -> Not deployed yet: the schema accepts `role: job`, and the `env`, `envFile`, -> and `configFile` fields, but Deployah does not apply them at deploy time -> yet. Changing `role` between `service` and `worker` on an existing release -> is rejected; delete the release and redeploy. +> Component `env`, `envFile`, and `configFile` are not applied to Deployments +> yet. Task `env` **is** inlined onto the Job. Changing `role` between +> `service` and `worker` on an existing release is rejected; delete the +> release and redeploy. + +## Tasks + +Quote `"on"` in YAML 1.1 so parsers do not treat it as a boolean. `on` is a +single value, not a list. See [Tasks](tasks.md) for how-to examples. + +| Field | Default | Notes | +|---|---|---| +| `from` | none | Component to inherit env, environments, profiles, and resources from. Also copies envFile and configFile paths. | +| `image` | from `from` | Replaces the parent image when set. `from` and/or `image` is required. | +| `command` / `args` | none | `command` is required when using the parent image. | +| `"on"` | none (required) | `preDeploy`, `postDeploy`, or `manual`. | +| `after` | none | Task names in the **same** `on` that must finish first. The dependency must be active in every environment the dependent is. Not allowed on `manual`. | +| `env` | inherited | Overlay on the parent map. Inlined onto the Job. | +| `envFile` / `configFile` | inherited | Inherited as fields; not mounted in this release. | +| `environments` | inherited | Replaces the parent filter when set. | +| `profiles` | inherited | Replaces the parent list when set. Applied to the Job pod (node selector, tolerations, security context). | +| `resourcePreset` / `resources` | inherited | Same rules as components. | +| `fanout` | count 1, parallelism 1 | Integer (`fanout: 4`) or `{count, parallelism}`. Applies to every `on`. `parallelism` must be `<= count` and at most 100000 (Kubernetes Indexed Job limit). | +| `timeout` | `5m` for hooks | Duration such as `5m`. Hook timeout must be less than the session `--timeout` at deploy or run time (default `10m`). Raise `--timeout` for a longer hook. No default for `manual`. | +| `backoffLimit` | `3` | Retries before the run is marked failed. | +| `ttlSecondsAfterFinished` | none (CLI runs: 7 days) | Seconds to keep a finished run. | + +`deployah run ` creates a Job for any task. It runs only +that task, not the tasks in its `after` list. Hook and CLI Jobs use the +default ServiceAccount in the release namespace. An exported chart overrides +a task the same way as a component (`--set migrate.image.tag=1.2.4`). Environment: @@ -190,7 +219,7 @@ Every example below is complete and valid. Copy one and change the values. **Smallest spec.** One service, one environment. ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: hello components: web: @@ -203,7 +232,7 @@ environments: **Two components.** A web app and an API in one project. ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: web: @@ -222,7 +251,7 @@ environments: from the platform file, not from here. ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: web: @@ -244,7 +273,7 @@ comes from the platform file. Set `subdomain` only when you want a different label, and `apex: true` for the bare domain. ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: web: @@ -257,7 +286,7 @@ components: **Set exact resources.** Use `resources` instead of a preset. ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: web: @@ -274,7 +303,7 @@ environments: **Autoscale on CPU.** Scale between 2 and 6 replicas at 70% CPU. ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: web: diff --git a/docs/tasks.md b/docs/tasks.md new file mode 100644 index 0000000..2fdac24 --- /dev/null +++ b/docs/tasks.md @@ -0,0 +1,131 @@ +# Tasks + +Run-to-completion work such as migrations, smoke checks, and one-off backfills. +You declare tasks next to components in `deployah.yaml`. Deployah runs hook +tasks around deploy, and you run any task yourself with `deployah run`. + +## Add a migrate task + +Point the task at a component with `from` so it reuses that image and env. +Set `"on": preDeploy` so it runs before the app starts on every install and +upgrade. Quote `"on"` in YAML 1.1 so it is not read as a boolean. + +```yaml +apiVersion: v1-alpha.5 +project: shop +components: + api: + image: ghcr.io/acme/shop:1.2.3 + env: + DATABASE_URL: ${DATABASE_URL} +tasks: + migrate: + from: api + "on": preDeploy + command: ["migrate", "up"] +``` + +`from` copies env, environments, profiles, and resources. It also copies +envFile and configFile paths, but those files are not mounted on the Job +yet (same as components). It does not copy command, args, or service fields +such as port. Profiles apply to the Job: node selector, tolerations, and +security context. Task `env` overlays the parent map. Runtime secrets for +tasks go in `env:` (inherited or overlay). `${...}` substitution works the +same as elsewhere in the spec. + +`command` is required when the task uses the parent image. If you set `image` +on the task, command is optional. + +## Run a smoke check after deploy + +```yaml +tasks: + smoke: + from: api + "on": postDeploy + command: ["curl", "-f", "http://api/health"] +``` + +`on` is one value: `preDeploy`, `postDeploy`, or `manual`. To run the same +command before and after deploy, define two tasks that share `from`. + +`after` orders tasks **inside the same `on`**. The named task must also run in +every environment the dependent runs in. Cross-phase `after` is an error. +`after` is not allowed on `manual` tasks. + +## Run a task yourself + +`deployah run` works for every task, including hooks you want to retry. +It runs only that task, not the tasks in its `after` list. + +```sh +deployah run backfill production --yes +``` + +Manual tasks exist only for the CLI. They are not part of the Helm release. + +```yaml +tasks: + backfill: + from: api + "on": manual + command: ["backfill"] +``` + +Wait is the default. `--detach` returns after the Job is created. Concurrent +runs are allowed; each run gets a unique Job name. + +## Fanout + +Fanout runs several indexed copies of a task. Use a number as a shortcut +(count, one at a time) or an object. It works on `preDeploy`, `postDeploy`, +and `manual`. + +```yaml +tasks: + migrate: + from: api + "on": preDeploy + command: ["migrate", "up"] + fanout: 2 # two copies on every deploy + backfill: + from: api + "on": manual + command: ["backfill"] + fanout: 4 # count 4, parallelism 1 + # fanout: + # count: 4 + # parallelism: 2 +``` + +`--count` and `--parallelism` on `deployah run` override that execution only. +Each copy sees `JOB_COMPLETION_INDEX` (0, 1, 2, ...). Parallelism is capped at +count and cannot exceed 100000 (the Kubernetes Indexed Job limit). + +## First install and the database + +On a first install, `preDeploy` runs **before** Deployments and Services. A +migrate task that talks to Postgres needs that database already reachable +(another release, a managed DB, or a job you ran first). `deployah plan` +prints this reminder on a fresh install. + +## Logs + +```sh +deployah logs shop --component=migrate --no-follow +deployah logs shop --component=backfill --no-follow +``` + +The component label is the task name. Finished Job pods are included, not +only running ones. + +## Rollback + +A Helm rollback does **not** run tasks. Failed hook Jobs are kept so you can +read logs. Migrations that already ran are not reverted; write a down +migration and `deployah run` it if you need that. + +## See also + +- [Spec reference](spec-reference.md#tasks) for every field +- [Troubleshooting](troubleshooting.md) for a failed hook or a tight timeout diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index c6ad322..32e1c82 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -32,6 +32,31 @@ error: variable ${IMAGE} not found Define the variable in the environment's `variables`, or in your env file or shell with the `DPY_VAR_` prefix. +**Hook task failed and was kept.** + +A failed `preDeploy` or `postDeploy` Job is not deleted. Read it with +`deployah logs --component= --no-follow`, fix the command or +the database, then deploy again. Helm recreates the Job (`before-hook-creation`). + +**Deploy timed out while hooks were still running.** + +Hook timeout defaults to `5m` and must be less than the `--timeout` used for +that deploy (default `10m`). Increase `--timeout` so it stays above every hook +timeout. A spec may set a hook timeout longer than the default `10m`; deploy +then needs a matching `--timeout`. Deployah does not raise the flag for you. +Serial hooks can add up to more than `--timeout`; plan shows each hook timeout +so you can see the budget. + +**A task did not run on deploy.** + +`"on": manual` tasks only run via `deployah run`. Hook tasks skipped for this +environment have an `environments` filter that does not match. + +**preDeploy cannot reach the database on first install.** + +On a first install, `preDeploy` runs before Deployments and Services. The +database must already be reachable. See [Tasks](tasks.md#first-install-and-the-database). + **Cannot connect to Kubernetes.** ```sh diff --git a/docs/workloads.md b/docs/workloads.md index f446238..0ecdb12 100644 --- a/docs/workloads.md +++ b/docs/workloads.md @@ -2,6 +2,8 @@ How Deployah turns a component into a Kubernetes workload: stateful sets and volumes, background workers, health checks, and Prometheus metrics. +Run-to-completion work lives under `tasks:` (see [Tasks](tasks.md)), not as +a component role. ## Stateful workloads @@ -29,7 +31,7 @@ not require that floor. `size` and `mountPath` are required: ```yaml -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: # Identity only: stable DNS / ordinals, no PVC diff --git a/examples/nginx/deployah.yaml b/examples/nginx/deployah.yaml index 9cc591f..9c9ff4d 100644 --- a/examples/nginx/deployah.yaml +++ b/examples/nginx/deployah.yaml @@ -1,4 +1,4 @@ -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: nginx components: web: diff --git a/internal/action/deploy_test.go b/internal/action/deploy_test.go index 6e6d60e..b629501 100644 --- a/internal/action/deploy_test.go +++ b/internal/action/deploy_test.go @@ -34,7 +34,7 @@ func (m *mockSpecLoader) Spec(_ context.Context, _ string) (*spec.Spec, error) { } var testManifest = &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "my-app", } diff --git a/internal/cmd/delete/delete.go b/internal/cmd/delete/delete.go index 80ca5af..8c867d4 100644 --- a/internal/cmd/delete/delete.go +++ b/internal/cmd/delete/delete.go @@ -3,6 +3,7 @@ package delete import ( "errors" "fmt" + "slices" "strings" "nabat.dev/nabat" @@ -11,6 +12,7 @@ import ( "deployah.dev/deployah/internal/cli" "deployah.dev/deployah/internal/cmd/cmdopts" "deployah.dev/deployah/internal/helm" + "deployah.dev/deployah/internal/k8s" "deployah.dev/deployah/internal/session" "deployah.dev/deployah/internal/spec" @@ -49,17 +51,21 @@ type DeletePreview struct { Revision int `json:"revision" yaml:"revision"` LastDeployed string `json:"lastDeployed" yaml:"lastDeployed"` Resources []ResourceInfo `json:"resources,omitempty" yaml:"resources,omitempty"` + // Jobs are leftover CLI or hook Jobs labeled for this project and + // environment. Live delete removes them even when the Helm release is + // already gone. + Jobs []string `json:"jobs,omitempty" yaml:"jobs,omitempty"` } // Register adds the delete command to app. func Register(app *nabat.App) { app.MustCommand("delete", nabat.WithDescription("Delete a deployed project in an environment"), - nabat.WithLongDescription("Delete (uninstall) a deployed project in an environment from the Kubernetes cluster."), + nabat.WithLongDescription("Delete (uninstall) a deployed project in an environment from the Kubernetes cluster. Also deletes leftover Jobs labeled for the project and environment, including CLI runs. --dry-run lists those Jobs even when the Helm release is already gone."), nabat.WithAliases("uninstall", "remove"), nabat.WithArg("project", "", nabat.WithRequired(), nabat.WithUsage("Project name to delete"), nabat.WithPrompt("Project name", "", nabat.WithHint("e.g. my-app"))), nabat.WithArg("environment", "", nabat.WithRequired(), nabat.WithUsage("Environment to delete from"), nabat.WithPrompt("Environment", "", nabat.WithHint("e.g. production"))), - nabat.WithFlag("yes", false, nabat.WithShort('y'), nabat.WithUsage("Skip confirmation prompt and continue even if the release is not found")), + nabat.WithFlag("yes", false, nabat.WithShort('y'), nabat.WithUsage("Skip confirmation prompt")), nabat.WithFlag("dry-run", false, nabat.WithUsage("Simulate the deletion without actually removing the project")), nabat.WithFlag("show-resources", false, nabat.WithUsage("Show detailed resources that would be deleted (implies --dry-run)")), nabat.WithSelectFlag("output", cli.OutputFormatTree, cli.DeleteOutputFormats, nabat.WithShort('o'), nabat.WithUsage("Output format for dry-run preview")), @@ -134,29 +140,31 @@ func runDelete(c *nabat.Context) error { c.Logger().Debug("checking project status", "project", opts.Project, "environment", opts.Environment) release, err := helmClient.GetRelease(c, opts.Project, opts.Environment) if err != nil { - if errors.Is(err, helm.ErrReleaseNotFound) { - c.Warn("Project not found", "project", opts.Project, "environment", opts.Environment) - if !opts.Yes { - return fmt.Errorf("project '%s' in environment '%s': %w — use --yes to ignore", opts.Project, opts.Environment, helm.ErrReleaseNotFound) - } - c.Info("Continuing with --yes despite missing project", "project", opts.Project) - } else { + if !errors.Is(err, helm.ErrReleaseNotFound) { return fmt.Errorf("check project status: %w", err) } + release = nil + } + + jobs, jobErr := listLabeledJobNames(c, cluster, opts.Project, opts.Environment) + if jobErr != nil { + return jobErr } if opts.DryRun { - return renderDryRunPreview(c, opts.Project, opts.Environment, release, opts.ShowResources, opts.Output) + return renderDryRunPreview(c, opts.Project, opts.Environment, release, jobs, opts.ShowResources, opts.Output) + } + + if nothingToDelete(release, jobs) { + c.Warn("Project not found, nothing to delete", "project", opts.Project, "environment", opts.Environment) + return nil } targetCtx := cluster.Context() if fallback, current := cluster.ContextFallback(); fallback { targetCtx = current } - prompt := fmt.Sprintf("Delete project '%s' in environment '%s'?", opts.Project, opts.Environment) - if targetCtx != "" { - prompt = fmt.Sprintf("Delete project '%s' in environment '%s' (context: %s)?", opts.Project, opts.Environment, targetCtx) - } + prompt := deleteConfirmPrompt(opts.Project, opts.Environment, targetCtx, release, jobs) confirmed, confirmErr := c.Confirm( prompt, nabat.WithAffirmative("Yes, delete it"), @@ -172,27 +180,63 @@ func runDelete(c *nabat.Context) error { return nil } - err = c.Spinner( - func(_ *nabat.Spinner) error { - return helmClient.DeleteRelease(c, opts.Project, opts.Environment, opts.Wait) - }, - nabat.WithTitle(fmt.Sprintf("Deleting '%s' in '%s'...", opts.Project, opts.Environment)), - ) - if err != nil { - return fmt.Errorf("delete release: %w", err) + if release != nil { + err = c.Spinner( + func(_ *nabat.Spinner) error { + return helmClient.DeleteRelease(c, opts.Project, opts.Environment, opts.Wait) + }, + nabat.WithTitle(fmt.Sprintf("Deleting '%s' in '%s'...", opts.Project, opts.Environment)), + ) + if err != nil { + return fmt.Errorf("delete release: %w", err) + } + } + + if delErr := deleteLabeledJobs(c, cluster, opts.Project, opts.Environment); delErr != nil { + return delErr } c.Success("Deleted", "project", opts.Project, "environment", opts.Environment) return nil } -func renderDryRunPreview(c *nabat.Context, project, environment string, release *v1.Release, showResources bool, format string) error { - if release == nil { - c.Warn("DRY RUN: Project not found — nothing to delete", "project", project, "environment", environment) +func deleteLabeledJobs(c *nabat.Context, cluster *session.Cluster, project, environment string) error { + cs, err := cluster.Kubernetes() + if err != nil { + c.Warn("Kubernetes client unavailable; leftover Jobs were not deleted", "err", err) + return nil + } + if err = k8s.DeleteJobs(c, cs, cluster.Namespace(), project, environment); err != nil { + return fmt.Errorf("delete leftover jobs: %w", err) + } + return nil +} + +func listLabeledJobNames(c *nabat.Context, cluster *session.Cluster, project, environment string) ([]string, error) { + cs, err := cluster.Kubernetes() + if err != nil { + c.Warn("Kubernetes client unavailable; leftover Jobs were not listed", "err", err) + return nil, nil + } + jobs, err := k8s.ListJobs(c, cs, cluster.Namespace(), project, environment) + if err != nil { + return nil, fmt.Errorf("list leftover jobs: %w", err) + } + names := make([]string, 0, len(jobs)) + for i := range jobs { + names = append(names, jobs[i].Name) + } + slices.Sort(names) + return names, nil +} + +func renderDryRunPreview(c *nabat.Context, project, environment string, release *v1.Release, jobs []string, showResources bool, format string) error { + if nothingToDelete(release, jobs) { + c.Warn("DRY RUN: Project not found, nothing to delete", "project", project, "environment", environment) return nil } - preview := buildPreview(project, environment, release, showResources) + preview := buildPreview(project, environment, release, jobs, showResources) switch format { case cli.OutputFormatJSON: @@ -204,15 +248,21 @@ func renderDryRunPreview(c *nabat.Context, project, environment string, release } } -func buildPreview(project, environment string, release *v1.Release, showResources bool) *DeletePreview { +func buildPreview(project, environment string, release *v1.Release, jobs []string, showResources bool) *DeletePreview { p := &DeletePreview{ - Project: project, - Environment: environment, - Release: release.Name, - Namespace: release.Namespace, - Status: "unknown", - LastDeployed: "unknown", + Project: project, + Environment: environment, + Jobs: jobs, } + if release == nil { + p.Status = "not found" + p.LastDeployed = "unknown" + return p + } + p.Release = release.Name + p.Namespace = release.Namespace + p.Status = "unknown" + p.LastDeployed = "unknown" if release.Info != nil { p.Status = release.Info.Status.String() if !release.Info.LastDeployed.IsZero() { @@ -228,25 +278,65 @@ func buildPreview(project, environment string, release *v1.Release, showResource return p } -func renderTree(c *nabat.Context, project, environment string, preview *DeletePreview) error { - c.Warn("DRY RUN — no changes will be made") +func nothingToDelete(release *v1.Release, jobs []string) bool { + return release == nil && len(jobs) == 0 +} + +func deleteConfirmPrompt(project, environment, targetCtx string, release *v1.Release, jobs []string) string { + var b strings.Builder + if release != nil { + fmt.Fprintf(&b, "Delete project '%s' in environment '%s'", project, environment) + } else { + fmt.Fprintf(&b, "Delete leftover Jobs for project '%s' in environment '%s': %s", + project, environment, strings.Join(jobs, ", ")) + } + if targetCtx != "" { + fmt.Fprintf(&b, " (context: %s)", targetCtx) + } + b.WriteByte('?') + return b.String() +} - children := []nabat.TreeNode{ - {Value: fmt.Sprintf("Release: %s", preview.Release)}, - {Value: fmt.Sprintf("Namespace: %s", preview.Namespace)}, - {Value: fmt.Sprintf("Status: %s", preview.Status)}, - {Value: fmt.Sprintf("Revision: %d", preview.Revision)}, - {Value: fmt.Sprintf("Last Deployed: %s", preview.LastDeployed)}, +func renderTree(c *nabat.Context, project, environment string, preview *DeletePreview) error { + c.Warn("DRY RUN, no changes will be made") + + var children []nabat.TreeNode + if preview.Release != "" { + children = []nabat.TreeNode{ + {Value: fmt.Sprintf("Release: %s", preview.Release)}, + {Value: fmt.Sprintf("Namespace: %s", preview.Namespace)}, + {Value: fmt.Sprintf("Status: %s", preview.Status)}, + {Value: fmt.Sprintf("Revision: %d", preview.Revision)}, + {Value: fmt.Sprintf("Last Deployed: %s", preview.LastDeployed)}, + } + } else { + children = []nabat.TreeNode{ + {Value: "Helm release: not found"}, + } } if len(preview.Resources) > 0 { children = append(children, buildResourceNodes(preview.Resources)) } + if len(preview.Jobs) > 0 { + leaves := make([]nabat.TreeNode, 0, len(preview.Jobs)) + for _, name := range preview.Jobs { + leaves = append(leaves, nabat.TreeNode{Value: name}) + } + children = append(children, nabat.TreeNode{ + Value: fmt.Sprintf("Leftover Jobs (%d)", len(preview.Jobs)), + Children: leaves, + }) + } root := fmt.Sprintf("%s (%s)", project, environment) c.Tree(root, children, nabat.WithTreeEnumerator(nabat.TreeRoundedEnumerator())) - c.Warn("This permanently deletes all resources and Helm release history") + if preview.Release != "" { + c.Warn("This permanently deletes all resources and Helm release history") + } else { + c.Warn("This deletes leftover Jobs labeled for this project and environment") + } c.Info("To perform the actual deletion, run without --dry-run", "command", fmt.Sprintf("deployah delete %s %s", project, environment), ) diff --git a/internal/cmd/delete/delete_test.go b/internal/cmd/delete/delete_test.go new file mode 100644 index 0000000..9e8e43b --- /dev/null +++ b/internal/cmd/delete/delete_test.go @@ -0,0 +1,291 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package delete + +import ( + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "helm.sh/helm/v4/pkg/release/common" + "nabat.dev/nabat" + "nabat.dev/nabat/nabattest" + + "deployah.dev/deployah/internal/cli" + + v1 "helm.sh/helm/v4/pkg/release/v1" +) + +func TestBuildPreview_LeftoverJobsWithoutRelease(t *testing.T) { + t.Parallel() + + p := buildPreview("shop", "dev", nil, []string{"shop-dev-backfill-abc"}, false) + assert.Equal(t, "shop", p.Project) + assert.Equal(t, "dev", p.Environment) + assert.Empty(t, p.Release) + assert.Equal(t, "not found", p.Status) + assert.Equal(t, []string{"shop-dev-backfill-abc"}, p.Jobs) +} + +func TestBuildPreview_ReleaseAndJobs(t *testing.T) { + t.Parallel() + + rel := &v1.Release{Name: "shop-dev", Namespace: "default", Version: 3} + p := buildPreview("shop", "dev", rel, []string{"shop-dev-backfill-abc", "shop-dev-migrate-xyz"}, false) + assert.Equal(t, "shop-dev", p.Release) + assert.Equal(t, "default", p.Namespace) + assert.Equal(t, 3, p.Revision) + assert.Equal(t, []string{"shop-dev-backfill-abc", "shop-dev-migrate-xyz"}, p.Jobs) +} + +func TestNothingToDelete(t *testing.T) { + t.Parallel() + + require.True(t, nothingToDelete(nil, nil)) + require.True(t, nothingToDelete(nil, []string{})) + require.False(t, nothingToDelete(nil, []string{"job"})) + require.False(t, nothingToDelete(&v1.Release{Name: "shop-dev"}, nil)) +} + +func TestBuildPreview_ReleaseInfoAndResources(t *testing.T) { + t.Parallel() + + deployed := time.Date(2026, 8, 16, 12, 0, 0, 0, time.UTC) + rel := &v1.Release{ + Name: "shop-dev", + Namespace: "default", + Version: 2, + Info: &v1.Info{ + Status: common.StatusDeployed, + LastDeployed: deployed, + }, + Manifest: strings.Join([]string{ + "apiVersion: apps/v1", + "kind: Deployment", + "metadata:", + " name: shop-dev-api", + "spec:", + " replicas: 2", + }, "\n"), + } + p := buildPreview("shop", "dev", rel, nil, true) + assert.Equal(t, "deployed", p.Status) + assert.Equal(t, "2026-08-16 12:00:00 UTC", p.LastDeployed) + require.Len(t, p.Resources, 1) + assert.Equal(t, "Deployment", p.Resources[0].Kind) + assert.Equal(t, "shop-dev-api", p.Resources[0].Name) + assert.Equal(t, "replicas: 2", p.Resources[0].Detail) +} + +func TestParseResources(t *testing.T) { + t.Parallel() + + manifest := strings.Join([]string{ + "", + "---", + "not: valid yaml: [", + "---", + "apiVersion: v1", + "kind: ConfigMap", + "metadata:", + " name: shop-config", + "---", + "apiVersion: apps/v1", + "kind: Deployment", + "metadata:", + " name: api", + "spec:", + " replicas: 3", + "---", + "apiVersion: v1", + "kind: Service", + "metadata:", + " name: api", + "spec:", + " type: ClusterIP", + " ports:", + " - port: 8080", + "---", + "apiVersion: v1", + "kind: Service", + "metadata:", + " name: bare", + "spec: {}", + "---", + "apiVersion: networking.k8s.io/v1", + "kind: Ingress", + "metadata:", + " name: api", + "spec:", + " rules:", + " - host: api.example.com", + "---", + "apiVersion: v1", + "kind: Secret", + "metadata:", + " name: tls", + "type: kubernetes.io/tls", + "---", + "apiVersion: v1", + "kind: Secret", + "metadata:", + " name: env", + "---", + "apiVersion: v1", + "kind: PersistentVolumeClaim", + "metadata:", + " name: data", + "spec:", + " resources:", + " requests:", + " storage: 10Gi", + "---", + "apiVersion: apps/v1", + "kind: StatefulSet", + "metadata:", + " name: db", + "spec:", + " replicas: 1", + }, "\n") + + got := parseResources(manifest) + require.Len(t, got, 9) + assert.Equal(t, ResourceInfo{APIVersion: "v1", Kind: "ConfigMap", Name: "shop-config"}, got[0]) + assert.Equal(t, "replicas: 3", got[1].Detail) + assert.Equal(t, "ClusterIP, port: 8080", got[2].Detail) + assert.Equal(t, "ClusterIP", got[3].Detail) + assert.Equal(t, "host: api.example.com", got[4].Detail) + assert.Equal(t, "kubernetes.io/tls", got[5].Detail) + assert.Equal(t, "Opaque", got[6].Detail) + assert.Equal(t, "storage: 10Gi", got[7].Detail) + assert.Equal(t, "replicas: 1", got[8].Detail) +} + +func TestRenderDryRunPreview(t *testing.T) { + t.Parallel() + + t.Run("nothing to delete", func(t *testing.T) { + t.Parallel() + c := nabatContext(t) + require.NoError(t, renderDryRunPreview(c, "shop", "dev", nil, nil, false, cli.OutputFormatTree)) + }) + + t.Run("leftover jobs tree", func(t *testing.T) { + t.Parallel() + c := nabatContext(t) + require.NoError(t, renderDryRunPreview(c, "shop", "dev", nil, []string{"shop-dev-backfill-abc"}, false, cli.OutputFormatTree)) + }) + + t.Run("release with resources", func(t *testing.T) { + t.Parallel() + c := nabatContext(t) + rel := &v1.Release{ + Name: "shop-dev", + Namespace: "default", + Version: 1, + Manifest: strings.Join([]string{ + "apiVersion: apps/v1", + "kind: Deployment", + "metadata:", + " name: api", + "spec:", + " replicas: 1", + "---", + "apiVersion: v1", + "kind: Service", + "metadata:", + " name: api", + "spec:", + " ports:", + " - port: 80", + }, "\n"), + } + require.NoError(t, renderDryRunPreview(c, "shop", "dev", rel, []string{"shop-dev-migrate-xyz"}, true, cli.OutputFormatTree)) + }) + + t.Run("json leftover jobs", func(t *testing.T) { + t.Parallel() + c := nabatContext(t) + require.NoError(t, renderDryRunPreview(c, "shop", "dev", nil, []string{"job-a"}, false, cli.OutputFormatJSON)) + }) + + t.Run("yaml leftover jobs", func(t *testing.T) { + t.Parallel() + c := nabatContext(t) + require.NoError(t, renderDryRunPreview(c, "shop", "dev", nil, []string{"job-a"}, false, cli.OutputFormatYAML)) + }) +} + +func nabatContext(t *testing.T) *nabat.Context { + t.Helper() + io, _, _, _ := nabattest.NewIO() + app := nabat.MustNew("test", nabat.WithIO(io)) + return nabattest.Context(t, app) +} + +func TestDeleteConfirmPrompt(t *testing.T) { + t.Parallel() + + rel := &v1.Release{Name: "shop-dev"} + tests := []struct { + name string + project string + environment string + targetCtx string + release *v1.Release + jobs []string + want string + }{ + { + name: "release without context", + project: "shop", + environment: "dev", + release: rel, + want: "Delete project 'shop' in environment 'dev'?", + }, + { + name: "release with context", + project: "shop", + environment: "dev", + targetCtx: "kind-deployah", + release: rel, + jobs: []string{"shop-dev-backfill-abc"}, + want: "Delete project 'shop' in environment 'dev' (context: kind-deployah)?", + }, + { + name: "leftover jobs only", + project: "shop", + environment: "dev", + jobs: []string{"shop-dev-backfill-abc", "shop-dev-migrate-xyz"}, + want: "Delete leftover Jobs for project 'shop' in environment 'dev': shop-dev-backfill-abc, shop-dev-migrate-xyz?", + }, + { + name: "leftover jobs with context", + project: "shop", + environment: "dev", + targetCtx: "kind-deployah", + jobs: []string{"shop-dev-backfill-abc"}, + want: "Delete leftover Jobs for project 'shop' in environment 'dev': shop-dev-backfill-abc (context: kind-deployah)?", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got := deleteConfirmPrompt(tt.project, tt.environment, tt.targetCtx, tt.release, tt.jobs) + assert.Equal(t, tt.want, got) + }) + } +} diff --git a/internal/cmd/deploy/deploy.go b/internal/cmd/deploy/deploy.go index be5adac..334dc90 100644 --- a/internal/cmd/deploy/deploy.go +++ b/internal/cmd/deploy/deploy.go @@ -146,6 +146,18 @@ func runDeploy(c *nabat.Context) error { printExplain(c, resolvedSpec) } + effective, effErr := spec.EffectiveTasks(manifest, opts.Environment, resolvedSpec) + if effErr != nil { + return effErr + } + tasks := make(map[string]spec.Task, len(effective)) + for name, rt := range effective { + tasks[name] = rt.Task + } + if timeoutErr := spec.CheckHookTaskTimeouts(tasks, sess.Timeout()); timeoutErr != nil { + return timeoutErr + } + cluster, err := sess.Target(c, opts.Environment) if err != nil { return fmt.Errorf("target cluster: %w", err) diff --git a/internal/cmd/initialize/components.go b/internal/cmd/initialize/components.go index 9101126..e36c7f5 100644 --- a/internal/cmd/initialize/components.go +++ b/internal/cmd/initialize/components.go @@ -27,7 +27,6 @@ func validateComponentNameUnique(name string, existing map[string]spec.Component var roleOrder = []spec.ComponentRole{ spec.ComponentRoleService, spec.ComponentRoleWorker, - spec.ComponentRoleJob, } // roleLabels maps each role to a select label with a short explanation and @@ -36,7 +35,6 @@ var roleOrder = []spec.ComponentRole{ var roleLabels = map[spec.ComponentRole]string{ spec.ComponentRoleService: "service - handles HTTP requests (web apps, APIs)", spec.ComponentRoleWorker: "worker - long-running background process, no HTTP (queue consumers)", - spec.ComponentRoleJob: "job - runs a task to completion, then exits (migrations, batch tasks)", } // roleFromLabel reverses roleLabels. It reports false when label does not diff --git a/internal/cmd/initialize/components_test.go b/internal/cmd/initialize/components_test.go index 72fb44a..7de379f 100644 --- a/internal/cmd/initialize/components_test.go +++ b/internal/cmd/initialize/components_test.go @@ -132,7 +132,7 @@ func TestKindFromLabel(t *testing.T) { } // TestNeedsHealthCheckQuestion verifies the health-check question follows -// [spec.Component.ListensOnPort]: worker and job components never get the +// [spec.Component.ListensOnPort]: worker components never get the // question, and a service component without a port doesn't either. func TestNeedsHealthCheckQuestion(t *testing.T) { t.Parallel() @@ -157,11 +157,6 @@ func TestNeedsHealthCheckQuestion(t *testing.T) { component: spec.Component{Role: spec.ComponentRoleWorker, Port: 8080}, want: false, }, - { - name: "job with a port set does not", - component: spec.Component{Role: spec.ComponentRoleJob, Port: 8080}, - want: false, - }, } for _, tt := range tests { diff --git a/internal/cmd/initialize/init_test.go b/internal/cmd/initialize/init_test.go index ed3bc3e..bb0ab09 100644 --- a/internal/cmd/initialize/init_test.go +++ b/internal/cmd/initialize/init_test.go @@ -19,7 +19,7 @@ func TestCheckOverwrite(t *testing.T) { t.Parallel() existing := filepath.Join(t.TempDir(), "deployah.yaml") - require.NoError(t, os.WriteFile(existing, []byte("apiVersion: v1-alpha.4\n"), 0o600)) + require.NoError(t, os.WriteFile(existing, []byte("apiVersion: v1-alpha.5\n"), 0o600)) missing := filepath.Join(t.TempDir(), "missing.yaml") tests := []struct { diff --git a/internal/cmd/initialize/noninteractive_test.go b/internal/cmd/initialize/noninteractive_test.go index 149df40..312710f 100644 --- a/internal/cmd/initialize/noninteractive_test.go +++ b/internal/cmd/initialize/noninteractive_test.go @@ -56,7 +56,7 @@ func TestInit_DefaultsProducesValidSpec(t *testing.T) { func TestInit_DefaultsWithoutForceAgainstExistingFileFails(t *testing.T) { dir := t.TempDir() outputPath := filepath.Join(dir, "deployah.yaml") - require.NoError(t, os.WriteFile(outputPath, []byte("apiVersion: v1-alpha.4\n"), 0o600)) + require.NoError(t, os.WriteFile(outputPath, []byte("apiVersion: v1-alpha.5\n"), 0o600)) io, _, _, _ := nabattest.NewIO() app := newInitApp(io) @@ -70,7 +70,7 @@ func TestInit_DefaultsWithoutForceAgainstExistingFileFails(t *testing.T) { func TestInit_DefaultsWithForceAgainstExistingFileSucceeds(t *testing.T) { dir := t.TempDir() outputPath := filepath.Join(dir, "deployah.yaml") - require.NoError(t, os.WriteFile(outputPath, []byte("apiVersion: v1-alpha.4\n"), 0o600)) + require.NoError(t, os.WriteFile(outputPath, []byte("apiVersion: v1-alpha.5\n"), 0o600)) io, _, _, _ := nabattest.NewIO() app := newInitApp(io) diff --git a/internal/cmd/initialize/summary.go b/internal/cmd/initialize/summary.go index 6ad1dc7..cdbae8f 100644 --- a/internal/cmd/initialize/summary.go +++ b/internal/cmd/initialize/summary.go @@ -287,6 +287,9 @@ func buildValidatedSpec(config *ProjectConfig) (*spec.Spec, error) { if err = spec.ValidateSpecComponents(&specData); err != nil { return nil, fmt.Errorf("component validation failed: %w", err) } + if err = spec.ValidateSpecTasks(&specData); err != nil { + return nil, fmt.Errorf("task validation failed: %w", err) + } if err = spec.FillSpecWithDefaults(&specData, specData.APIVersion); err != nil { return nil, fmt.Errorf("failed to apply defaults to spec: %w", err) diff --git a/internal/cmd/initialize/summary_test.go b/internal/cmd/initialize/summary_test.go index d2fdd67..80e5d0c 100644 --- a/internal/cmd/initialize/summary_test.go +++ b/internal/cmd/initialize/summary_test.go @@ -30,7 +30,7 @@ func TestShowSummaryAndSave_RoleAwareComponentsProduceValidSpec(t *testing.T) { outputPath := filepath.Join(dir, "deployah.yaml") // One component per role the wizard offers: service (with a port and - // an HTTP health check), worker, and job. + // an HTTP health check) and worker. config := &ProjectConfig{ Name: "shop", EnvironmentNames: []string{"local"}, @@ -50,11 +50,6 @@ func TestShowSummaryAndSave_RoleAwareComponentsProduceValidSpec(t *testing.T) { Image: "shop/worker:1.0.0", ResourcePreset: spec.ResourcePresetSmall, }, - "migrate": { - Role: spec.ComponentRoleJob, - Image: "shop/migrate:1.0.0", - ResourcePreset: spec.ResourcePresetSmall, - }, }, OutputPath: outputPath, } diff --git a/internal/cmd/logs/logs.go b/internal/cmd/logs/logs.go index 6e6be55..5136d58 100644 --- a/internal/cmd/logs/logs.go +++ b/internal/cmd/logs/logs.go @@ -118,9 +118,9 @@ func runLogs(c *nabat.Context) error { return fmt.Errorf("parse label selector: %w", err) } - containerState, err := stern.NewContainerState(stern.RUNNING) + containerStates, err := logContainerStates() if err != nil { - return fmt.Errorf("invalid container-state %q: %w", containerState, err) + return err } funs := map[string]any{ @@ -164,7 +164,7 @@ func runLogs(c *nabat.Context) error { Template: tmpl, LabelSelector: labelSelector, FieldSelector: fields.Everything(), - ContainerStates: []stern.ContainerState{containerState}, + ContainerStates: containerStates, Follow: !opts.NoFollow, Resource: opts.Resource, OnlyLogLines: opts.OnlyLogLines, @@ -197,3 +197,15 @@ func runLogs(c *nabat.Context) error { } return nil } + +func logContainerStates() ([]stern.ContainerState, error) { + running, err := stern.NewContainerState(stern.RUNNING) + if err != nil { + return nil, fmt.Errorf("invalid container-state %q: %w", stern.RUNNING, err) + } + terminated, err := stern.NewContainerState(stern.TERMINATED) + if err != nil { + return nil, fmt.Errorf("invalid container-state %q: %w", stern.TERMINATED, err) + } + return []stern.ContainerState{running, terminated}, nil +} diff --git a/internal/cmd/logs/logs_test.go b/internal/cmd/logs/logs_test.go new file mode 100644 index 0000000..9e32c2e --- /dev/null +++ b/internal/cmd/logs/logs_test.go @@ -0,0 +1,71 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package logs + +import ( + "testing" + + "github.com/stern/stern/stern" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "nabat.dev/nabat" + "nabat.dev/nabat/nabattest" +) + +func TestLogContainerStates_IncludesTerminated(t *testing.T) { + t.Parallel() + + states, err := logContainerStates() + require.NoError(t, err) + require.Len(t, states, 2) + assert.Equal(t, stern.RUNNING, string(states[0])) + assert.Equal(t, stern.TERMINATED, string(states[1])) +} + +func TestRunLogs_FlagValidation(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + args []string + want string + }{ + { + name: "template and template-file together", + args: []string{"logs", "shop", "--template", "{{.Message}}", "--template-file", "log.tmpl"}, + want: "cannot specify both --template and --template-file", + }, + { + name: "resource missing name", + args: []string{"logs", "shop", "--resource", "job/"}, + want: "resource format must be", + }, + { + name: "resource missing slash", + args: []string{"logs", "shop", "--resource", "job"}, + want: "resource format must be", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + io, _, _, _ := nabattest.NewIO() + app := nabat.MustNew("deployah", nabat.WithIO(io)) + Register(app) + err := nabattest.Run(t, app, tt.args) + require.Error(t, err) + assert.Contains(t, err.Error(), tt.want) + }) + } +} diff --git a/internal/cmd/plan/plan_test.go b/internal/cmd/plan/plan_test.go index b6d0621..64ae0a5 100644 --- a/internal/cmd/plan/plan_test.go +++ b/internal/cmd/plan/plan_test.go @@ -212,7 +212,7 @@ data: ` func testManifest() *spec.Spec { - return &spec.Spec{Project: "web", APIVersion: "v1-alpha.4"} + return &spec.Spec{Project: "web", APIVersion: spec.CurrentManifestVersion} } func testOptions() *Options { diff --git a/internal/cmd/root.go b/internal/cmd/root.go index 85a65fd..a3e26b4 100644 --- a/internal/cmd/root.go +++ b/internal/cmd/root.go @@ -35,6 +35,7 @@ import ( "deployah.dev/deployah/internal/cmd/list" "deployah.dev/deployah/internal/cmd/logs" "deployah.dev/deployah/internal/cmd/resolve" + "deployah.dev/deployah/internal/cmd/run" "deployah.dev/deployah/internal/cmd/shell" "deployah.dev/deployah/internal/cmd/status" "deployah.dev/deployah/internal/cmd/validate" @@ -67,7 +68,7 @@ func NewApp(opts ...nabat.Option) *nabat.App { nabat.WithFlag("namespace", "", nabat.WithShort('n'), nabat.WithUsage("Kubernetes namespace to use for Deployah operations (defaults to current context namespace)"), nabat.WithPersistent()), nabat.WithFlag("kubeconfig", "", nabat.WithShort('k'), nabat.WithUsage("Path to the kubeconfig file to use (defaults to standard kubeconfig resolution)"), nabat.WithPersistent()), nabat.WithFlag("context", "", nabat.WithUsage("Kubernetes context to use (overrides the current context and any environment 'context' field)"), nabat.WithPersistent()), - nabat.WithFlag("timeout", session.DefaultTimeout, nabat.WithShort('t'), nabat.WithUsage("Timeout for Deployah operations (install/upgrade, list, status, logs, delete)"), nabat.WithPersistent()), + nabat.WithFlag("timeout", session.DefaultTimeout, nabat.WithShort('t'), nabat.WithUsage("Timeout for Deployah operations (install/upgrade, list, status, logs, delete, run)"), nabat.WithPersistent()), nabat.WithExtension(logging.New(logging.WithVerboseFlag("debug"))), // plan.ErrChangesPresent is a normal CI signal (exit code 2, see // Execute), not a failure, so it gets no error banner. Every other @@ -127,6 +128,7 @@ func NewApp(opts ...nabat.Option) *nabat.App { logs.Register(app) planCmd.Register(app) resolve.Register(app) + run.Register(app) shell.Register(app) status.Register(app) validate.Register(app) diff --git a/internal/cmd/run/doc.go b/internal/cmd/run/doc.go new file mode 100644 index 0000000..c1316aa --- /dev/null +++ b/internal/cmd/run/doc.go @@ -0,0 +1,19 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +// Package run implements the deployah run command. +// +// The command creates a one-off Kubernetes Job for a spec task, waits +// for it to finish unless --detach is set, and does not walk after +// dependencies. +package run diff --git a/internal/cmd/run/run.go b/internal/cmd/run/run.go new file mode 100644 index 0000000..bb7f52e --- /dev/null +++ b/internal/cmd/run/run.go @@ -0,0 +1,198 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package run + +import ( + "context" + "fmt" + "time" + + "k8s.io/client-go/kubernetes" + "nabat.dev/nabat" + + "deployah.dev/deployah/internal/cmd/cmdopts" + "deployah.dev/deployah/internal/k8s" + "deployah.dev/deployah/internal/session" + "deployah.dev/deployah/internal/spec" + + batchv1 "k8s.io/api/batch/v1" +) + +// Options holds command-line flags for run. +type Options struct { + Task string `nabat:"task"` + Environment string `nabat:"environment"` + Detach bool `nabat:"detach"` + Count int `nabat:"count"` + Parallelism int `nabat:"parallelism"` + Yes bool `nabat:"yes"` +} + +// Register adds the run command to app. +func Register(app *nabat.App) { + app.MustCommand("run", + nabat.WithDescription("Run a spec task as a one-off Job"), + nabat.WithLongDescription("Create a Kubernetes Job for a task from the spec. Works for preDeploy, postDeploy, and manual tasks. Runs only the named task; tasks listed in its after field are not run. Waits for completion unless --detach is set."), + nabat.WithArg("task", "", nabat.WithRequired(), nabat.WithUsage("Task name to run"), nabat.WithPrompt("Task", "", nabat.WithHint("e.g. migrate, backfill"))), + nabat.WithArg("environment", "", nabat.WithRequired(), nabat.WithUsage("Environment to run in"), nabat.WithPrompt("Environment", "", nabat.WithHint("e.g. prod, staging"))), + nabat.WithFlag("detach", false, nabat.WithUsage("Return after creating the Job without waiting for completion")), + nabat.WithFlag("count", 0, nabat.WithUsage("Override fanout count for this run")), + nabat.WithFlag("parallelism", 0, nabat.WithUsage("Override how many copies may run at once")), + nabat.WithFlag("yes", false, nabat.WithShort('y'), nabat.WithUsage("Run without an interactive confirmation prompt")), + nabat.WithExample(` +# Run a manual backfill and wait for it to finish +deployah run backfill production + +# Run without waiting +deployah run backfill production --detach + +# Override fanout for this run +deployah run backfill production --count 4 --parallelism 2`), + nabat.WithRun(runTask), + ) +} + +func runTask(c *nabat.Context) error { + opts := &Options{} + if err := c.Bind(opts); err != nil { + return fmt.Errorf("binding options: %w", err) + } + if opts.Count < 0 || opts.Parallelism < 0 { + return fmt.Errorf("--count and --parallelism must be zero or positive") + } + if opts.Parallelism > spec.MaxFanoutParallelism { + return fmt.Errorf("--parallelism must be at most %d", spec.MaxFanoutParallelism) + } + if opts.Count > 0 && opts.Parallelism > opts.Count { + return fmt.Errorf("--parallelism must be less than or equal to --count") + } + + sess := session.FromContext(c) + + platform, platformErr := sess.Platform() + if platformErr != nil { + return fmt.Errorf("load platform file: %w", platformErr) + } + + manifest, err := spec.Load(c, sess.SpecPath(), opts.Environment, platform) + if err != nil { + return fmt.Errorf("load spec: %w", err) + } + + rt, err := resolveRunTask(manifest, platform, opts.Environment, opts.Task) + if err != nil { + return err + } + + if !opts.Detach { + if toErr := spec.CheckTaskTimeout(opts.Task, rt.Task, sess.Timeout()); toErr != nil { + return toErr + } + } + + confirmed, confirmErr := c.Confirm( + fmt.Sprintf("Run task %q in %s?", opts.Task, opts.Environment), + nabat.WithAffirmative("Yes, run it"), + nabat.WithNegative("No, cancel"), + nabat.WithYes(opts.Yes), + nabat.WithBypassHint("--yes"), + ) + if confirmErr != nil { + return confirmErr + } + if !confirmed { + c.Info("Run cancelled") + return nil + } + + cluster, err := sess.Target(c, opts.Environment) + if err != nil { + return fmt.Errorf("target cluster: %w", err) + } + cmdopts.WarnContextFallback(c, cluster, opts.Environment) + + cs, err := cluster.Kubernetes() + if err != nil { + return fmt.Errorf("kubernetes client: %w", err) + } + + job, err := k8s.BuildTaskJob(k8s.TaskJobOptions{ + Project: manifest.Project, + Environment: opts.Environment, + Namespace: cluster.Namespace(), + TaskName: opts.Task, + Task: rt.Task, + Count: opts.Count, + Parallelism: opts.Parallelism, + Profile: rt.MergedProfile, + }) + if err != nil { + return fmt.Errorf("build job for %s: %w", opts.Task, err) + } + + return executeRun(c, cs, sess.Timeout(), job, opts.Detach) +} + +func executeRun(c *nabat.Context, cs kubernetes.Interface, timeout time.Duration, job *batchv1.Job, detach bool) error { + created, err := k8s.CreateTaskJob(c, cs, job) + if err != nil { + return err + } + c.Success("Created Job", "name", created.Name, "namespace", created.Namespace) + + if detach { + c.Info("Detached; the Job continues in the cluster") + return nil + } + + waitCtx, cancel := context.WithTimeout(c, timeout) + defer cancel() + if waitErr := k8s.WaitForJob(waitCtx, cs, created.Namespace, created.Name); waitErr != nil { + return waitErr + } + c.Success("Job completed", "name", created.Name) + return nil +} + +func resolveRunTask(manifest *spec.Spec, platform *spec.PlatformConfig, environment, name string) (spec.ResolvedTask, error) { + envIdentity := spec.NormalizeEnv(environment) + if platform != nil { + resolved, _, err := spec.Resolve(manifest, platform, envIdentity, spec.SubstitutionReport{}) + if err != nil { + return spec.ResolvedTask{}, fmt.Errorf("resolve spec: %w", err) + } + rt, ok := resolved.Tasks[name] + if !ok { + if _, exists := manifest.Tasks[name]; exists { + return spec.ResolvedTask{}, fmt.Errorf("task %s is skipped in environment %s", name, environment) + } + return spec.ResolvedTask{}, fmt.Errorf("unknown task %s", name) + } + return rt, nil + } + + merged, ok := manifest.MergedTask(name) + if !ok { + return spec.ResolvedTask{}, fmt.Errorf("unknown task %s", name) + } + if len(merged.Profiles) > 0 { + return spec.ResolvedTask{}, fmt.Errorf("task %s sets profiles but no platform file was found", name) + } + if len(merged.Environments) > 0 { + if _, match := spec.MatchEnvKey(environment, merged.Environments); !match { + return spec.ResolvedTask{}, fmt.Errorf("task %s is skipped in environment %s", name, environment) + } + } + return spec.ResolvedTask{Task: merged}, nil +} diff --git a/internal/cmd/run/run_test.go b/internal/cmd/run/run_test.go new file mode 100644 index 0000000..43e4766 --- /dev/null +++ b/internal/cmd/run/run_test.go @@ -0,0 +1,268 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package run + +import ( + "fmt" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/client-go/kubernetes/fake" + "nabat.dev/nabat" + "nabat.dev/nabat/nabattest" + + "deployah.dev/deployah/internal/k8s" + "deployah.dev/deployah/internal/spec" + + batchv1 "k8s.io/api/batch/v1" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + k8stesting "k8s.io/client-go/testing" +) + +func testManifest() *spec.Spec { + return &spec.Spec{ + APIVersion: spec.CurrentManifestVersion, + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "ghcr.io/acme/shop:1.2.3", Env: map[string]string{"DATABASE_URL": "postgres://db"}}, + }, + Tasks: map[string]spec.Task{ + "migrate": { + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"migrate", "up"}, + }, + "backfill": { + From: "api", + On: spec.TaskOnManual, + Command: []string{"backfill"}, + Environments: []string{"prod"}, + }, + }, + } +} + +func TestResolveRunTask(t *testing.T) { + t.Parallel() + + m := testManifest() + + t.Run("merges from parent", func(t *testing.T) { + t.Parallel() + rt, err := resolveRunTask(m, nil, "dev", "migrate") + require.NoError(t, err) + assert.Equal(t, "ghcr.io/acme/shop:1.2.3", rt.Task.Image) + assert.Equal(t, "postgres://db", rt.Task.Env["DATABASE_URL"]) + assert.Equal(t, []string{"migrate", "up"}, rt.Task.Command) + }) + + t.Run("unknown task", func(t *testing.T) { + t.Parallel() + _, err := resolveRunTask(m, nil, "dev", "missing") + require.Error(t, err) + assert.Contains(t, err.Error(), "unknown task") + }) + + t.Run("skipped in this environment", func(t *testing.T) { + t.Parallel() + _, err := resolveRunTask(m, nil, "dev", "backfill") + require.Error(t, err) + assert.Contains(t, err.Error(), "skipped") + }) + + t.Run("empty inherited profiles do not require a platform file", func(t *testing.T) { + t.Parallel() + local := testManifest() + api := local.Components["api"] + api.Profiles = []string{} + local.Components["api"] = api + rt, err := resolveRunTask(local, nil, "dev", "migrate") + require.NoError(t, err) + assert.Equal(t, "ghcr.io/acme/shop:1.2.3", rt.Task.Image) + }) + + t.Run("inherited profiles require a platform file", func(t *testing.T) { + t.Parallel() + local := testManifest() + api := local.Components["api"] + api.Profiles = []string{"batch"} + local.Components["api"] = api + _, err := resolveRunTask(local, nil, "dev", "migrate") + require.Error(t, err) + assert.Contains(t, err.Error(), "no platform file") + }) + + t.Run("platform resolve merges and skips by environment", func(t *testing.T) { + t.Parallel() + platform := &spec.PlatformConfig{ + APIVersion: "platform/v1-alpha.3", + Environments: map[string]spec.PlatformEnvironment{ + "dev": {Context: "kind"}, + "prod": {Context: "kind"}, + }, + } + rt, err := resolveRunTask(m, platform, "dev", "migrate") + require.NoError(t, err) + assert.Equal(t, "ghcr.io/acme/shop:1.2.3", rt.Task.Image) + + _, err = resolveRunTask(m, platform, "dev", "backfill") + require.Error(t, err) + assert.Contains(t, err.Error(), "skipped") + + _, err = resolveRunTask(m, platform, "dev", "missing") + require.Error(t, err) + assert.Contains(t, err.Error(), "unknown task") + }) +} + +func TestRunTask_FlagValidation(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + args []string + want string + }{ + { + name: "negative count", + args: []string{"run", "migrate", "dev", "--count", "-1"}, + want: "zero or positive", + }, + { + name: "parallelism above indexed job limit", + args: []string{"run", "migrate", "dev", "--parallelism", "100001"}, + want: "at most", + }, + { + name: "parallelism greater than count", + args: []string{"run", "migrate", "dev", "--count", "2", "--parallelism", "3"}, + want: "less than or equal to --count", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + io, _, _, _ := nabattest.NewIO() + app := nabat.MustNew("deployah", nabat.WithIO(io)) + Register(app) + err := nabattest.Run(t, app, tt.args) + require.Error(t, err) + assert.Contains(t, err.Error(), tt.want) + }) + } +} + +func TestExecuteRun_DetachAndWait(t *testing.T) { + t.Parallel() + + opts := k8s.TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "backfill", + Task: spec.Task{Image: "busybox:1.36", Command: []string{"true"}}, + } + + t.Run("detach returns after create", func(t *testing.T) { + t.Parallel() + cs := fake.NewSimpleClientset() + job := mustBuildJob(t, opts, "shop-dev-backfill-detach") + c := nabatContext(t) + require.NoError(t, executeRun(c, cs, time.Minute, job, true)) + got, err := cs.BatchV1().Jobs("default").Get(t.Context(), "shop-dev-backfill-detach", metav1.GetOptions{}) + require.NoError(t, err) + assert.Equal(t, "shop-dev-backfill-detach", got.Name) + }) + + t.Run("wait succeeds when the job completes", func(t *testing.T) { + t.Parallel() + cs := fake.NewSimpleClientset() + jobGetWithStatus(cs, func(job *batchv1.Job) { + job.Status.Succeeded = 1 + }) + job := mustBuildJob(t, opts, "shop-dev-backfill-wait") + c := nabatContext(t) + require.NoError(t, executeRun(c, cs, time.Minute, job, false)) + }) + + t.Run("wait fails when the job fails", func(t *testing.T) { + t.Parallel() + cs := fake.NewSimpleClientset() + jobGetWithStatus(cs, func(job *batchv1.Job) { + job.Status.Conditions = []batchv1.JobCondition{{ + Type: batchv1.JobFailed, + Status: corev1.ConditionTrue, + Message: "backoff limit exceeded", + }} + }) + job := mustBuildJob(t, opts, "shop-dev-backfill-fail") + c := nabatContext(t) + err := executeRun(c, cs, time.Minute, job, false) + require.Error(t, err) + assert.Contains(t, err.Error(), "backoff limit exceeded") + }) + + t.Run("create error", func(t *testing.T) { + t.Parallel() + cs := fake.NewSimpleClientset() + cs.PrependReactor("create", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + return true, nil, fmt.Errorf("quota exceeded") + }) + job := mustBuildJob(t, opts, "shop-dev-backfill-create") + c := nabatContext(t) + err := executeRun(c, cs, time.Minute, job, true) + require.Error(t, err) + assert.Contains(t, err.Error(), "quota exceeded") + }) +} + +func mustBuildJob(t *testing.T, opts k8s.TaskJobOptions, name string) *batchv1.Job { + t.Helper() + job, err := k8s.BuildTaskJob(opts) + require.NoError(t, err) + job.Name = name + job.GenerateName = "" + return job +} + +func jobGetWithStatus(cs *fake.Clientset, mutate func(*batchv1.Job)) { + cs.PrependReactor("get", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + get, ok := action.(k8stesting.GetAction) + if !ok { + return false, nil, nil + } + obj, err := cs.Tracker().Get(batchv1.SchemeGroupVersion.WithResource("jobs"), get.GetNamespace(), get.GetName()) + if err != nil { + return true, nil, err + } + job, ok := obj.(*batchv1.Job) + if !ok { + return true, nil, fmt.Errorf("unexpected job object %T", obj) + } + job = job.DeepCopy() + mutate(job) + return true, job, nil + }) +} + +func nabatContext(t *testing.T) *nabat.Context { + t.Helper() + io, _, _, _ := nabattest.NewIO() + app := nabat.MustNew("test", nabat.WithIO(io)) + return nabattest.Context(t, app) +} diff --git a/internal/cmd/validate/validate.go b/internal/cmd/validate/validate.go index abc4021..05d6b04 100644 --- a/internal/cmd/validate/validate.go +++ b/internal/cmd/validate/validate.go @@ -99,6 +99,9 @@ func runManifestOnly(c *nabat.Context, rt *session.Session) error { if compErr := spec.ValidateSpecComponents(rawSpec); compErr != nil { return compErr } + if taskErr := spec.ValidateSpecTasks(rawSpec); taskErr != nil { + return taskErr + } platform, platformErr := rt.Platform() if platformErr != nil { return fmt.Errorf("platform file error: %w", platformErr) diff --git a/internal/e2e/e2e_test.go b/internal/e2e/e2e_test.go index fe69a3c..9c3c409 100644 --- a/internal/e2e/e2e_test.go +++ b/internal/e2e/e2e_test.go @@ -46,6 +46,7 @@ import ( "deployah.dev/deployah/internal/localkube" appsv1 "k8s.io/api/apps/v1" + batchv1 "k8s.io/api/batch/v1" corev1 "k8s.io/api/core/v1" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apiextensionsclient "k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset" @@ -87,6 +88,7 @@ type expectations struct { Services []expectedService `yaml:"services"` PVCs []expectedPVC `yaml:"pvcs"` Pods expectedPods `yaml:"pods"` + Jobs []expectedJob `yaml:"jobs"` } type expectedDeployment struct { @@ -121,6 +123,11 @@ type expectedPVC struct { Storage string `yaml:"storage"` } +type expectedJob struct { + Name string `yaml:"name"` + Succeeded int32 `yaml:"succeeded"` +} + type expectedPods struct { LabelSelector string `yaml:"labelSelector"` MinCount int `yaml:"minCount"` @@ -191,6 +198,7 @@ func (s *E2ESuite) TestStatefulScale() { for _, name := range []string{"deployah.yaml", "deployah-replicas-2.yaml"} { data, readErr := os.ReadFile(filepath.Join(src, name)) // #nosec G304 -- fixture under testdata require.NoError(t, readErr) + // #nosec G703 -- dir is t.TempDir() and name comes from the literal above require.NoError(t, os.WriteFile(filepath.Join(dir, name), data, 0o600)) } @@ -211,7 +219,7 @@ func (s *E2ESuite) TestStatefulScale() { require.NoError(t, wait.For( conditions.New(res).ResourceMatch(&appsv1.StatefulSet{ - ObjectMeta: metav1.ObjectMeta{Name: stsName, Namespace: "default"}, + Name: stsName, Namespace: "default", }, func(obj k8s.Object) bool { live, ok := obj.(*appsv1.StatefulSet) return ok && live.Status.ReadyReplicas >= 1 @@ -222,12 +230,13 @@ func (s *E2ESuite) TestStatefulScale() { replicas2, readErr := os.ReadFile("deployah-replicas-2.yaml") // #nosec G304 -- temp fixture copy require.NoError(t, readErr) + // #nosec G703 -- constant name, written into the temp working dir require.NoError(t, os.WriteFile("deployah.yaml", replicas2, 0o600)) run(t, "deploy", "dev", "--context", "kind-deployah", "--yes") require.NoError(t, wait.For( conditions.New(res).ResourceMatch(&appsv1.StatefulSet{ - ObjectMeta: metav1.ObjectMeta{Name: stsName, Namespace: "default"}, + Name: stsName, Namespace: "default", }, func(obj k8s.Object) bool { live, ok := obj.(*appsv1.StatefulSet) return ok && live.Spec.Replicas != nil && @@ -353,14 +362,76 @@ func (s *E2ESuite) TestDeployScenarios() { } } -func (s *E2ESuite) assertExpectations(t testing.TB, exp expectations) { +func (s *E2ESuite) TestTaskRun() { + t := s.T() + s.prepareTaskdemo(t) + + run(t, "deploy", "dev", "--context", "kind-deployah", "--yes") + run(t, "run", "backfill", "dev", "--context", "kind-deployah", "--yes") + run(t, "run", "backfill", "dev", "--context", "kind-deployah", "--yes") + + res := s.client.Resources("default") + var jobs batchv1.JobList + require.NoError(t, res.List(t.Context(), &jobs, + resources.WithLabelSelector("deployah.dev/project=taskdemo,deployah.dev/component=backfill"))) + assert.GreaterOrEqual(t, len(jobs.Items), 2, "two runs get unique Job names") + for _, job := range jobs.Items { + assert.GreaterOrEqual(t, job.Status.Succeeded, int32(1), "job %s", job.Name) + } +} + +func (s *E2ESuite) TestTaskLogs() { + t := s.T() + s.prepareTaskdemo(t) + + run(t, "deploy", "dev", "--context", "kind-deployah", "--yes") + run(t, "run", "backfill", "dev", "--context", "kind-deployah", "--yes") + out := run(t, "logs", "taskdemo", "--component=backfill", "--environment=dev", + "--no-follow", "--context", "kind-deployah") + assert.Contains(t, out, "backfill-ok") +} + +func (s *E2ESuite) TestDeleteCleansCLIJobs() { + t := s.T() + s.prepareTaskdemo(t) + + run(t, "deploy", "dev", "--context", "kind-deployah", "--yes") + run(t, "run", "backfill", "dev", "--context", "kind-deployah", "--yes") + run(t, "delete", "taskdemo", "dev", "--yes", "--wait", "--allow-missing-platform", + "--context", "kind-deployah") + + res := s.client.Resources("default") + var jobs batchv1.JobList + require.NoError(t, res.List(t.Context(), &jobs, + resources.WithLabelSelector("deployah.dev/project=taskdemo,deployah.dev/environment=dev"))) + assert.Empty(t, jobs.Items) +} + +// prepareTaskdemo copies the task-migrate-smoke scenario into a temp dir, +// makes it the working directory, and registers a best-effort delete. +func (s *E2ESuite) prepareTaskdemo(t *testing.T) { t.Helper() + src := filepath.Join(s.testdataDir, "task-migrate-smoke") + dir := t.TempDir() + copyTree(t, src, dir) + t.Chdir(dir) + t.Cleanup(func() { + if err := runErr(t, "delete", "taskdemo", "dev", + "--yes", "--wait", "--allow-missing-platform", + "--context", "kind-deployah"); err != nil { + t.Logf("cleanup delete failed (non-fatal): %v", err) + } + }) +} + +func (s *E2ESuite) assertExpectations(tb testing.TB, exp expectations) { + tb.Helper() res := s.client.Resources(exp.Namespace) - ctx := t.Context() + ctx := tb.Context() for _, dep := range exp.Deployments { target := &appsv1.Deployment{ - ObjectMeta: metav1.ObjectMeta{Name: dep.Name, Namespace: exp.Namespace}, + Name: dep.Name, Namespace: exp.Namespace, } // Spelled out rather than using the DeploymentAvailable shorthand, so @@ -371,39 +442,39 @@ func (s *E2ESuite) assertExpectations(t testing.TB, exp expectations) { wait.WithTimeout(5*time.Minute), wait.WithInterval(2*time.Second), ) - require.NoErrorf(t, err, "deployment %s/%s never became Available", + require.NoErrorf(tb, err, "deployment %s/%s never became Available", exp.Namespace, dep.Name) var live appsv1.Deployment - require.NoError(t, res.Get(ctx, dep.Name, exp.Namespace, &live)) - dumpActual(t, &live) + require.NoError(tb, res.Get(ctx, dep.Name, exp.Namespace, &live)) + dumpActual(tb, &live) for key, val := range dep.Labels { // subset match - assert.Equalf(t, val, live.Labels[key], + assert.Equalf(tb, val, live.Labels[key], "deployment %s label %s", dep.Name, key) } containers := live.Spec.Template.Spec.Containers - require.NotEmptyf(t, containers, "deployment %s has no containers", dep.Name) - assert.Equalf(t, dep.Image, containers[0].Image, + require.NotEmptyf(tb, containers, "deployment %s has no containers", dep.Name) + assert.Equalf(tb, dep.Image, containers[0].Image, "deployment %s image", dep.Name) if dep.PortName != "" { - require.NotEmptyf(t, containers[0].Ports, + require.NotEmptyf(tb, containers[0].Ports, "deployment %s has no ports", dep.Name) - assert.Equalf(t, dep.PortName, containers[0].Ports[0].Name, + assert.Equalf(tb, dep.PortName, containers[0].Ports[0].Name, "deployment %s port name", dep.Name) } if dep.Replicas > 0 { - require.NotNil(t, live.Spec.Replicas) - assert.Equalf(t, dep.Replicas, *live.Spec.Replicas, + require.NotNil(tb, live.Spec.Replicas) + assert.Equalf(tb, dep.Replicas, *live.Spec.Replicas, "deployment %s replicas", dep.Name) } } for _, sts := range exp.StatefulSets { target := &appsv1.StatefulSet{ - ObjectMeta: metav1.ObjectMeta{Name: sts.Name, Namespace: exp.Namespace}, + Name: sts.Name, Namespace: exp.Namespace, } err := wait.For( conditions.New(res).ResourceMatch(target, func(obj k8s.Object) bool { @@ -417,60 +488,60 @@ func (s *E2ESuite) assertExpectations(t testing.TB, exp expectations) { wait.WithTimeout(5*time.Minute), wait.WithInterval(2*time.Second), ) - require.NoErrorf(t, err, "statefulset %s/%s never became ready", + require.NoErrorf(tb, err, "statefulset %s/%s never became ready", exp.Namespace, sts.Name) var live appsv1.StatefulSet - require.NoError(t, res.Get(ctx, sts.Name, exp.Namespace, &live)) - dumpActual(t, &live) + require.NoError(tb, res.Get(ctx, sts.Name, exp.Namespace, &live)) + dumpActual(tb, &live) for key, val := range sts.Labels { - assert.Equalf(t, val, live.Labels[key], + assert.Equalf(tb, val, live.Labels[key], "statefulset %s label %s", sts.Name, key) } containers := live.Spec.Template.Spec.Containers - require.NotEmptyf(t, containers, "statefulset %s has no containers", sts.Name) - assert.Equalf(t, sts.Image, containers[0].Image, + require.NotEmptyf(tb, containers, "statefulset %s has no containers", sts.Name) + assert.Equalf(tb, sts.Image, containers[0].Image, "statefulset %s image", sts.Name) if sts.PortName != "" { - require.NotEmptyf(t, containers[0].Ports, + require.NotEmptyf(tb, containers[0].Ports, "statefulset %s has no ports", sts.Name) - assert.Equalf(t, sts.PortName, containers[0].Ports[0].Name, + assert.Equalf(tb, sts.PortName, containers[0].Ports[0].Name, "statefulset %s port name", sts.Name) } if sts.Replicas > 0 { - require.NotNil(t, live.Spec.Replicas) - assert.Equalf(t, sts.Replicas, *live.Spec.Replicas, + require.NotNil(tb, live.Spec.Replicas) + assert.Equalf(tb, sts.Replicas, *live.Spec.Replicas, "statefulset %s replicas", sts.Name) } } for _, svc := range exp.Services { var live corev1.Service - require.NoError(t, res.Get(ctx, svc.Name, exp.Namespace, &live)) - dumpActual(t, &live) + require.NoError(tb, res.Get(ctx, svc.Name, exp.Namespace, &live)) + dumpActual(tb, &live) - require.NotEmptyf(t, live.Spec.Ports, "service %s has no ports", svc.Name) - assert.Equalf(t, svc.Port, live.Spec.Ports[0].Port, "service %s port", svc.Name) + require.NotEmptyf(tb, live.Spec.Ports, "service %s has no ports", svc.Name) + assert.Equalf(tb, svc.Port, live.Spec.Ports[0].Port, "service %s port", svc.Name) // TargetPort is an intstr; a named port lives in StrVal, not IntVal. if svc.TargetPortName != "" { - assert.Equalf(t, svc.TargetPortName, live.Spec.Ports[0].TargetPort.StrVal, + assert.Equalf(tb, svc.TargetPortName, live.Spec.Ports[0].TargetPort.StrVal, "service %s targetPort name", svc.Name) } if svc.ClusterIP == "None" { - assert.Equalf(t, corev1.ClusterIPNone, live.Spec.ClusterIP, + assert.Equalf(tb, corev1.ClusterIPNone, live.Spec.ClusterIP, "service %s should be headless", svc.Name) } for key, val := range svc.Selector { - assert.Equalf(t, val, live.Spec.Selector[key], + assert.Equalf(tb, val, live.Spec.Selector[key], "service %s selector %s", svc.Name, key) } } for _, wantPVC := range exp.PVCs { var pvcs corev1.PersistentVolumeClaimList - require.NoError(t, res.List(ctx, &pvcs)) + require.NoError(tb, res.List(ctx, &pvcs)) matched := 0 for _, pvc := range pvcs.Items { if !strings.HasPrefix(pvc.Name, wantPVC.NamePrefix) { @@ -478,52 +549,68 @@ func (s *E2ESuite) assertExpectations(t testing.TB, exp expectations) { } matched++ if wantPVC.Phase != "" { - assert.Equalf(t, wantPVC.Phase, string(pvc.Status.Phase), + assert.Equalf(tb, wantPVC.Phase, string(pvc.Status.Phase), "pvc %s phase", pvc.Name) } if wantPVC.Storage != "" { req := pvc.Spec.Resources.Requests[corev1.ResourceStorage] - assert.Equalf(t, wantPVC.Storage, req.String(), + assert.Equalf(tb, wantPVC.Storage, req.String(), "pvc %s storage", pvc.Name) } } - assert.GreaterOrEqualf(t, matched, wantPVC.MinCount, + assert.GreaterOrEqualf(tb, matched, wantPVC.MinCount, "pvcs with prefix %s", wantPVC.NamePrefix) } if exp.Pods.LabelSelector != "" { var pods corev1.PodList - require.NoError(t, res.List(ctx, &pods, + require.NoError(tb, res.List(ctx, &pods, resources.WithLabelSelector(exp.Pods.LabelSelector))) - assert.GreaterOrEqualf(t, len(pods.Items), exp.Pods.MinCount, + assert.GreaterOrEqualf(tb, len(pods.Items), exp.Pods.MinCount, "pods matching %s", exp.Pods.LabelSelector) for _, pod := range pods.Items { - assert.Equalf(t, exp.Pods.Phase, string(pod.Status.Phase), + assert.Equalf(tb, exp.Pods.Phase, string(pod.Status.Phase), "pod %s phase", pod.Name) } } + + for _, wantJob := range exp.Jobs { + target := &batchv1.Job{ + Name: wantJob.Name, Namespace: exp.Namespace, + } + err := wait.For( + conditions.New(res).ResourceMatch(target, func(obj k8s.Object) bool { + live, ok := obj.(*batchv1.Job) + return ok && live.Status.Succeeded >= wantJob.Succeeded + }), + wait.WithTimeout(5*time.Minute), + wait.WithInterval(2*time.Second), + ) + require.NoErrorf(tb, err, "job %s/%s never reached succeeded>=%d", + exp.Namespace, wantJob.Name, wantJob.Succeeded) + } } -func newKlient(t testing.TB, kubeconfigPath, contextName string) klient.Client { - t.Helper() +func newKlient(tb testing.TB, kubeconfigPath, contextName string) klient.Client { + tb.Helper() rules := clientcmd.NewDefaultClientConfigLoadingRules() rules.ExplicitPath = kubeconfigPath overrides := &clientcmd.ConfigOverrides{CurrentContext: contextName} restCfg, err := clientcmd.NewNonInteractiveDeferredLoadingClientConfig( rules, overrides).ClientConfig() - require.NoErrorf(t, err, "rest config from %s (context %s)", + require.NoErrorf(tb, err, "rest config from %s (context %s)", kubeconfigPath, contextName) c, err := klient.New(restCfg) - require.NoError(t, err, "build klient") + require.NoError(tb, err, "build klient") return c } -func discoverScenarios(t testing.TB, testdataDir string) []scenario { - t.Helper() +func discoverScenarios(tb testing.TB, testdataDir string) []scenario { + tb.Helper() entries, err := os.ReadDir(testdataDir) - require.NoErrorf(t, err, "read %s", testdataDir) + require.NoErrorf(tb, err, "read %s", testdataDir) var found []scenario for _, entry := range entries { @@ -538,18 +625,18 @@ func discoverScenarios(t testing.TB, testdataDir string) []scenario { } raw, readErr := os.ReadFile(specPath) // #nosec G304 -- path under testdata/ - require.NoError(t, readErr) + require.NoError(tb, readErr) var spec struct { Project string `yaml:"project"` } - require.NoError(t, yaml.Unmarshal(raw, &spec)) - require.NotEmptyf(t, spec.Project, "%s has no project field", specPath) + require.NoError(tb, yaml.Unmarshal(raw, &spec)) + require.NotEmptyf(tb, spec.Project, "%s has no project field", specPath) found = append(found, scenario{ Name: entry.Name(), Dir: dir, Project: spec.Project, }) } - require.NotEmptyf(t, found, "no scenarios found in %s", testdataDir) + require.NotEmptyf(tb, found, "no scenarios found in %s", testdataDir) return found } @@ -558,39 +645,39 @@ func regularFileExists(path string) bool { return err == nil && info.Mode().IsRegular() } -func loadExpectations(t testing.TB, dir string) expectations { - t.Helper() +func loadExpectations(tb testing.TB, dir string) expectations { + tb.Helper() raw, err := os.ReadFile(filepath.Join(dir, "expect.yaml")) // #nosec G304 -- path under testdata/ - require.NoError(t, err) + require.NoError(tb, err) var exp expectations - require.NoError(t, yaml.Unmarshal(raw, &exp)) - require.NotEmptyf(t, exp.Env, "%s/expect.yaml has no env field", dir) + require.NoError(tb, yaml.Unmarshal(raw, &exp)) + require.NotEmptyf(tb, exp.Env, "%s/expect.yaml has no env field", dir) if exp.Namespace == "" { exp.Namespace = "default" } return exp } -func run(t testing.TB, args ...string) string { - t.Helper() - stdout, _ := runCapture(t, args...) +func run(tb testing.TB, args ...string) string { + tb.Helper() + stdout, _ := runCapture(tb, args...) return stdout } // runCapture runs deployah and returns stdout and stderr on success. -func runCapture(t testing.TB, args ...string) (stdout, stderr string) { - t.Helper() +func runCapture(tb testing.TB, args ...string) (stdout, stderr string) { + tb.Helper() appIO, _, out, errOut := nabattest.NewIO() app := cmd.NewApp(nabat.WithIO(appIO)) - err := nabattest.Run(t, app, args) - require.NoErrorf(t, err, "deployah %s\nstderr:\n%s", + err := nabattest.Run(tb, app, args) + require.NoErrorf(tb, err, "deployah %s\nstderr:\n%s", strings.Join(args, " "), errOut.String()) return out.String(), errOut.String() } -func copyTree(t testing.TB, src, dst string) { - t.Helper() +func copyTree(tb testing.TB, src, dst string) { + tb.Helper() err := filepath.WalkDir(src, func(path string, d os.DirEntry, walkErr error) error { if walkErr != nil { return walkErr @@ -603,7 +690,9 @@ func copyTree(t testing.TB, src, dst string) { if d.IsDir() { return os.MkdirAll(target, 0o750) } - in, openErr := os.Open(path) // #nosec G304 -- path under testdata/ + // G122 flags the Walk-callback path as symlink-TOCTOU prone; src is + // testdata/ and dst is a t.TempDir(), both test-controlled. + in, openErr := os.Open(path) // #nosec G304 G122 -- path under testdata/ if openErr != nil { return openErr } @@ -619,41 +708,41 @@ func copyTree(t testing.TB, src, dst string) { } return closeErr }) - require.NoError(t, err) + require.NoError(tb, err) } -func readFixtureFile(t testing.TB, path string) string { - t.Helper() +func readFixtureFile(tb testing.TB, path string) string { + tb.Helper() raw, err := os.ReadFile(path) // #nosec G304 -- path under test-controlled temp dir - require.NoError(t, err) + require.NoError(tb, err) return string(raw) } -func newApiextensionsClient(t testing.TB, kubeconfigPath, contextName string) apiextensionsclient.Interface { - t.Helper() +func newApiextensionsClient(tb testing.TB, kubeconfigPath, contextName string) apiextensionsclient.Interface { + tb.Helper() rules := clientcmd.NewDefaultClientConfigLoadingRules() rules.ExplicitPath = kubeconfigPath overrides := &clientcmd.ConfigOverrides{CurrentContext: contextName} restCfg, err := clientcmd.NewNonInteractiveDeferredLoadingClientConfig( rules, overrides).ClientConfig() - require.NoError(t, err) + require.NoError(tb, err) cs, err := apiextensionsclient.NewForConfig(restCfg) - require.NoError(t, err) + require.NoError(tb, err) return cs } -func getCRD(t testing.TB, ext apiextensionsclient.Interface, name string) *apiextensionsv1.CustomResourceDefinition { - t.Helper() +func getCRD(tb testing.TB, ext apiextensionsclient.Interface, name string) *apiextensionsv1.CustomResourceDefinition { + tb.Helper() crd, err := ext.ApiextensionsV1().CustomResourceDefinitions().Get( - t.Context(), name, metav1.GetOptions{}) - require.NoError(t, err) + tb.Context(), name, metav1.GetOptions{}) + require.NoError(tb, err) return crd } -func waitCRDEstablished(t testing.TB, ext apiextensionsclient.Interface, name string) *apiextensionsv1.CustomResourceDefinition { - t.Helper() +func waitCRDEstablished(tb testing.TB, ext apiextensionsclient.Interface, name string) *apiextensionsv1.CustomResourceDefinition { + tb.Helper() var latest *apiextensionsv1.CustomResourceDefinition - require.NoError(t, wait.For(func(ctx context.Context) (bool, error) { + require.NoError(tb, wait.For(func(ctx context.Context) (bool, error) { crd, err := ext.ApiextensionsV1().CustomResourceDefinitions().Get( ctx, name, metav1.GetOptions{}) if err != nil { @@ -668,15 +757,15 @@ func waitCRDEstablished(t testing.TB, ext apiextensionsclient.Interface, name st } return false, nil }, wait.WithTimeout(2*time.Minute), wait.WithInterval(time.Second))) - require.NotNil(t, latest) + require.NotNil(tb, latest) return latest } -func runErr(t testing.TB, args ...string) error { - t.Helper() +func runErr(tb testing.TB, args ...string) error { + tb.Helper() appIO, _, _, errOut := nabattest.NewIO() app := cmd.NewApp(nabat.WithIO(appIO)) - err := nabattest.Run(t, app, args) + err := nabattest.Run(tb, app, args) if err == nil { return nil } @@ -686,47 +775,47 @@ func runErr(t testing.TB, args ...string) error { return err } -func requireEngine(t testing.TB) { - t.Helper() +func requireEngine(tb testing.TB) { + tb.Helper() if err := exec.Command("docker", "info").Run(); err != nil { if os.Getenv("CI") == "true" { - t.Fatalf("container engine required in CI: %v", err) + tb.Fatalf("container engine required in CI: %v", err) } - t.Skipf("no container engine: %v", err) + tb.Skipf("no container engine: %v", err) } } -func requireNoCollision(t testing.TB) { - t.Helper() +func requireNoCollision(tb testing.TB) { + tb.Helper() m, err := localkube.New() - require.NoError(t, err) + require.NoError(tb, err) defer m.Close() //nolint:errcheck // best-effort cleanup of provider resources - _, getErr := m.Get(t.Context(), "deployah") + _, getErr := m.Get(tb.Context(), "deployah") if errors.Is(getErr, localkube.ErrNotFound) { return // no existing cluster, nothing to do } - require.NoError(t, getErr) + require.NoError(tb, getErr) if os.Getenv("DEPLOYAH_E2E_FORCE") != "1" { - t.Fatal("cluster 'deployah' already exists; " + + tb.Fatal("cluster 'deployah' already exists; " + "set DEPLOYAH_E2E_FORCE=1 to destroy and recreate it") } - t.Log("DEPLOYAH_E2E_FORCE=1: destroying the existing cluster") - require.NoError(t, runErr(t, "cluster", "down", "--force")) + tb.Log("DEPLOYAH_E2E_FORCE=1: destroying the existing cluster") + require.NoError(tb, runErr(tb, "cluster", "down", "--force")) } // dumpActual logs a live object as YAML when DEPLOYAH_E2E_DUMP=1, so a new // scenario's expect.yaml can be curated from what deployah actually renders. -func dumpActual(t testing.TB, obj any) { - t.Helper() +func dumpActual(tb testing.TB, obj any) { + tb.Helper() if os.Getenv("DEPLOYAH_E2E_DUMP") != "1" { return } out, err := yaml.Marshal(obj) if err != nil { - t.Logf("dump failed: %v", err) + tb.Logf("dump failed: %v", err) return } - t.Logf("ACTUAL:\n%s", out) + tb.Logf("ACTUAL:\n%s", out) } diff --git a/internal/e2e/testdata/basic-web-service/deployah.yaml b/internal/e2e/testdata/basic-web-service/deployah.yaml index 9186089..1dec9dd 100644 --- a/internal/e2e/testdata/basic-web-service/deployah.yaml +++ b/internal/e2e/testdata/basic-web-service/deployah.yaml @@ -1,4 +1,4 @@ -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: basic-web-service components: web: diff --git a/internal/e2e/testdata/crd-lifecycle/deployah.yaml b/internal/e2e/testdata/crd-lifecycle/deployah.yaml index 9609419..ea48346 100644 --- a/internal/e2e/testdata/crd-lifecycle/deployah.yaml +++ b/internal/e2e/testdata/crd-lifecycle/deployah.yaml @@ -1,4 +1,4 @@ -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: crd-lifecycle components: web: diff --git a/internal/e2e/testdata/stateful-basic/deployah.yaml b/internal/e2e/testdata/stateful-basic/deployah.yaml index fe8caaf..989ad7e 100644 --- a/internal/e2e/testdata/stateful-basic/deployah.yaml +++ b/internal/e2e/testdata/stateful-basic/deployah.yaml @@ -1,4 +1,4 @@ -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: stateful-basic components: cache: diff --git a/internal/e2e/testdata/stateful-scale/deployah-replicas-2.yaml b/internal/e2e/testdata/stateful-scale/deployah-replicas-2.yaml index 74c658f..a006825 100644 --- a/internal/e2e/testdata/stateful-scale/deployah-replicas-2.yaml +++ b/internal/e2e/testdata/stateful-scale/deployah-replicas-2.yaml @@ -1,4 +1,4 @@ -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: stateful-scale components: cache: diff --git a/internal/e2e/testdata/stateful-scale/deployah.yaml b/internal/e2e/testdata/stateful-scale/deployah.yaml index c3dfa28..c9f0970 100644 --- a/internal/e2e/testdata/stateful-scale/deployah.yaml +++ b/internal/e2e/testdata/stateful-scale/deployah.yaml @@ -1,4 +1,4 @@ -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: stateful-scale components: cache: diff --git a/internal/e2e/testdata/task-migrate-smoke/deployah.yaml b/internal/e2e/testdata/task-migrate-smoke/deployah.yaml new file mode 100644 index 0000000..ad69493 --- /dev/null +++ b/internal/e2e/testdata/task-migrate-smoke/deployah.yaml @@ -0,0 +1,29 @@ +apiVersion: v1-alpha.5 +project: taskdemo +components: + api: + image: nginx:latest + port: 80 + environments: [dev] + resourcePreset: nano + env: + DATABASE_URL: postgres://example +tasks: + migrate: + from: api + image: busybox:1.36 + "on": preDeploy + command: ["true"] + fanout: 2 + smoke: + from: api + image: busybox:1.36 + "on": postDeploy + command: ["true"] + backfill: + from: api + image: busybox:1.36 + "on": manual + command: ["echo", "backfill-ok"] +environments: + dev: {} diff --git a/internal/e2e/testdata/task-migrate-smoke/expect.yaml b/internal/e2e/testdata/task-migrate-smoke/expect.yaml new file mode 100644 index 0000000..1d6b862 --- /dev/null +++ b/internal/e2e/testdata/task-migrate-smoke/expect.yaml @@ -0,0 +1,22 @@ +env: dev +namespace: default +deployments: + - name: taskdemo-dev-api + replicas: 1 + image: docker.io/library/nginx:latest + portName: http + labels: + deployah.dev/project: taskdemo + deployah.dev/environment: dev + deployah.dev/component: api +services: + - name: taskdemo-dev-api + port: 80 + targetPortName: http + selector: + app.kubernetes.io/instance: taskdemo-dev + app.kubernetes.io/name: api +pods: + labelSelector: "deployah.dev/project=taskdemo,deployah.dev/environment=dev,deployah.dev/component=api" + minCount: 1 + phase: Running diff --git a/internal/e2e/testdata/worker-basic/deployah.yaml b/internal/e2e/testdata/worker-basic/deployah.yaml index 7b4f080..0be213a 100644 --- a/internal/e2e/testdata/worker-basic/deployah.yaml +++ b/internal/e2e/testdata/worker-basic/deployah.yaml @@ -1,4 +1,4 @@ -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: worker-basic components: worker: diff --git a/internal/helm/cache_test.go b/internal/helm/cache_test.go index 122b853..2eaea8d 100644 --- a/internal/helm/cache_test.go +++ b/internal/helm/cache_test.go @@ -38,11 +38,11 @@ func TestPrepareChart_CacheSurvivesCallerCleanup(t *testing.T) { t.Parallel() cache := NewChartCache(time.Hour) manifest := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "cache-test", Components: map[string]spec.Component{"web": serviceComponent()}, } - require.NoError(t, spec.FillSpecWithDefaults(manifest, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(manifest, spec.CurrentManifestVersion)) returnedPath, err := PrepareChart(t.Context(), manifest, "production", nil, cache) require.NoError(t, err) @@ -52,11 +52,7 @@ func TestPrepareChart_CacheSurvivesCallerCleanup(t *testing.T) { cachedPath, found := cache.get(key) require.True(t, found, "PrepareChart must register a cache entry on a miss") - t.Cleanup(func() { - if removeErr := os.RemoveAll(cachedPath); removeErr != nil { - t.Logf("cleanup: remove cached chart dir: %v", removeErr) - } - }) + t.Cleanup(func() { removeChartDir(t, cachedPath) }) assert.NotEqual(t, returnedPath, cachedPath, "PrepareChart must return a copy on a cache miss, not the cache's own backing directory") @@ -69,6 +65,26 @@ func TestPrepareChart_CacheSurvivesCallerCleanup(t *testing.T) { assert.True(t, stillFound, "the cache entry must survive the caller cleaning up its own returned copy") } +// backdate rewinds an entry's creation time so TTL behavior can be tested +// without sleeping. It takes the same lock as the production accessors. +func (c *ChartCache) backdate(tb testing.TB, cacheKey string, d time.Duration) { + tb.Helper() + c.mu.Lock() + defer c.mu.Unlock() + entry, exists := c.entries[cacheKey] + require.True(tb, exists, "no cache entry %q to backdate", cacheKey) + entry.createdAt = entry.createdAt.Add(-d) +} + +// removeChartDir deletes a prepared chart directory. Cleanup failures are +// logged rather than failed, so they cannot mask the test result. +func removeChartDir(tb testing.TB, path string) { + tb.Helper() + if err := os.RemoveAll(path); err != nil { + tb.Logf("cleanup: remove chart dir %s: %v", path, err) + } +} + // TestPrepareChart_RequiresCache verifies PrepareChart rejects a nil cache. func TestPrepareChart_RequiresCache(t *testing.T) { t.Parallel() @@ -137,23 +153,23 @@ func TestChartCache_GetMissExpiredAndMissingDir(t *testing.T) { // TestChartCache_CleanupExpired removes expired entries and their directories. func TestChartCache_CleanupExpired(t *testing.T) { t.Parallel() - cache := NewChartCache(time.Millisecond) + cache := NewChartCache(time.Hour) dir := t.TempDir() keep := t.TempDir() cache.set("old", dir) cache.set("fresh", keep) - require.Eventually(t, func() bool { - _, ok := cache.get("old") - return !ok - }, 50*time.Millisecond, time.Millisecond) + // Backdate "old" rather than waiting out a short TTL: with a TTL small + // enough to sleep through, any scheduling delay before cleanupExpired + // runs expires "fresh" too and the test fails on a loaded machine. + cache.backdate(t, "old", 2*time.Hour) + _, found := cache.get("old") + require.False(t, found, "backdated entry must read as expired") - // Refresh the fresh entry so only "old" is past TTL when cleanup runs. - cache.set("fresh", keep) cache.cleanupExpired() assert.Equal(t, 1, cache.entryCount()) - _, found := cache.get("fresh") + _, found = cache.get("fresh") assert.True(t, found) _, err := os.Stat(dir) assert.ErrorIs(t, err, os.ErrNotExist) @@ -205,14 +221,14 @@ func TestGenerateKey_ResolvedSpecContentInvalidates(t *testing.T) { cache := NewChartCache(time.Hour) base := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "cache-key", Components: map[string]spec.Component{"web": serviceComponent()}, } - require.NoError(t, spec.FillSpecWithDefaults(base, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(base, spec.CurrentManifestVersion)) changed := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "cache-key", Components: map[string]spec.Component{ "web": { @@ -222,7 +238,7 @@ func TestGenerateKey_ResolvedSpecContentInvalidates(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(changed, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(changed, spec.CurrentManifestVersion)) resolvedA := &spec.ResolvedSpec{Spec: base, Env: spec.NormalizeEnv("production")} resolvedB := &spec.ResolvedSpec{Spec: changed, Env: spec.NormalizeEnv("production")} diff --git a/internal/helm/chart/Chart.yaml.gotmpl b/internal/helm/chart/Chart.yaml.gotmpl index 2c30cb6..0274db3 100644 --- a/internal/helm/chart/Chart.yaml.gotmpl +++ b/internal/helm/chart/Chart.yaml.gotmpl @@ -13,7 +13,11 @@ dependencies: version: "~0.1.0-alpha.1" repository: "file://charts/deployah" import-values: -{{- range $name, $component := .Spec.Components }} +{{- range $name := .ComponentNames }} + - child: exports.defaults + parent: {{ $name }} +{{- end }} +{{- range $name := .TaskNames }} - child: exports.defaults parent: {{ $name }} {{- end }} diff --git a/internal/helm/chart/charts/deployah/templates/job.yaml b/internal/helm/chart/charts/deployah/templates/job.yaml new file mode 100644 index 0000000..b6acfa4 --- /dev/null +++ b/internal/helm/chart/charts/deployah/templates/job.yaml @@ -0,0 +1,78 @@ +{{- define "deployah.job" -}} +{{- if .Values.job.enabled -}} +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ include "common.names.fullname" . }} + namespace: {{ include "common.names.namespace" . | quote }} + {{- $labels := include "common.tplvalues.merge" (dict "values" (list .Values.labels .Values.commonLabels) "context" .) | fromYaml }} + labels: {{- include "common.labels.standard" ( dict "customLabels" $labels "context" $ ) | nindent 4 }} + annotations: + {{- if .Values.job.hook }} + helm.sh/hook: {{ .Values.job.hook | quote }} + helm.sh/hook-weight: {{ .Values.job.hookWeight | quote }} + helm.sh/hook-delete-policy: {{ .Values.job.hookDeletePolicy | quote }} + {{- end }} + {{- $annotations := include "common.tplvalues.merge" (dict "values" (list .Values.annotations .Values.commonAnnotations) "context" .) | fromYaml }} + {{- if $annotations }} + {{- include "common.tplvalues.render" ( dict "value" $annotations "context" $ ) | nindent 4 }} + {{- end }} +spec: + completionMode: Indexed + completions: {{ .Values.job.completions }} + parallelism: {{ .Values.job.parallelism }} + backoffLimit: {{ .Values.job.backoffLimit }} + {{- if .Values.job.activeDeadlineSeconds }} + activeDeadlineSeconds: {{ .Values.job.activeDeadlineSeconds }} + {{- end }} + {{- if hasKey .Values.job "ttlSecondsAfterFinished" }} + ttlSecondsAfterFinished: {{ .Values.job.ttlSecondsAfterFinished }} + {{- end }} + template: + metadata: + {{- $podLabels := include "common.tplvalues.merge" (dict "values" (list .Values.podLabels .Values.commonLabels) "context" .) | fromYaml }} + labels: {{- include "common.labels.standard" ( dict "customLabels" $podLabels "context" $ ) | nindent 8 }} + {{- if .Values.podAnnotations }} + annotations: {{- toYaml .Values.podAnnotations | nindent 8 }} + {{- end }} + spec: + restartPolicy: OnFailure + automountServiceAccountToken: false + {{- include "common.images.renderPullSecrets" (dict "images" (list .Values.image) "context" $) | nindent 6 }} + {{- if .Values.nodeSelector }} + nodeSelector: {{- toYaml .Values.nodeSelector | nindent 8 }} + {{- end }} + {{- if .Values.tolerations }} + tolerations: {{- toYaml .Values.tolerations | nindent 8 }} + {{- end }} + {{- if .Values.podSecurityContext.enabled }} + securityContext: {{- omit .Values.podSecurityContext "enabled" | toYaml | nindent 8 }} + {{- end }} + containers: + - name: {{ .Chart.Name }} + {{- if .Values.containerSecurityContext.enabled }} + securityContext: {{- omit .Values.containerSecurityContext "enabled" | toYaml | nindent 12 }} + {{- end }} + image: {{ include "common.images.image" (dict "imageRoot" .Values.image "global" .Values.global) }} + imagePullPolicy: {{ default (eq .Values.image.tag "latest" | ternary "Always" "IfNotPresent") .Values.image.pullPolicy }} + {{- if .Values.command }} + command: {{- toYaml .Values.command | nindent 12 }} + {{- end }} + {{- if .Values.args }} + args: {{- toYaml .Values.args | nindent 12 }} + {{- end }} + {{- if .Values.envVars }} + env: + {{- $env := .Values.envVars }} + {{- range $key := keys $env | sortAlpha }} + {{- $val := index $env $key }} + - name: {{ $key | quote }} + value: {{ $val | quote }} + {{- end }} + {{- end }} + {{- if .Values.resources }} + resources: {{- toYaml .Values.resources | nindent 12 }} + {{- end }} +{{- end }} +{{- end }} diff --git a/internal/helm/chart/charts/deployah/values.yaml b/internal/helm/chart/charts/deployah/values.yaml index 4f648a3..b565e42 100644 --- a/internal/helm/chart/charts/deployah/values.yaml +++ b/internal/helm/chart/charts/deployah/values.yaml @@ -875,6 +875,40 @@ exports: ## labels: {} + ## Job: run-to-completion work (Deployah tasks) + ## + job: + ## @param job.enabled Create a Job for this subchart + ## + enabled: false + + ## @param job.completions Indexed Job completions (fanout count) + ## + completions: 1 + + ## @param job.parallelism How many indexed copies may run at once + ## + parallelism: 1 + + ## @param job.backoffLimit Retries before the Job is marked failed + ## + backoffLimit: 3 + + ## @param job.ttlSecondsAfterFinished Seconds to keep a finished Job (0 deletes immediately). Omitted when unset. + ## + + ## @param job.hook Helm hook events (empty for non-hook Jobs) + ## + hook: "" + + ## @param job.hookWeight Helm hook-weight + ## + hookWeight: 0 + + ## @param job.hookDeletePolicy Helm hook-delete-policy + ## + hookDeletePolicy: before-hook-creation,hook-succeeded + ## Cronjob: create jobs on a repeated schedule ## Ref: https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/ ## diff --git a/internal/helm/generate.go b/internal/helm/generate.go index c94e9e0..ac2a00d 100644 --- a/internal/helm/generate.go +++ b/internal/helm/generate.go @@ -68,8 +68,12 @@ type ChartData struct { } // Values is the data map for values.yaml templating. Values map[string]any - // Spec is the source Deployah spec for dynamic sub-charts. - Spec *spec.Spec + // ComponentNames are the sorted names of the component sub-charts + // created for this environment. + ComponentNames []string + // TaskNames are the sorted names of the task sub-charts created for this + // environment. Only hook tasks get one. + TaskNames []string } // GenerateReleaseName returns the Helm release name for project and @@ -128,6 +132,14 @@ func PrepareChart(ctx context.Context, manifest *spec.Spec, desiredEnvironment s const root = "chart" + // Resolve the sub-chart names once, before creating anything on disk, so + // Chart.yaml and the sub-chart directories below cannot disagree. + componentNames := activeComponentNames(manifest, desiredEnvironment) + taskNames, err := hookTaskNames(manifest, desiredEnvironment, resolved) + if err != nil { + return "", fmt.Errorf("failed to resolve task sub-chart names: %w", err) + } + tmpDir, err := os.MkdirTemp("", "deployah-chart-*") if err != nil { return "", fmt.Errorf("failed to create temp dir: %w", err) @@ -137,7 +149,8 @@ func PrepareChart(ctx context.Context, manifest *spec.Spec, desiredEnvironment s chartData.Chart.Name = manifest.Project chartData.Chart.Version = "0.1.0" chartData.Values = map[string]any{} - chartData.Spec = manifest + chartData.ComponentNames = componentNames + chartData.TaskNames = taskNames err = fs.WalkDir(ChartTemplateFS, root, func(path string, d fs.DirEntry, walkErr error) error { if walkErr != nil { @@ -181,8 +194,11 @@ func PrepareChart(ctx context.Context, manifest *spec.Spec, desiredEnvironment s return nil } - // Prepend "_" so Helm sees files go:embed excludes (those starting with "_"). - if strings.Contains(path, "templates/") { + // Prepend "_" so Helm sees files go:embed excludes (those starting + // with "_"). Only library templates under charts/ are partials + // (define helpers). Parent chart/templates/ files are real + // resources and must keep their names so Helm renders them. + if strings.Contains(path, "charts/") && strings.Contains(path, "templates/") { ext := filepath.Ext(d.Name()) base := strings.TrimSuffix(d.Name(), ext) if slices.Contains([]string{".yaml", ".tpl", ".txt"}, ext) { @@ -200,9 +216,12 @@ func PrepareChart(ctx context.Context, manifest *spec.Spec, desiredEnvironment s return "", fmt.Errorf("failed to expand embedded chart: %w", err) } - if err = createComponentSubCharts(tmpDir, manifest, desiredEnvironment); err != nil { + if err = createComponentSubCharts(tmpDir, componentNames); err != nil { return "", fmt.Errorf("failed to create component sub-charts: %w", err) } + if err = createTaskSubCharts(tmpDir, taskNames); err != nil { + return "", fmt.Errorf("failed to create task sub-charts: %w", err) + } values, err := MapSpecToChartValues(manifest, desiredEnvironment, resolved) if err != nil { @@ -224,23 +243,19 @@ func PrepareChart(ctx context.Context, manifest *spec.Spec, desiredEnvironment s return createChartCopy(tmpDir) } -// createComponentSubCharts creates sub-chart directories for each -// component active in desiredEnvironment. A component excluded from this -// environment gets no subchart at all: an empty subchart would still -// render default-valued resources (e.g. a Service from app.yaml's base -// values.yaml), leaking them into an environment the component was never -// meant to reach. -func createComponentSubCharts(chartDir string, manifest *spec.Spec, desiredEnvironment string) error { +// createComponentSubCharts creates a sub-chart directory for each name in +// componentNames, as returned by [activeComponentNames]. A component +// excluded from the target environment is absent from that list and gets no +// subchart at all: an empty subchart would still render default-valued +// resources (e.g. a Service from app.yaml's base values.yaml), leaking them +// into an environment the component was never meant to reach. +func createComponentSubCharts(chartDir string, componentNames []string) error { chartsDir := filepath.Join(chartDir, "charts") if err := os.MkdirAll(chartsDir, 0o750); err != nil { return fmt.Errorf("failed to create charts directory: %w", err) } - for componentName, component := range manifest.Components { - if !componentActiveInEnvironment(component, desiredEnvironment) { - continue - } - + for _, componentName := range componentNames { componentChartDir := filepath.Join(chartsDir, componentName) if err := os.MkdirAll(componentChartDir, 0o750); err != nil { return fmt.Errorf("failed to create component chart directory for %s: %w", componentName, err) @@ -281,12 +296,25 @@ func createComponentAppTemplate(templatesDir string) error { return os.WriteFile(filepath.Join(templatesDir, "app.yaml"), []byte(appTemplate), 0o600) } +// activeComponentNames returns the sorted names of the components that get a +// sub-chart in desiredEnvironment. +func activeComponentNames(manifest *spec.Spec, desiredEnvironment string) []string { + names := make([]string, 0, len(manifest.Components)) + for name, component := range manifest.Components { + if componentActiveInEnvironment(component, desiredEnvironment) { + names = append(names, name) + } + } + slices.Sort(names) + return names +} + // componentActiveInEnvironment reports whether component belongs in the // chart for desiredEnvironment: true when it has no explicit Environments // filter (active everywhere), or desiredEnvironment matches one of them // via [spec.MatchEnvKey] (the same matcher [spec.Resolve] uses). Shared by -// [MapSpecToChartValues] and [createComponentSubCharts] so both agree on -// the active component set. +// [MapSpecToChartValues] and [activeComponentNames] so values and sub-charts +// agree on the active component set. func componentActiveInEnvironment(component spec.Component, desiredEnvironment string) bool { if len(component.Environments) == 0 { return true @@ -327,9 +355,6 @@ func MapSpecToChartValues(m *spec.Spec, desiredEnvironment string, resolved *spe }, } - if component.Role == spec.ComponentRoleJob { - return nil, fmt.Errorf("role %s is not supported yet", component.Role) - } // TODO: Implement handling for component envFile // Exclude Deployah-specific environment variables (those prefixed with DPY_VAR_) and provide the remaining variables to the component @@ -553,15 +578,22 @@ func MapSpecToChartValues(m *spec.Spec, desiredEnvironment string, resolved *spe values[componentName] = componentValues } + resolvedTasks, err := applyTaskChartValues(values, m, desiredEnvironment, resolved) + if err != nil { + return nil, err + } + // Write the deployah.resolved block so the hostname guard can compare - // values across deploys. - if len(resolvedComponents) > 0 { - values["deployah"] = map[string]any{ + // values across deploys, and so plan/guards can see active tasks. + if len(resolvedComponents) > 0 || len(resolvedTasks) > 0 { + deployahVals := map[string]any{ "resolved": map[string]any{ "schemaVersion": resolvedSchemaVersion, "components": resolvedComponents, + "tasks": resolvedTasks, }, } + values["deployah"] = deployahVals } return values, nil diff --git a/internal/helm/generate_task_test.go b/internal/helm/generate_task_test.go new file mode 100644 index 0000000..95c04c3 --- /dev/null +++ b/internal/helm/generate_task_test.go @@ -0,0 +1,559 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package helm + +import ( + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "sigs.k8s.io/yaml" + + "deployah.dev/deployah/internal/k8s" + "deployah.dev/deployah/internal/spec" + + batchv1 "k8s.io/api/batch/v1" + corev1 "k8s.io/api/core/v1" +) + +func taskSpec() *spec.Spec { + backoff := spec.DefaultBackoffLimit + return &spec.Spec{ + APIVersion: spec.CurrentManifestVersion, + Project: "shop", + Components: map[string]spec.Component{ + "api": { + Role: spec.ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + Port: 8080, + Env: map[string]string{"DATABASE_URL": "postgres://db", "LOG": "info"}, + }, + }, + Tasks: map[string]spec.Task{ + "migrate": { + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"migrate", "up"}, + Timeout: spec.DefaultHookTaskTimeout, + BackoffLimit: &backoff, + Env: map[string]string{"LOG": "debug"}, + }, + "smoke": { + From: "api", + On: spec.TaskOnPostDeploy, + Command: []string{"curl", "-f", "http://api/health"}, + Timeout: spec.DefaultHookTaskTimeout, + BackoffLimit: &backoff, + }, + "backfill": { + From: "api", + On: spec.TaskOnManual, + Command: []string{"backfill"}, + Fanout: spec.Fanout{Count: 4, Parallelism: 2}, + }, + }, + } +} + +func TestMapSpecToChartValues_HookTasks(t *testing.T) { + t.Parallel() + + m := taskSpec() + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) + for name, task := range m.Tasks { + if p, ok := m.Components[task.From]; ok { + cp := p + m.Tasks[name] = task.MergeFrom(&cp) + } + } + + vals, err := MapSpecToChartValues(m, "dev", nil) + require.NoError(t, err) + + migrate := mustNestedMap(t, vals, "migrate") + job := mustNestedMap(t, migrate, "job") + assert.Equal(t, true, job["enabled"]) + assert.Equal(t, "pre-install,pre-upgrade", job["hook"]) + assert.Equal(t, 0, job["hookWeight"]) + assert.Equal(t, hookDeletePolicy, job["hookDeletePolicy"]) + assert.Equal(t, 1, job["completions"]) + assert.Equal(t, 1, job["parallelism"]) + assert.Equal(t, spec.DefaultBackoffLimit, job["backoffLimit"]) + assert.Equal(t, 300, job["activeDeadlineSeconds"]) + assert.Equal(t, []string{"migrate", "up"}, migrate["command"]) + assert.Equal(t, map[string]string{"DATABASE_URL": "postgres://db", "LOG": "debug"}, migrate["envVars"]) + + labels, ok := migrate["commonLabels"].(map[string]string) + require.True(t, ok) + assert.Equal(t, "migrate", labels[spec.LabelComponent]) + + smoke := mustNestedMap(t, vals, "smoke") + smokeJob := mustNestedMap(t, smoke, "job") + assert.Equal(t, "post-install,post-upgrade", smokeJob["hook"]) + + _, hasManual := vals["backfill"] + assert.False(t, hasManual, "manual tasks must be absent from chart values") + + deployah := mustNestedMap(t, vals, "deployah") + resolved := mustNestedMap(t, deployah, "resolved") + require.Contains(t, resolved, "tasks") + _, hasSA := deployah["tasks"] + assert.False(t, hasSA, "hook tasks must not request a dedicated ServiceAccount") +} + +func TestMapSpecToChartValues_ManualOnlyOmitsTaskValues(t *testing.T) { + t.Parallel() + + m := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Role: spec.ComponentRoleService, Image: "nginx:latest", Port: 80}, + }, + Tasks: map[string]spec.Task{ + "backfill": {From: "api", On: spec.TaskOnManual, Command: []string{"true"}}, + }, + } + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) + vals, err := MapSpecToChartValues(m, "dev", nil) + require.NoError(t, err) + _, hasBackfill := vals["backfill"] + assert.False(t, hasBackfill) + if deployah, ok := vals["deployah"].(map[string]any); ok { + _, hasSA := deployah["tasks"] + assert.False(t, hasSA) + } +} + +// TestPrepareChart_ChartYAMLImportsOnlySubCharts pins Chart.yaml to the same +// names as [createComponentSubCharts] and [createTaskSubCharts]. A manual task +// and a component scoped to another environment get no sub-chart, so an +// import-values entry under either name would point at a chart that is not +// there. +func TestPrepareChart_ChartYAMLImportsOnlySubCharts(t *testing.T) { + t.Parallel() + + m := taskSpec() + m.Components["worker"] = spec.Component{ + Role: spec.ComponentRoleWorker, + Image: "ghcr.io/acme/shop:1.2.3", + Environments: []string{"prod"}, + } + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) + + cache := NewChartCache(time.Hour) + const environment = "dev" + chartDir, err := PrepareChart(t.Context(), m, environment, nil, cache) + require.NoError(t, err) + t.Cleanup(func() { removeChartDirs(t, cache, m, environment, chartDir) }) + + raw, err := os.ReadFile(filepath.Join(chartDir, "Chart.yaml")) // #nosec G304 -- chartDir is the temp dir PrepareChart just created + require.NoError(t, err) + + var chart struct { + Dependencies []struct { + ImportValues []struct { + Parent string `json:"parent"` + } `json:"import-values"` + } `json:"dependencies"` + } + require.NoError(t, yaml.Unmarshal(raw, &chart)) + require.Len(t, chart.Dependencies, 1) + + parents := make([]string, 0, len(chart.Dependencies[0].ImportValues)) + for _, iv := range chart.Dependencies[0].ImportValues { + parents = append(parents, iv.Parent) + } + // Components first, then tasks, each sorted: the order is part of what + // keeps a regenerated chart byte-identical. + assert.Equal(t, []string{"api", "migrate", "smoke"}, parents, + "manual task backfill and prod-only component worker must not be imported") + assert.DirExists(t, filepath.Join(chartDir, "charts", "migrate")) + assert.NoDirExists(t, filepath.Join(chartDir, "charts", "backfill")) + assert.NoDirExists(t, filepath.Join(chartDir, "charts", "worker")) +} + +// removeChartDirs deletes both the copy PrepareChart returned and the +// directory backing its cache entry. +func removeChartDirs(tb testing.TB, cache *ChartCache, m *spec.Spec, environment, returned string) { + tb.Helper() + removeChartDir(tb, returned) + key, err := cache.GenerateKey(m, environment, nil) + if err != nil { + tb.Logf("cleanup: cache key: %v", err) + return + } + if cached, found := cache.get(key); found { + removeChartDir(tb, cached) + } +} + +func TestHookTasksForChart(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + spec *spec.Spec + environment string + wantWeight map[string]int + }{ + { + name: "after sets hook weight", + spec: &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "seed": {From: "api", On: spec.TaskOnPreDeploy, After: []string{"migrate"}, Command: []string{"true"}}, + "migrate": {From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + }, + }, + environment: "dev", + wantWeight: map[string]int{"migrate": 0, "seed": 1}, + }, + { + name: "skips inherited other environment", + spec: &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest", Environments: []string{"prod"}}, + }, + Tasks: map[string]spec.Task{ + "migrate": {From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + }, + }, + environment: "dev", + wantWeight: map[string]int{}, + }, + { + name: "includes matching environment", + spec: &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest", Environments: []string{"prod"}}, + }, + Tasks: map[string]spec.Task{ + "migrate": {From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + }, + }, + environment: "prod", + wantWeight: map[string]int{"migrate": 0}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got, err := hookTasksForChart(tt.spec, tt.environment, nil) + require.NoError(t, err) + weights := make(map[string]int, len(got)) + for name, rt := range got { + weights[name] = rt.HookWeight + } + assert.Equal(t, tt.wantWeight, weights) + }) + } +} + +func TestMapTaskToChartValues_DigestArgsEphemeralTTL(t *testing.T) { + t.Parallel() + + ttl := 60 + m := &spec.Spec{Project: "shop"} + rt := spec.ResolvedTask{ + Task: spec.Task{ + Image: "nginx@sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", + On: spec.TaskOnPreDeploy, + Command: []string{"migrate"}, + Args: []string{"up"}, + Resources: spec.Resources{ + CPU: spec.MustQuantity("100m"), + Memory: spec.MustQuantity("128Mi"), + EphemeralStorage: spec.MustQuantity("1Gi"), + }, + TTLSecondsAfterFinished: &ttl, + Timeout: spec.DefaultHookTaskTimeout, + }, + HookWeight: 2, + } + + vals, err := mapTaskToChartValues(m, "migrate", rt, "dev") + require.NoError(t, err) + + image := mustNestedMap(t, vals, "image") + assert.Equal(t, "docker.io/library/nginx", image["repository"]) + assert.Equal(t, "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", image["digest"]) + _, hasTag := image["tag"] + assert.False(t, hasTag) + assert.Equal(t, []string{"migrate"}, vals["command"]) + assert.Equal(t, []string{"up"}, vals["args"]) + + resources := mustNestedMap(t, vals, "resources") + requests := mustNestedMap(t, resources, "requests") + assert.Equal(t, "100m", requests["cpu"]) + assert.Equal(t, "128Mi", requests["memory"]) + assert.Equal(t, "1Gi", requests["ephemeral-storage"]) + + job := mustNestedMap(t, vals, "job") + assert.Equal(t, 2, job["hookWeight"]) + assert.Equal(t, 60, job["ttlSecondsAfterFinished"]) + assert.Equal(t, 300, job["activeDeadlineSeconds"]) +} + +func TestHookTasksForChart_Cycle(t *testing.T) { + t.Parallel() + + m := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "a": {From: "api", On: spec.TaskOnPreDeploy, After: []string{"b"}, Command: []string{"true"}}, + "b": {From: "api", On: spec.TaskOnPreDeploy, After: []string{"a"}, Command: []string{"true"}}, + }, + } + _, err := hookTasksForChart(m, "dev", nil) + require.Error(t, err) + assert.Contains(t, err.Error(), "cycle") +} + +func TestMapSpecToChartValues_AppliesTaskProfile(t *testing.T) { + t.Parallel() + + m := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Role: spec.ComponentRoleService, Image: "nginx:latest", Port: 80}, + }, + Tasks: map[string]spec.Task{ + "migrate": {From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + }, + } + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) + task, ok := m.MergedTask("migrate") + require.True(t, ok) + resolved := &spec.ResolvedSpec{ + Tasks: map[string]spec.ResolvedTask{ + "migrate": { + Task: task, + MergedProfile: &spec.PlatformProfile{ + NodeSelector: map[string]string{"workload": "batch"}, + }, + }, + }, + } + vals, err := MapSpecToChartValues(m, "dev", resolved) + require.NoError(t, err) + migrate := mustNestedMap(t, vals, "migrate") + assert.Equal(t, map[string]string{"workload": "batch"}, migrate["nodeSelector"]) +} + +func TestHelmJob_CommandBracesAreData(t *testing.T) { + t.Parallel() + + m := hookRenderSpec(spec.Task{ + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"echo", "{{ .Release.Name }}"}, + }) + job := renderHookJob(t, m, "dev", "migrate") + require.Len(t, job.Spec.Template.Spec.Containers, 1) + assert.Equal(t, []string{"echo", "{{ .Release.Name }}"}, job.Spec.Template.Spec.Containers[0].Command) +} + +func TestHelmJob_EnvBracesAreData(t *testing.T) { + t.Parallel() + + m := hookRenderSpec(spec.Task{ + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"true"}, + Env: map[string]string{"NOTE": "{{ .Release.Name }}"}, + }) + job := renderHookJob(t, m, "dev", "migrate") + require.Len(t, job.Spec.Template.Spec.Containers, 1) + assert.Equal(t, []corev1.EnvVar{ + {Name: "DATABASE_URL", Value: "postgres://db"}, + {Name: "LOG", Value: "info"}, + {Name: "NOTE", Value: "{{ .Release.Name }}"}, + }, job.Spec.Template.Spec.Containers[0].Env) +} + +func TestHelmJob_TTLSecondsAfterFinished(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + ttl *int + want *int32 + }{ + {name: "zero deletes immediately", ttl: new(0), want: new(int32(0))}, + {name: "positive value", ttl: new(60), want: new(int32(60))}, + {name: "omitted", ttl: nil, want: nil}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + m := hookRenderSpec(spec.Task{ + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"true"}, + TTLSecondsAfterFinished: tt.ttl, + }) + job := renderHookJob(t, m, "dev", "migrate") + assert.Equal(t, tt.want, job.Spec.TTLSecondsAfterFinished) + }) + } +} + +func TestHelmJob_ShortNameImageGetsRegistryPrefix(t *testing.T) { + t.Parallel() + + m := hookRenderSpec(spec.Task{ + From: "api", + On: spec.TaskOnPreDeploy, + Image: "nginx:latest", + Command: []string{"true"}, + }) + job := renderHookJob(t, m, "dev", "migrate") + require.Len(t, job.Spec.Template.Spec.Containers, 1) + assert.Equal(t, "docker.io/library/nginx:latest", job.Spec.Template.Spec.Containers[0].Image) +} + +func TestHelmJobPodMatchesBuildTaskJob(t *testing.T) { + t.Parallel() + + task := spec.Task{ + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"migrate", "up"}, + Args: []string{"--strict"}, + Env: map[string]string{"LOG": "debug"}, + Fanout: spec.Fanout{Count: 2, Parallelism: 1}, + Timeout: spec.DefaultHookTaskTimeout, + } + m := hookRenderSpec(task) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) + merged, ok := m.MergedTask("migrate") + require.True(t, ok) + + helmJob := renderHookJob(t, m, "dev", "migrate") + cliJob, err := k8s.BuildTaskJob(k8s.TaskJobOptions{ + Project: m.Project, + Environment: "dev", + Namespace: "default", + TaskName: "migrate", + Task: merged, + }) + require.NoError(t, err) + + helmPod := helmJob.Spec.Template.Spec + cliPod := cliJob.Spec.Template.Spec + require.Len(t, helmPod.Containers, 1) + require.Len(t, cliPod.Containers, 1) + assert.Equal(t, cliPod.Containers[0].Image, helmPod.Containers[0].Image) + assert.Equal(t, cliPod.Containers[0].Command, helmPod.Containers[0].Command) + assert.Equal(t, cliPod.Containers[0].Args, helmPod.Containers[0].Args) + assert.ElementsMatch(t, cliPod.Containers[0].Env, helmPod.Containers[0].Env) + assertResourceListsEqual(t, cliPod.Containers[0].Resources.Requests, helmPod.Containers[0].Resources.Requests) + require.NotNil(t, helmPod.AutomountServiceAccountToken) + require.NotNil(t, cliPod.AutomountServiceAccountToken) + assert.False(t, *helmPod.AutomountServiceAccountToken) + assert.False(t, *cliPod.AutomountServiceAccountToken) + assert.Empty(t, helmPod.ServiceAccountName) + assert.Empty(t, cliPod.ServiceAccountName) + assert.Equal(t, cliPod.RestartPolicy, helmPod.RestartPolicy) + require.NotNil(t, helmJob.Spec.Completions) + require.NotNil(t, cliJob.Spec.Completions) + assert.Equal(t, *cliJob.Spec.Completions, *helmJob.Spec.Completions) + require.NotNil(t, helmJob.Spec.Parallelism) + require.NotNil(t, cliJob.Spec.Parallelism) + assert.Equal(t, *cliJob.Spec.Parallelism, *helmJob.Spec.Parallelism) + require.NotNil(t, helmJob.Spec.BackoffLimit) + require.NotNil(t, cliJob.Spec.BackoffLimit) + assert.Equal(t, *cliJob.Spec.BackoffLimit, *helmJob.Spec.BackoffLimit) + require.NotNil(t, helmJob.Spec.ActiveDeadlineSeconds) + require.NotNil(t, cliJob.Spec.ActiveDeadlineSeconds) + assert.Equal(t, *cliJob.Spec.ActiveDeadlineSeconds, *helmJob.Spec.ActiveDeadlineSeconds) +} + +func hookRenderSpec(task spec.Task) *spec.Spec { + return &spec.Spec{ + APIVersion: spec.CurrentManifestVersion, + Project: "shop", + Components: map[string]spec.Component{ + "api": { + Role: spec.ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + Port: 8080, + Env: map[string]string{"DATABASE_URL": "postgres://db", "LOG": "info"}, + }, + }, + Tasks: map[string]spec.Task{ + "migrate": task, + }, + Environments: map[string]spec.Environment{ + "dev": {}, + }, + } +} + +func renderHookJob(t *testing.T, manifest *spec.Spec, env, taskName string) *batchv1.Job { + t.Helper() + require.NoError(t, spec.FillSpecWithDefaults(manifest, spec.CurrentManifestVersion)) + for name, task := range manifest.Tasks { + if p, ok := manifest.Components[task.From]; ok { + cp := p + manifest.Tasks[name] = task.MergeFrom(&cp) + } + } + + client, err := NewClient(WithNamespace("default")) + require.NoError(t, err) + result, cleanup, err := client.RenderOffline(t.Context(), manifest, env, nil, nil) + require.NoError(t, err) + if cleanup != nil { + t.Cleanup(cleanup) + } + + suffix := "-" + taskName + for _, h := range result.Hooks { + if h == nil || h.Manifest == "" { + continue + } + var job batchv1.Job + if unmarshalErr := yaml.Unmarshal([]byte(h.Manifest), &job); unmarshalErr != nil { + continue + } + if job.Kind == "Job" && strings.HasSuffix(job.Name, suffix) { + return &job + } + } + t.Fatalf("no hook Job ending with %q", suffix) + return nil +} + +func assertResourceListsEqual(t *testing.T, a, b corev1.ResourceList) { + t.Helper() + require.Equal(t, len(a), len(b), "resource list length") + for name, aq := range a { + bq, ok := b[name] + require.True(t, ok, "missing resource %s", name) + assert.True(t, aq.Equal(bq), "resource %s: %s vs %s", name, aq.String(), bq.String()) + } +} diff --git a/internal/helm/generate_test.go b/internal/helm/generate_test.go index 16cbb8b..385eb9f 100644 --- a/internal/helm/generate_test.go +++ b/internal/helm/generate_test.go @@ -287,11 +287,11 @@ func TestMapSpecToChartValues_EnvironmentFilterPrefixMatch(t *testing.T) { comp := serviceComponent() comp.Environments = tt.filter m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Components: map[string]spec.Component{"web": comp}, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) vals, err := MapSpecToChartValues(m, tt.environment, nil) require.NoError(t, err) @@ -312,7 +312,7 @@ func TestMapSpecToChartValues_SelfSignedTLS(t *testing.T) { subdomain := "api" m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "local": {}, @@ -329,7 +329,7 @@ func TestMapSpecToChartValues_SelfSignedTLS(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) resolved := &spec.ResolvedSpec{ Spec: m, @@ -375,7 +375,7 @@ func TestMapSpecToChartValues_SelfSignedTLS_Unmaterialized(t *testing.T) { subdomain := "api" m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "local": {}, @@ -392,7 +392,7 @@ func TestMapSpecToChartValues_SelfSignedTLS_Unmaterialized(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) resolved := &spec.ResolvedSpec{ Spec: m, @@ -418,7 +418,7 @@ func TestMapSpecToChartValues_SecretNameTLS(t *testing.T) { subdomain := "api" m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -435,7 +435,7 @@ func TestMapSpecToChartValues_SecretNameTLS(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) resolved := &spec.ResolvedSpec{ Spec: m, @@ -466,7 +466,7 @@ func TestMapSpecToChartValues_CertManagerTLS(t *testing.T) { subdomain := "api" m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -483,7 +483,7 @@ func TestMapSpecToChartValues_CertManagerTLS(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) resolved := &spec.ResolvedSpec{ Spec: m, @@ -514,7 +514,7 @@ func TestMapSpecToChartValues_Autoscaling(t *testing.T) { t.Parallel() m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -536,7 +536,7 @@ func TestMapSpecToChartValues_Autoscaling(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) vals, err := MapSpecToChartValues(m, "production", nil) require.NoError(t, err) @@ -556,7 +556,7 @@ func TestMapSpecToChartValues_Profiles(t *testing.T) { t.Parallel() m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -569,7 +569,7 @@ func TestMapSpecToChartValues_Profiles(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) resolved := &spec.ResolvedSpec{ Spec: m, @@ -643,7 +643,7 @@ func TestMapSpecToChartValues_StatefulComponent(t *testing.T) { replicas := 2 m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -662,7 +662,7 @@ func TestMapSpecToChartValues_StatefulComponent(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) resolved := &spec.ResolvedSpec{ Spec: m, @@ -713,7 +713,7 @@ func TestMapSpecToChartValues_StatefulIdentityOnly(t *testing.T) { replicas := 2 m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -728,7 +728,7 @@ func TestMapSpecToChartValues_StatefulIdentityOnly(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) vals, err := MapSpecToChartValues(m, "production", nil) require.NoError(t, err) @@ -746,7 +746,7 @@ func TestMapSpecToChartValues_StatelessWithPersistence(t *testing.T) { t.Parallel() m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -764,7 +764,7 @@ func TestMapSpecToChartValues_StatelessWithPersistence(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) vals, err := MapSpecToChartValues(m, "production", nil) require.NoError(t, err) @@ -784,7 +784,7 @@ func TestMapSpecToChartValues_Replicas(t *testing.T) { replicas := 3 m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -798,7 +798,7 @@ func TestMapSpecToChartValues_Replicas(t *testing.T) { }, }, } - require.NoError(t, spec.FillSpecWithDefaults(m, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(m, spec.CurrentManifestVersion)) vals, err := MapSpecToChartValues(m, "production", nil) require.NoError(t, err) diff --git a/internal/helm/helm_error_test.go b/internal/helm/helm_error_test.go index 725c235..2f44da2 100644 --- a/internal/helm/helm_error_test.go +++ b/internal/helm/helm_error_test.go @@ -223,11 +223,11 @@ func TestInstallApp_PendingReleaseRejects(t *testing.T) { } manifest := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "pending-app", Components: map[string]spec.Component{"web": serviceComponent()}, } - require.NoError(t, spec.FillSpecWithDefaults(manifest, "v1-alpha.4")) + require.NoError(t, spec.FillSpecWithDefaults(manifest, spec.CurrentManifestVersion)) releaseName := GenerateReleaseName(manifest.Project, "production") now := time.Now() diff --git a/internal/helm/task.go b/internal/helm/task.go new file mode 100644 index 0000000..640493d --- /dev/null +++ b/internal/helm/task.go @@ -0,0 +1,190 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package helm + +import ( + "fmt" + "os" + "path/filepath" + "slices" + "strings" + + "deployah.dev/deployah/internal/spec" +) + +const hookDeletePolicy = "before-hook-creation,hook-succeeded" + +// createTaskSubCharts creates a sub-chart directory for each name in +// taskNames, as returned by [hookTaskNames]. Manual tasks and tasks from +// other environments are absent from that list and get no subchart. +func createTaskSubCharts(chartDir string, taskNames []string) error { + chartsDir := filepath.Join(chartDir, "charts") + if err := os.MkdirAll(chartsDir, 0o750); err != nil { + return fmt.Errorf("failed to create charts directory: %w", err) + } + + for _, name := range taskNames { + taskChartDir := filepath.Join(chartsDir, name) + if err := os.MkdirAll(taskChartDir, 0o750); err != nil { + return fmt.Errorf("failed to create task chart directory for %s: %w", name, err) + } + if err := createComponentChartYAML(taskChartDir, name); err != nil { + return fmt.Errorf("failed to create Chart.yaml for task %s: %w", name, err) + } + templatesDir := filepath.Join(taskChartDir, "templates") + if err := os.MkdirAll(templatesDir, 0o750); err != nil { + return fmt.Errorf("failed to create templates directory for task %s: %w", name, err) + } + if err := createTaskJobTemplate(templatesDir); err != nil { + return fmt.Errorf("failed to create job.yaml template for task %s: %w", name, err) + } + } + return nil +} + +func createTaskJobTemplate(templatesDir string) error { + body := `{{- include "deployah.job" . -}}` + return os.WriteFile(filepath.Join(templatesDir, "job.yaml"), []byte(body), 0o600) +} + +// hookTaskNames returns the sorted names of the hook tasks that get a +// sub-chart in this environment. +func hookTaskNames(m *spec.Spec, desiredEnvironment string, resolved *spec.ResolvedSpec) ([]string, error) { + hooks, err := hookTasksForChart(m, desiredEnvironment, resolved) + if err != nil { + return nil, err + } + names := make([]string, 0, len(hooks)) + for name := range hooks { + names = append(names, name) + } + slices.Sort(names) + return names, nil +} + +// hookTasksForChart returns merged hook tasks that belong in this +// environment. Manual tasks are omitted. +func hookTasksForChart(m *spec.Spec, desiredEnvironment string, resolved *spec.ResolvedSpec) (map[string]spec.ResolvedTask, error) { + all, err := spec.EffectiveTasks(m, desiredEnvironment, resolved) + if err != nil { + return nil, err + } + out := make(map[string]spec.ResolvedTask, len(all)) + for name, rt := range all { + if rt.Task.On.IsHook() { + out[name] = rt + } + } + return out, nil +} + +func applyTaskChartValues(values map[string]any, m *spec.Spec, desiredEnvironment string, resolved *spec.ResolvedSpec) (map[string]any, error) { + resolvedTasks := make(map[string]any) + hooks, err := hookTasksForChart(m, desiredEnvironment, resolved) + if err != nil { + return nil, err + } + for name, rt := range hooks { + taskValues, mapErr := mapTaskToChartValues(m, name, rt, desiredEnvironment) + if mapErr != nil { + return nil, fmt.Errorf("task %s: %w", name, mapErr) + } + values[name] = taskValues + resolvedTasks[name] = map[string]any{ + "on": string(rt.Task.On), + "hookWeight": rt.HookWeight, + "timeout": rt.Task.Timeout, + } + } + return resolvedTasks, nil +} + +func mapTaskToChartValues(m *spec.Spec, name string, rt spec.ResolvedTask, desiredEnvironment string) (map[string]any, error) { + fields, err := spec.NewTaskJobSpec(rt.Task, 0, 0) + if err != nil { + return nil, err + } + image, tag := parseContainerImage(fields.Image) + + requests := map[string]any{} + if fields.Resources.CPU != nil && !fields.Resources.CPU.IsZero() { + requests["cpu"] = fields.Resources.CPU.String() + } + if fields.Resources.Memory != nil && !fields.Resources.Memory.IsZero() { + requests["memory"] = fields.Resources.Memory.String() + } + if fields.Resources.EphemeralStorage != nil && !fields.Resources.EphemeralStorage.IsZero() { + requests["ephemeral-storage"] = fields.Resources.EphemeralStorage.String() + } + resources := map[string]any{} + if len(requests) > 0 { + resources["requests"] = requests + } + + imageValues := map[string]any{"repository": image} + if tag != "" { + if strings.HasPrefix(tag, "sha256:") { + imageValues["digest"] = tag + } else { + imageValues["tag"] = tag + } + } + + job := map[string]any{ + "enabled": true, + "hook": rt.Task.HelmHookEvents(), + "hookWeight": rt.HookWeight, + "hookDeletePolicy": hookDeletePolicy, + "completions": int(fields.Completions), + "parallelism": int(fields.Parallelism), + "backoffLimit": int(fields.BackoffLimit), + } + if fields.ActiveDeadlineSeconds != nil { + job["activeDeadlineSeconds"] = int(*fields.ActiveDeadlineSeconds) + } + if fields.TTLSecondsAfterFinished != nil { + job["ttlSecondsAfterFinished"] = int(*fields.TTLSecondsAfterFinished) + } + + values := map[string]any{ + "commonLabels": map[string]string{ + spec.LabelProject: m.Project, + spec.LabelComponent: name, + spec.LabelEnvironment: spec.NormalizeEnv(desiredEnvironment).K8sSafe, + }, + "commonAnnotations": map[string]string{ + spec.AnnotationSource: spec.SourceSpec, + spec.AnnotationProject: m.Project, + }, + "image": imageValues, + "resources": resources, + "job": job, + "service": map[string]any{ + "enabled": false, + }, + } + if len(fields.Command) > 0 { + values["command"] = fields.Command + } + if len(fields.Args) > 0 { + values["args"] = fields.Args + } + if len(fields.Env) > 0 { + values["envVars"] = fields.Env + } + if applyErr := applyMergedProfile(values, rt.MergedProfile); applyErr != nil { + return nil, applyErr + } + return values, nil +} diff --git a/internal/k8s/jobs.go b/internal/k8s/jobs.go new file mode 100644 index 0000000..7aa5e4c --- /dev/null +++ b/internal/k8s/jobs.go @@ -0,0 +1,289 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package k8s + +import ( + "context" + "errors" + "fmt" + "maps" + "slices" + "time" + + "k8s.io/apimachinery/pkg/util/wait" + "k8s.io/client-go/kubernetes" + + "deployah.dev/deployah/internal/spec" + + batchv1 "k8s.io/api/batch/v1" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +const ( + jobGenerateNameMax = 57 + // jobNotFoundLimit is how many consecutive Get NotFound results to + // retry before treating the Job as gone. Covers brief apiserver lag + // after Create without spinning until ctx times out if the Job was + // deleted. + jobNotFoundLimit = 5 +) + +// TaskJobOptions controls a CLI-created Job. +type TaskJobOptions struct { + Project string + Environment string + Namespace string + TaskName string + Task spec.Task + Count int + Parallelism int + Profile *spec.PlatformProfile +} + +// BuildTaskJob builds an Indexed batch/v1 Job for a CLI run. The name is +// left empty; GenerateName is set so concurrent runs do not collide. +// serviceAccountName is omitted so the pod uses the namespace default +// ServiceAccount. +func BuildTaskJob(opts TaskJobOptions) (*batchv1.Job, error) { + fields, err := spec.NewTaskJobSpec(opts.Task, opts.Count, opts.Parallelism) + if err != nil { + return nil, err + } + + envName := spec.NormalizeEnv(opts.Environment).K8sSafe + release := opts.Project + "-" + envName + + ttl := int32(spec.DefaultCLIJobTTLSeconds) + if fields.TTLSecondsAfterFinished != nil { + ttl = *fields.TTLSecondsAfterFinished + } + + envVars := make([]corev1.EnvVar, 0, len(fields.Env)) + for _, k := range slices.Sorted(maps.Keys(fields.Env)) { + envVars = append(envVars, corev1.EnvVar{Name: k, Value: fields.Env[k]}) + } + + container := corev1.Container{ + Name: opts.TaskName, + Image: fields.Image, + Env: envVars, + } + if len(fields.Command) > 0 { + container.Command = fields.Command + } + if len(fields.Args) > 0 { + container.Args = fields.Args + } + if req := resourceList(fields.Resources); len(req) > 0 { + container.Resources.Requests = req + } + + podLabels := map[string]string{ + spec.LabelProject: opts.Project, + spec.LabelComponent: opts.TaskName, + spec.LabelEnvironment: envName, + spec.LabelManagedBy: spec.ManagedByValue, + } + var podAnnotations map[string]string + podSpec := corev1.PodSpec{ + RestartPolicy: corev1.RestartPolicyOnFailure, + AutomountServiceAccountToken: new(false), + Containers: []corev1.Container{container}, + } + applyProfileToPod(&podSpec, podLabels, &podAnnotations, opts.Profile) + + job := &batchv1.Job{ + GenerateName: jobGenerateName(release, opts.TaskName), + Namespace: opts.Namespace, + Labels: map[string]string{ + spec.LabelProject: opts.Project, + spec.LabelComponent: opts.TaskName, + spec.LabelEnvironment: envName, + spec.LabelManagedBy: spec.ManagedByValue, + }, + Annotations: map[string]string{ + spec.AnnotationSource: spec.SourceSpec, + spec.AnnotationProject: opts.Project, + }, + Spec: batchv1.JobSpec{ + CompletionMode: new(batchv1.IndexedCompletion), + Completions: new(fields.Completions), + Parallelism: new(fields.Parallelism), + BackoffLimit: new(fields.BackoffLimit), + TTLSecondsAfterFinished: new(ttl), + ActiveDeadlineSeconds: fields.ActiveDeadlineSeconds, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Labels: podLabels, + Annotations: podAnnotations, + }, + Spec: podSpec, + }, + }, + } + return job, nil +} + +func applyProfileToPod(pod *corev1.PodSpec, labels map[string]string, annotations *map[string]string, profile *spec.PlatformProfile) { + if profile == nil { + return + } + if len(profile.NodeSelector) > 0 { + pod.NodeSelector = maps.Clone(profile.NodeSelector) + } + if len(profile.Tolerations) > 0 { + pod.Tolerations = slices.Clone(profile.Tolerations) + } + if len(profile.PodLabels) > 0 { + maps.Copy(labels, profile.PodLabels) + } + if len(profile.PodAnnotations) > 0 { + *annotations = maps.Clone(profile.PodAnnotations) + } + if profile.SecurityContext != nil { + pod.SecurityContext = profile.SecurityContext.DeepCopy() + } + if profile.ContainerSecurityContext != nil && len(pod.Containers) > 0 { + pod.Containers[0].SecurityContext = profile.ContainerSecurityContext.DeepCopy() + } +} + +func jobGenerateName(release, task string) string { + prefix := release + "-" + task + "-" + if len(prefix) > jobGenerateNameMax { + prefix = prefix[:jobGenerateNameMax] + } + return prefix +} + +func resourceList(res spec.Resources) corev1.ResourceList { + out := corev1.ResourceList{} + if res.CPU != nil && !res.CPU.IsZero() { + out[corev1.ResourceCPU] = *res.CPU + } + if res.Memory != nil && !res.Memory.IsZero() { + out[corev1.ResourceMemory] = *res.Memory + } + if res.EphemeralStorage != nil && !res.EphemeralStorage.IsZero() { + out[corev1.ResourceEphemeralStorage] = *res.EphemeralStorage + } + return out +} + +// CreateTaskJob creates the Job and returns the server copy (with Name). +func CreateTaskJob(ctx context.Context, cs kubernetes.Interface, job *batchv1.Job) (*batchv1.Job, error) { + created, err := cs.BatchV1().Jobs(job.Namespace).Create(ctx, job, metav1.CreateOptions{}) + if err != nil { + return nil, fmt.Errorf("create job: %w", err) + } + return created, nil +} + +// WaitForJob waits until the Job succeeds or fails. ctx should already +// carry the session timeout. Get errors that are not a permanent API +// status (Forbidden, Unauthorized, Invalid, BadRequest) are retried +// until ctx is done. Consecutive NotFound results are retried a few +// times, then treated as a permanent miss (the Job was deleted or never +// became visible). Every failure is wrapped as "wait for job ". +func WaitForJob(ctx context.Context, cs kubernetes.Interface, namespace, name string) error { + notFound := 0 + err := wait.PollUntilContextCancel(ctx, time.Second, true, func(ctx context.Context) (bool, error) { + job, err := cs.BatchV1().Jobs(namespace).Get(ctx, name, metav1.GetOptions{}) + if err != nil { + if ctx.Err() != nil { + return false, err + } + if apierrors.IsNotFound(err) { + notFound++ + if notFound >= jobNotFoundLimit { + return false, err + } + return false, nil + } + notFound = 0 + if isPermanentJobGet(err) { + return false, err + } + return false, nil + } + notFound = 0 + want := int32(1) + if job.Spec.Completions != nil { + want = *job.Spec.Completions + } + if job.Status.Succeeded >= want { + return true, nil + } + for _, cond := range job.Status.Conditions { + if cond.Type == batchv1.JobFailed && cond.Status == corev1.ConditionTrue { + msg := cond.Message + if msg == "" { + msg = cond.Reason + } + return false, fmt.Errorf("job %s failed: %s", name, msg) + } + } + return false, nil + }) + if err != nil { + return fmt.Errorf("wait for job %s: %w", name, err) + } + return nil +} + +func isPermanentJobGet(err error) bool { + return apierrors.IsForbidden(err) || + apierrors.IsUnauthorized(err) || + apierrors.IsInvalid(err) || + apierrors.IsBadRequest(err) +} + +// ListJobs returns Jobs labeled with project and environment. +func ListJobs(ctx context.Context, cs kubernetes.Interface, namespace, project, environment string) ([]batchv1.Job, error) { + selector, err := BuildLabelSelector(project, environment) + if err != nil { + return nil, fmt.Errorf("build job selector: %w", err) + } + list, err := cs.BatchV1().Jobs(namespace).List(ctx, metav1.ListOptions{ + LabelSelector: selector.String(), + }) + if err != nil { + return nil, fmt.Errorf("list jobs: %w", err) + } + return list.Items, nil +} + +// DeleteJobs deletes Jobs labeled with project and environment. +// A NotFound result is ignored (the Job is already gone). Other delete +// errors are collected with [errors.Join] so one failure does not skip +// the rest. +func DeleteJobs(ctx context.Context, cs kubernetes.Interface, namespace, project, environment string) error { + jobs, err := ListJobs(ctx, cs, namespace, project, environment) + if err != nil { + return err + } + propagation := metav1.DeletePropagationBackground + var errs []error + for i := range jobs { + job := &jobs[i] + if delErr := cs.BatchV1().Jobs(namespace).Delete(ctx, job.Name, metav1.DeleteOptions{ + PropagationPolicy: &propagation, + }); delErr != nil && !apierrors.IsNotFound(delErr) { + errs = append(errs, fmt.Errorf("delete job %s: %w", job.Name, delErr)) + } + } + return errors.Join(errs...) +} diff --git a/internal/k8s/jobs_test.go b/internal/k8s/jobs_test.go new file mode 100644 index 0000000..e4adb48 --- /dev/null +++ b/internal/k8s/jobs_test.go @@ -0,0 +1,580 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package k8s + +import ( + "context" + "errors" + "strings" + "testing" + "testing/synctest" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/client-go/kubernetes/fake" + + "deployah.dev/deployah/internal/spec" + + batchv1 "k8s.io/api/batch/v1" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + k8stesting "k8s.io/client-go/testing" +) + +func TestBuildTaskJob_IndexedAndLabels(t *testing.T) { + t.Parallel() + + job, err := BuildTaskJob(TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "backfill", + Task: spec.Task{ + Image: "busybox:1.36", + Command: []string{"backfill"}, + Env: map[string]string{"MODE": "full"}, + Fanout: spec.Fanout{Count: 4, Parallelism: 2}, + }, + }) + require.NoError(t, err) + + assert.Equal(t, "shop-dev-backfill-", job.GenerateName) + assert.Empty(t, job.Name) + require.NotNil(t, job.Spec.CompletionMode) + assert.Equal(t, batchv1.IndexedCompletion, *job.Spec.CompletionMode) + require.NotNil(t, job.Spec.Completions) + assert.Equal(t, int32(4), *job.Spec.Completions) + require.NotNil(t, job.Spec.Parallelism) + assert.Equal(t, int32(2), *job.Spec.Parallelism) + assert.Equal(t, "shop", job.Labels[spec.LabelProject]) + assert.Equal(t, "backfill", job.Labels[spec.LabelComponent]) + assert.Equal(t, "dev", job.Labels[spec.LabelEnvironment]) + require.Len(t, job.Spec.Template.Spec.Containers, 1) + assert.Equal(t, "busybox:1.36", job.Spec.Template.Spec.Containers[0].Image) + assert.Equal(t, []string{"backfill"}, job.Spec.Template.Spec.Containers[0].Command) + assert.Equal(t, []corev1.EnvVar{{Name: "MODE", Value: "full"}}, job.Spec.Template.Spec.Containers[0].Env) + require.NotNil(t, job.Spec.Template.Spec.AutomountServiceAccountToken) + assert.False(t, *job.Spec.Template.Spec.AutomountServiceAccountToken) + assert.Empty(t, job.Spec.Template.Spec.ServiceAccountName) + require.NotNil(t, job.Spec.TTLSecondsAfterFinished) + assert.Equal(t, int32(spec.DefaultCLIJobTTLSeconds), *job.Spec.TTLSecondsAfterFinished) +} + +func TestBuildTaskJob_EnvSorted(t *testing.T) { + t.Parallel() + + job, err := BuildTaskJob(TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "migrate", + Task: spec.Task{ + Image: "busybox:1.36", + Command: []string{"true"}, + Env: map[string]string{"LOG": "debug", "DATABASE_URL": "postgres://db", "A": "1"}, + }, + }) + require.NoError(t, err) + assert.Equal(t, []corev1.EnvVar{ + {Name: "A", Value: "1"}, + {Name: "DATABASE_URL", Value: "postgres://db"}, + {Name: "LOG", Value: "debug"}, + }, job.Spec.Template.Spec.Containers[0].Env) +} + +func TestBuildTaskJob_FlagOverrides(t *testing.T) { + t.Parallel() + + job, err := BuildTaskJob(TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "migrate", + Task: spec.Task{ + Image: "busybox:1.36", + Command: []string{"true"}, + Fanout: spec.Fanout{Count: 1, Parallelism: 1}, + }, + Count: 3, + Parallelism: 2, + }) + require.NoError(t, err) + require.NotNil(t, job.Spec.Completions) + assert.Equal(t, int32(3), *job.Spec.Completions) + require.NotNil(t, job.Spec.Parallelism) + assert.Equal(t, int32(2), *job.Spec.Parallelism) +} + +func TestJobGenerateName_Truncates(t *testing.T) { + t.Parallel() + + got := jobGenerateName("shop-dev", "backfill") + assert.Equal(t, "shop-dev-backfill-", got) + + long := jobGenerateName(strings.Repeat("a", 40), strings.Repeat("b", 40)) + assert.Equal(t, jobGenerateNameMax, len(long)) + assert.Equal(t, (strings.Repeat("a", 40) + "-" + strings.Repeat("b", 40) + "-")[:jobGenerateNameMax], long) +} + +func TestBuildTaskJob_EphemeralStorageArgsAndTTL(t *testing.T) { + t.Parallel() + + ttl := 30 + job, err := BuildTaskJob(TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "migrate", + Task: spec.Task{ + Image: "busybox:1.36", + Command: []string{"migrate"}, + Args: []string{"up"}, + Resources: spec.Resources{ + CPU: spec.MustQuantity("100m"), + Memory: spec.MustQuantity("128Mi"), + EphemeralStorage: spec.MustQuantity("1Gi"), + }, + TTLSecondsAfterFinished: &ttl, + }, + }) + require.NoError(t, err) + require.Len(t, job.Spec.Template.Spec.Containers, 1) + assert.Equal(t, []string{"up"}, job.Spec.Template.Spec.Containers[0].Args) + req := job.Spec.Template.Spec.Containers[0].Resources.Requests + assert.True(t, spec.MustQuantity("100m").Equal(req[corev1.ResourceCPU])) + assert.True(t, spec.MustQuantity("128Mi").Equal(req[corev1.ResourceMemory])) + assert.True(t, spec.MustQuantity("1Gi").Equal(req[corev1.ResourceEphemeralStorage])) + require.NotNil(t, job.Spec.TTLSecondsAfterFinished) + assert.Equal(t, int32(30), *job.Spec.TTLSecondsAfterFinished) +} + +func TestCreateTaskJob_Error(t *testing.T) { + t.Parallel() + + cs := fake.NewSimpleClientset() + cs.PrependReactor("create", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + return true, nil, errors.New("quota exceeded") + }) + _, err := CreateTaskJob(t.Context(), cs, namedJob(t, TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "backfill", + Task: spec.Task{Image: "busybox:1.36", Command: []string{"true"}}, + }, "shop-dev-backfill-create")) + require.Error(t, err) + assert.Contains(t, err.Error(), "quota exceeded") +} + +func TestCreateTaskJob_UniqueNames(t *testing.T) { + t.Parallel() + + cs := fake.NewSimpleClientset() + opts := TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "backfill", + Task: spec.Task{Image: "busybox:1.36", Command: []string{"true"}}, + } + + first, err := CreateTaskJob(t.Context(), cs, namedJob(t, opts, "shop-dev-backfill-a")) + require.NoError(t, err) + second, err := CreateTaskJob(t.Context(), cs, namedJob(t, opts, "shop-dev-backfill-b")) + require.NoError(t, err) + assert.NotEqual(t, first.Name, second.Name) +} + +func namedJob(t *testing.T, opts TaskJobOptions, name string) *batchv1.Job { + t.Helper() + job, err := BuildTaskJob(opts) + require.NoError(t, err) + job.Name = name + job.GenerateName = "" + return job +} + +func TestWaitForJob_SuccessAndFailure(t *testing.T) { + t.Parallel() + + t.Run("succeeds when completions are met", func(t *testing.T) { + t.Parallel() + job := &batchv1.Job{ + Name: "ok", Namespace: "default", + Spec: batchv1.JobSpec{Completions: new(int32(1))}, + Status: batchv1.JobStatus{Succeeded: 1}, + } + cs := fake.NewSimpleClientset(job) + require.NoError(t, WaitForJob(t.Context(), cs, "default", "ok")) + }) + + t.Run("fails when the job condition is Failed", func(t *testing.T) { + t.Parallel() + job := &batchv1.Job{ + Name: "bad", Namespace: "default", + Spec: batchv1.JobSpec{Completions: new(int32(1))}, + Status: batchv1.JobStatus{ + Conditions: []batchv1.JobCondition{{ + Type: batchv1.JobFailed, + Status: "True", + Message: "backoff limit exceeded", + }}, + }, + } + cs := fake.NewSimpleClientset(job) + err := WaitForJob(t.Context(), cs, "default", "bad") + require.Error(t, err) + assert.Contains(t, err.Error(), "backoff limit exceeded") + }) + + t.Run("failed condition without message uses reason", func(t *testing.T) { + t.Parallel() + job := &batchv1.Job{ + Name: "bad", Namespace: "default", + Spec: batchv1.JobSpec{Completions: new(int32(1))}, + Status: batchv1.JobStatus{ + Conditions: []batchv1.JobCondition{{ + Type: batchv1.JobFailed, + Status: "True", + Reason: "BackoffLimitExceeded", + }}, + }, + } + cs := fake.NewSimpleClientset(job) + err := WaitForJob(t.Context(), cs, "default", "bad") + require.Error(t, err) + assert.Contains(t, err.Error(), "BackoffLimitExceeded") + }) + + t.Run("nil completions treats one success as done", func(t *testing.T) { + t.Parallel() + job := &batchv1.Job{ + Name: "ok", + Namespace: "default", + Status: batchv1.JobStatus{Succeeded: 1}, + } + cs := fake.NewSimpleClientset(job) + require.NoError(t, WaitForJob(t.Context(), cs, "default", "ok")) + }) +} + +func TestWaitForJob_RetriesTransientGet(t *testing.T) { + t.Parallel() + + synctest.Test(t, func(t *testing.T) { + job := &batchv1.Job{ + Name: "ok", Namespace: "default", + Spec: batchv1.JobSpec{Completions: new(int32(1))}, + Status: batchv1.JobStatus{Succeeded: 1}, + } + cs := fake.NewSimpleClientset(job) + gets := 0 + cs.PrependReactor("get", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + gets++ + if gets == 1 { + return true, nil, apierrors.NewTooManyRequests("slow down", 1) + } + return false, nil, nil + }) + require.NoError(t, WaitForJob(t.Context(), cs, "default", "ok")) + assert.GreaterOrEqual(t, gets, 2) + }) +} + +func TestWaitForJob_RetriesBriefNotFound(t *testing.T) { + t.Parallel() + + synctest.Test(t, func(t *testing.T) { + job := &batchv1.Job{ + Name: "ok", Namespace: "default", + Spec: batchv1.JobSpec{Completions: new(int32(1))}, + Status: batchv1.JobStatus{Succeeded: 1}, + } + cs := fake.NewSimpleClientset(job) + gets := 0 + cs.PrependReactor("get", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + gets++ + if gets == 1 { + return true, nil, apierrors.NewNotFound( + schema.GroupResource{Group: "batch", Resource: "jobs"}, + "ok", + ) + } + return false, nil, nil + }) + require.NoError(t, WaitForJob(t.Context(), cs, "default", "ok")) + assert.GreaterOrEqual(t, gets, 2) + }) +} + +func TestWaitForJob_GivesUpAfterConsecutiveNotFound(t *testing.T) { + t.Parallel() + + synctest.Test(t, func(t *testing.T) { + cs := fake.NewSimpleClientset() + err := WaitForJob(t.Context(), cs, "default", "gone") + require.Error(t, err) + assert.ErrorContains(t, err, "wait for job") + assert.True(t, apierrors.IsNotFound(err)) + }) +} + +func TestWaitForJob_PermanentGetError(t *testing.T) { + t.Parallel() + + cs := fake.NewSimpleClientset() + cs.PrependReactor("get", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + return true, nil, apierrors.NewForbidden( + schema.GroupResource{Group: "batch", Resource: "jobs"}, + "denied", + errors.New("denied"), + ) + }) + err := WaitForJob(t.Context(), cs, "default", "denied") + require.Error(t, err) + assert.ErrorContains(t, err, "wait for job") +} + +func TestWaitForJob_RetriesTransportError(t *testing.T) { + t.Parallel() + + synctest.Test(t, func(t *testing.T) { + job := &batchv1.Job{ + Name: "ok", Namespace: "default", + Spec: batchv1.JobSpec{Completions: new(int32(1))}, + Status: batchv1.JobStatus{Succeeded: 1}, + } + cs := fake.NewSimpleClientset(job) + gets := 0 + cs.PrependReactor("get", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + gets++ + if gets == 1 { + return true, nil, errors.New("connection reset") + } + return false, nil, nil + }) + require.NoError(t, WaitForJob(t.Context(), cs, "default", "ok")) + assert.GreaterOrEqual(t, gets, 2) + }) +} + +func TestWaitForJob_TimeoutNamesJob(t *testing.T) { + t.Parallel() + + synctest.Test(t, func(t *testing.T) { + job := &batchv1.Job{ + Name: "slow", Namespace: "default", + Spec: batchv1.JobSpec{Completions: new(int32(1))}, + } + cs := fake.NewSimpleClientset(job) + ctx, cancel := context.WithTimeout(t.Context(), time.Second) + defer cancel() + err := WaitForJob(ctx, cs, "default", "slow") + require.Error(t, err) + assert.ErrorIs(t, err, context.DeadlineExceeded) + assert.ErrorContains(t, err, "wait for job slow") + }) +} + +func TestListJobs(t *testing.T) { + t.Parallel() + + keep := &batchv1.Job{ + Name: "other", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "other", spec.LabelEnvironment: "dev"}, + } + match := &batchv1.Job{ + Name: "shop-job", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "shop", spec.LabelEnvironment: "dev"}, + } + cs := fake.NewSimpleClientset(keep, match) + got, err := ListJobs(t.Context(), cs, "default", "shop", "dev") + require.NoError(t, err) + require.Len(t, got, 1) + assert.Equal(t, "shop-job", got[0].Name) +} + +func TestListJobs_Error(t *testing.T) { + t.Parallel() + + cs := fake.NewSimpleClientset() + cs.PrependReactor("list", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + return true, nil, errors.New("apiserver timeout") + }) + _, err := ListJobs(t.Context(), cs, "default", "shop", "dev") + require.Error(t, err) + assert.Contains(t, err.Error(), "apiserver timeout") + + err = DeleteJobs(t.Context(), cs, "default", "shop", "dev") + require.Error(t, err) + assert.Contains(t, err.Error(), "apiserver timeout") +} + +func TestDeleteJobs(t *testing.T) { + t.Parallel() + + keep := &batchv1.Job{ + Name: "other", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "other", spec.LabelEnvironment: "dev"}, + } + drop := &batchv1.Job{ + Name: "shop-job", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "shop", spec.LabelEnvironment: "dev"}, + } + cs := fake.NewSimpleClientset(keep, drop) + require.NoError(t, DeleteJobs(t.Context(), cs, "default", "shop", "dev")) + + list, err := cs.BatchV1().Jobs("default").List(t.Context(), metav1.ListOptions{}) + require.NoError(t, err) + require.Len(t, list.Items, 1) + assert.Equal(t, "other", list.Items[0].Name) +} + +func TestDeleteJobs_IgnoresNotFound(t *testing.T) { + t.Parallel() + + gone := &batchv1.Job{ + Name: "shop-gone", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "shop", spec.LabelEnvironment: "dev"}, + } + stay := &batchv1.Job{ + Name: "shop-stay", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "shop", spec.LabelEnvironment: "dev"}, + } + cs := fake.NewSimpleClientset(gone, stay) + cs.PrependReactor("delete", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + del, ok := action.(k8stesting.DeleteAction) + if !ok || del.GetName() != "shop-gone" { + return false, nil, nil + } + return true, nil, apierrors.NewNotFound( + schema.GroupResource{Group: "batch", Resource: "jobs"}, + "shop-gone", + ) + }) + require.NoError(t, DeleteJobs(t.Context(), cs, "default", "shop", "dev")) + + list, err := cs.BatchV1().Jobs("default").List(t.Context(), metav1.ListOptions{}) + require.NoError(t, err) + require.Len(t, list.Items, 1) + assert.Equal(t, "shop-gone", list.Items[0].Name) +} + +func TestDeleteJobs_JoinsDeleteErrors(t *testing.T) { + t.Parallel() + + first := &batchv1.Job{ + Name: "shop-a", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "shop", spec.LabelEnvironment: "dev"}, + } + second := &batchv1.Job{ + Name: "shop-b", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "shop", spec.LabelEnvironment: "dev"}, + } + cs := fake.NewSimpleClientset(first, second) + cs.PrependReactor("delete", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + del, ok := action.(k8stesting.DeleteAction) + if !ok { + return false, nil, nil + } + return true, nil, apierrors.NewForbidden( + schema.GroupResource{Group: "batch", Resource: "jobs"}, + del.GetName(), + errors.New("denied"), + ) + }) + err := DeleteJobs(t.Context(), cs, "default", "shop", "dev") + require.Error(t, err) + assert.ErrorContains(t, err, "shop-a") + assert.ErrorContains(t, err, "shop-b") +} + +func TestDeleteJobs_BackgroundPropagation(t *testing.T) { + t.Parallel() + + job := &batchv1.Job{ + Name: "shop-job", Namespace: "default", + Labels: map[string]string{spec.LabelProject: "shop", spec.LabelEnvironment: "dev"}, + } + cs := fake.NewSimpleClientset(job) + var got *metav1.DeletionPropagation + cs.PrependReactor("delete", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) { + del, ok := action.(k8stesting.DeleteAction) + if !ok { + return false, nil, nil + } + opts := del.GetDeleteOptions() + got = opts.PropagationPolicy + return false, nil, nil + }) + require.NoError(t, DeleteJobs(t.Context(), cs, "default", "shop", "dev")) + require.NotNil(t, got) + assert.Equal(t, metav1.DeletePropagationBackground, *got) +} + +func TestBuildTaskJob_InvalidTimeout(t *testing.T) { + t.Parallel() + + _, err := BuildTaskJob(TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "migrate", + Task: spec.Task{ + Image: "busybox:1.36", + Command: []string{"true"}, + Timeout: "not-a-duration", + }, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "timeout") +} + +func TestBuildTaskJob_AppliesProfile(t *testing.T) { + t.Parallel() + + job, err := BuildTaskJob(TaskJobOptions{ + Project: "shop", + Environment: "dev", + Namespace: "default", + TaskName: "migrate", + Task: spec.Task{Image: "busybox:1.36", Command: []string{"true"}}, + Profile: &spec.PlatformProfile{ + NodeSelector: map[string]string{"workload": "batch"}, + PodLabels: map[string]string{"tier": "jobs"}, + PodAnnotations: map[string]string{"team": "platform"}, + Tolerations: []corev1.Toleration{ + {Key: "batch", Operator: corev1.TolerationOpExists, Effect: corev1.TaintEffectNoSchedule}, + }, + SecurityContext: &corev1.PodSecurityContext{RunAsNonRoot: new(true)}, + ContainerSecurityContext: &corev1.SecurityContext{ReadOnlyRootFilesystem: new(true)}, + }, + }) + require.NoError(t, err) + assert.Equal(t, map[string]string{"workload": "batch"}, job.Spec.Template.Spec.NodeSelector) + assert.Equal(t, "jobs", job.Spec.Template.Labels["tier"]) + assert.Equal(t, "platform", job.Spec.Template.Annotations["team"]) + require.Len(t, job.Spec.Template.Spec.Tolerations, 1) + assert.Equal(t, "batch", job.Spec.Template.Spec.Tolerations[0].Key) + require.NotNil(t, job.Spec.Template.Spec.SecurityContext) + require.NotNil(t, job.Spec.Template.Spec.SecurityContext.RunAsNonRoot) + assert.True(t, *job.Spec.Template.Spec.SecurityContext.RunAsNonRoot) + require.NotNil(t, job.Spec.Template.Spec.Containers[0].SecurityContext) + require.NotNil(t, job.Spec.Template.Spec.Containers[0].SecurityContext.ReadOnlyRootFilesystem) + assert.True(t, *job.Spec.Template.Spec.Containers[0].SecurityContext.ReadOnlyRootFilesystem) +} diff --git a/internal/plan/build.go b/internal/plan/build.go index 7e9e590..c307324 100644 --- a/internal/plan/build.go +++ b/internal/plan/build.go @@ -17,6 +17,7 @@ package plan import ( "context" "fmt" + "slices" "helm.sh/helm/v4/pkg/postrenderer" @@ -85,6 +86,49 @@ func BuildPlan(ctx context.Context, client BuildClient, manifest *spec.Spec, env FreshInstall: prevRelease == nil, Warning: warning, } + p.Tasks, err = TasksFromSpec(manifest, environment, resolved) + if err != nil { + return nil, nil, cleanup, fmt.Errorf("tasks: %w", err) + } return p, result, cleanup, nil } + +// TasksFromSpec builds the plan Tasks section from the spec. When resolved +// is nil, only tasks that apply to environment are included. +func TasksFromSpec(manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) ([]PlannedTask, error) { + tasks, err := spec.EffectiveTasks(manifest, environment, resolved) + if err != nil { + return nil, err + } + names := make([]string, 0, len(tasks)) + for name := range tasks { + names = append(names, name) + } + slices.Sort(names) + out := make([]PlannedTask, 0, len(names)) + for _, name := range names { + rt := tasks[name] + out = append(out, PlannedTask{ + Name: name, + On: plannedTaskOn(rt.Task.On), + Timeout: rt.Task.Timeout, + HookWeight: rt.HookWeight, + Manual: rt.Task.On == spec.TaskOnManual, + }) + } + return out, nil +} + +func plannedTaskOn(on spec.TaskOn) string { + switch on { + case spec.TaskOnPreDeploy: + return TaskOnPreDeploy + case spec.TaskOnPostDeploy: + return TaskOnPostDeploy + case spec.TaskOnManual: + return TaskOnManual + default: + return string(on) + } +} diff --git a/internal/plan/format_json.go b/internal/plan/format_json.go index 04ff79d..728ff05 100644 --- a/internal/plan/format_json.go +++ b/internal/plan/format_json.go @@ -49,8 +49,19 @@ type JSONDocument struct { // Drift and DriftIncomplete are omitted when there is nothing to report // (either --drift wasn't requested, or it found nothing); the schema // does not distinguish those two cases. - Drift []JSONChange `json:"drift,omitempty"` - DriftIncomplete []string `json:"drift_incomplete,omitempty"` + Drift []JSONChange `json:"drift,omitempty"` + DriftIncomplete []string `json:"drift_incomplete,omitempty"` + Tasks []JSONTask `json:"tasks,omitempty"` + FirstInstallNote string `json:"first_install_note,omitempty"` +} + +// JSONTask is one entry in [JSONDocument.Tasks]. +type JSONTask struct { + Name string `json:"name"` + On string `json:"on"` + Timeout string `json:"timeout,omitempty"` + HookWeight int `json:"hook_weight"` + Manual bool `json:"manual,omitempty"` } // JSONChange is one entry in [JSONDocument.Changes]. @@ -119,6 +130,11 @@ func NewJSONDocument(p *Plan) *JSONDocument { } doc.DriftIncomplete = p.DriftIncomplete + for _, task := range p.Tasks { + doc.Tasks = append(doc.Tasks, JSONTask(task)) + } + doc.FirstInstallNote = p.FirstInstallTaskNote() + return doc } diff --git a/internal/plan/format_json_test.go b/internal/plan/format_json_test.go index 9680288..0646c17 100644 --- a/internal/plan/format_json_test.go +++ b/internal/plan/format_json_test.go @@ -288,3 +288,72 @@ func TestRenderJSON_DriftChangesAndIncomplete(t *testing.T) { require.True(t, ok) assert.Equal(t, []any{"ConfigMap/default/web-config"}, incomplete) } + +func TestRenderJSON_TasksSection(t *testing.T) { + t.Parallel() + + note := (&Plan{ + Header: Header{FreshInstall: true}, + Tasks: []PlannedTask{{On: TaskOnPreDeploy}}, + }).FirstInstallTaskNote() + tests := []struct { + name string + plan *Plan + want []JSONTask + wantNote string + wantRaw []string + }{ + { + name: "fresh install with preDeploy", + plan: &Plan{ + Header: Header{ + Project: "shop", + Environment: "dev", + Release: "shop-dev", + FreshInstall: true, + }, + Tasks: []PlannedTask{ + {Name: "migrate", On: TaskOnPreDeploy, Timeout: "5m", HookWeight: 0}, + {Name: "backfill", On: TaskOnManual, Manual: true}, + }, + }, + want: []JSONTask{ + {Name: "migrate", On: TaskOnPreDeploy, Timeout: "5m", HookWeight: 0}, + {Name: "backfill", On: TaskOnManual, Manual: true}, + }, + wantNote: note, + wantRaw: []string{`"hook_weight": 0`}, + }, + { + name: "upgrade omits first install note", + plan: &Plan{ + Header: Header{FreshInstall: false}, + Tasks: []PlannedTask{{Name: "migrate", On: TaskOnPreDeploy, HookWeight: 0}}, + }, + want: []JSONTask{{Name: "migrate", On: TaskOnPreDeploy, HookWeight: 0}}, + wantRaw: []string{`"hook_weight": 0`}, + }, + { + name: "fresh install without preDeploy omits note", + plan: &Plan{ + Header: Header{FreshInstall: true}, + Tasks: []PlannedTask{{Name: "backfill", On: TaskOnManual, Manual: true}}, + }, + want: []JSONTask{{Name: "backfill", On: TaskOnManual, Manual: true}}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + var buf strings.Builder + require.NoError(t, RenderJSON(&buf, tt.plan)) + var doc JSONDocument + require.NoError(t, json.Unmarshal([]byte(buf.String()), &doc)) + assert.Equal(t, tt.want, doc.Tasks) + assert.Equal(t, tt.wantNote, doc.FirstInstallNote) + for _, raw := range tt.wantRaw { + assert.Contains(t, buf.String(), raw) + } + }) + } +} diff --git a/internal/plan/format_text.go b/internal/plan/format_text.go index 2cb429a..351888c 100644 --- a/internal/plan/format_text.go +++ b/internal/plan/format_text.go @@ -18,6 +18,7 @@ import ( "fmt" "io" "regexp" + "slices" "strconv" "strings" @@ -117,6 +118,10 @@ func RenderText(w io.Writer, p *Plan, opts TextOptions) error { return err } + if err := writeTasks(w, p, opts); err != nil { + return err + } + if len(p.Changes) == 0 { if _, err := fmt.Fprintln(w, opts.Theme.Style(theme.StatusSuccess).Render("No changes.")); err != nil { return err @@ -371,6 +376,68 @@ func writeHookNote(w io.Writer, p *Plan, opts TextOptions) error { return err } +func writeTasks(w io.Writer, p *Plan, opts TextOptions) error { + if len(p.Tasks) == 0 { + return nil + } + + heading := opts.Theme.Style(theme.TextTitle).Render("Tasks:") + if _, err := fmt.Fprintln(w, "\n"+heading); err != nil { + return err + } + + groups := []struct { + title string + on string + }{ + {TaskOnPreDeploy, TaskOnPreDeploy}, + {TaskOnPostDeploy, TaskOnPostDeploy}, + {"manual (CLI only)", TaskOnManual}, + } + for _, g := range groups { + var items []PlannedTask + for _, task := range p.Tasks { + if task.On == g.on { + items = append(items, task) + } + } + if g.on != TaskOnManual { + slices.SortFunc(items, func(a, b PlannedTask) int { + if a.HookWeight != b.HookWeight { + return a.HookWeight - b.HookWeight + } + return strings.Compare(a.Name, b.Name) + }) + } + if len(items) == 0 { + continue + } + if _, err := fmt.Fprintf(w, " %s\n", g.title); err != nil { + return err + } + for _, task := range items { + line := " " + task.Name + if task.Timeout != "" { + line += " (timeout " + task.Timeout + ")" + } + if !task.Manual { + line += fmt.Sprintf(" weight %d", task.HookWeight) + } + if _, err := fmt.Fprintln(w, line); err != nil { + return err + } + } + } + + if note := p.FirstInstallTaskNote(); note != "" { + styled := opts.Theme.Style(theme.TextMuted).Render("Note: " + note) + if _, err := fmt.Fprintln(w, styled); err != nil { + return err + } + } + return nil +} + // String renders the summary trailer, e.g. // "1 to add, 1 to change, 1 to destroy". func (s Summary) String() string { diff --git a/internal/plan/format_text_test.go b/internal/plan/format_text_test.go index f97d302..5f195ff 100644 --- a/internal/plan/format_text_test.go +++ b/internal/plan/format_text_test.go @@ -20,6 +20,8 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + + "deployah.dev/deployah/internal/spec" ) // TestRenderText_HeaderAndMixedChanges covers the named case. @@ -340,7 +342,180 @@ func TestRenderText_DriftSection_FreshInstallIsNoOp(t *testing.T) { assert.NotContains(t, got, "No drift detected.") } -// TestMapCompactPath covers the named case. +func TestRenderText_TasksSection(t *testing.T) { + t.Parallel() + + grouped := []PlannedTask{ + {Name: "migrate", On: TaskOnPreDeploy, Timeout: "5m", HookWeight: 0}, + {Name: "seed", On: TaskOnPreDeploy, Timeout: "5m", HookWeight: 1}, + {Name: "smoke", On: TaskOnPostDeploy, Timeout: "5m", HookWeight: 0}, + {Name: "backfill", On: TaskOnManual, Manual: true}, + } + tests := []struct { + name string + plan *Plan + contains []string + omits []string + }{ + { + name: "groups by phase and notes first install", + plan: &Plan{ + Header: Header{ + Project: "shop", + Environment: "dev", + Release: "shop-dev", + FreshInstall: true, + }, + Tasks: grouped, + }, + contains: []string{ + "Tasks:", + "preDeploy", + "migrate (timeout 5m) weight 0", + "seed (timeout 5m) weight 1", + "postDeploy", + "manual (CLI only)", + "backfill", + "database must already be reachable", + }, + }, + { + name: "upgrade omits first install note", + plan: &Plan{ + Header: Header{FreshInstall: false}, + Tasks: grouped[:1], + }, + contains: []string{"Tasks:", "migrate (timeout 5m) weight 0"}, + omits: []string{"database must already be reachable"}, + }, + { + name: "postDeploy only skips empty groups", + plan: &Plan{ + Tasks: []PlannedTask{{Name: "smoke", On: TaskOnPostDeploy, Timeout: "5m"}}, + }, + contains: []string{"Tasks:", "postDeploy", "smoke (timeout 5m) weight 0"}, + omits: []string{"preDeploy", "manual"}, + }, + { + name: "task without timeout", + plan: &Plan{ + Tasks: []PlannedTask{{Name: "migrate", On: TaskOnPreDeploy}}, + }, + contains: []string{"migrate weight 0"}, + omits: []string{"timeout"}, + }, + { + name: "only manual group", + plan: &Plan{ + Tasks: []PlannedTask{{Name: "backfill", On: TaskOnManual, Manual: true}}, + }, + contains: []string{"Tasks:", "manual (CLI only)", "backfill"}, + omits: []string{"preDeploy", "weight"}, + }, + { + name: "no tasks omits section", + plan: &Plan{Header: Header{FreshInstall: true}}, + omits: []string{"Tasks:"}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + var buf strings.Builder + require.NoError(t, RenderText(&buf, tt.plan, TextOptions{})) + got := buf.String() + for _, s := range tt.contains { + assert.Contains(t, got, s) + } + for _, s := range tt.omits { + assert.NotContains(t, got, s) + } + }) + } +} + +func TestPlan_FirstInstallTaskNote(t *testing.T) { + t.Parallel() + + const note = "preDeploy runs before other resources on a first install; the database must already be reachable." + tests := []struct { + name string + plan *Plan + want string + }{ + {name: "nil plan", plan: nil, want: ""}, + {name: "not fresh install", plan: &Plan{Tasks: []PlannedTask{{On: TaskOnPreDeploy}}}, want: ""}, + {name: "fresh install without preDeploy", plan: &Plan{Header: Header{FreshInstall: true}, Tasks: []PlannedTask{{On: TaskOnManual}}}, want: ""}, + {name: "fresh install with preDeploy", plan: &Plan{Header: Header{FreshInstall: true}, Tasks: []PlannedTask{{On: TaskOnPreDeploy}}}, want: note}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + assert.Equal(t, tt.want, tt.plan.FirstInstallTaskNote()) + }) + } +} + +func TestTasksFromSpec(t *testing.T) { + t.Parallel() + + m := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest", Environments: []string{"prod"}}, + }, + Tasks: map[string]spec.Task{ + "migrate": {From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + "smoke": {From: "api", On: spec.TaskOnPostDeploy, Environments: []string{"dev", "prod"}, Command: []string{"true"}}, + "nightly": {From: "api", On: spec.TaskOnManual, Environments: []string{"prod"}, Command: []string{"true"}}, + }, + } + tests := []struct { + name string + environment string + want []PlannedTask + }{ + { + name: "dev skips inherited prod-only parent", + environment: "dev", + want: []PlannedTask{ + {Name: "smoke", On: TaskOnPostDeploy}, + }, + }, + { + name: "prod includes inherited and explicit", + environment: "prod", + want: []PlannedTask{ + {Name: "migrate", On: TaskOnPreDeploy}, + {Name: "nightly", On: TaskOnManual, Manual: true}, + {Name: "smoke", On: TaskOnPostDeploy}, + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got, err := TasksFromSpec(m, tt.environment, nil) + require.NoError(t, err) + assert.Equal(t, tt.want, got) + }) + } +} + +func TestTasksFromSpec_Cycle(t *testing.T) { + t.Parallel() + m := &spec.Spec{ + Project: "shop", + Tasks: map[string]spec.Task{ + "a": {On: spec.TaskOnPreDeploy, After: []string{"b"}, Command: []string{"true"}}, + "b": {On: spec.TaskOnPreDeploy, After: []string{"a"}, Command: []string{"true"}}, + }, + } + _, err := TasksFromSpec(m, "dev", nil) + require.Error(t, err) + assert.Contains(t, err.Error(), "cycle") +} + func TestMapCompactPath(t *testing.T) { t.Parallel() tests := []struct { @@ -355,8 +530,11 @@ func TestMapCompactPath(t *testing.T) { {"metadata.labels.foo", "", false}, } for _, tt := range tests { - display, ok := mapCompactPath(tt.path) - assert.Equal(t, tt.ok, ok, "path %s", tt.path) - assert.Equal(t, tt.display, display, "path %s", tt.path) + t.Run(tt.path, func(t *testing.T) { + t.Parallel() + display, ok := mapCompactPath(tt.path) + assert.Equal(t, tt.ok, ok) + assert.Equal(t, tt.display, display) + }) } } diff --git a/internal/plan/types.go b/internal/plan/types.go index 2afbd6d..6f863f0 100644 --- a/internal/plan/types.go +++ b/internal/plan/types.go @@ -159,6 +159,44 @@ type Plan struct { // (e.g. missing RBAC), so the plan can say it is incomplete instead of // silently omitting them. DriftIncomplete []string + + // Tasks lists spec tasks active in this environment, grouped by the + // renderer into preDeploy, postDeploy, and manual. + Tasks []PlannedTask +} + +const ( + // TaskOnPreDeploy is a hook that runs before other resources. + TaskOnPreDeploy = "preDeploy" + // TaskOnPostDeploy is a hook that runs after the app is ready. + TaskOnPostDeploy = "postDeploy" + // TaskOnManual is a task that runs only via the CLI. + TaskOnManual = "manual" +) + +// PlannedTask is one spec task shown in the plan Tasks section. +type PlannedTask struct { + Name string + On string + Timeout string + HookWeight int + Manual bool +} + +// FirstInstallTaskNote returns a warning when this plan is a fresh install +// that includes a preDeploy task. Helm runs those hooks before other +// resources, so the database they talk to must already exist. It returns +// "" otherwise. +func (p *Plan) FirstInstallTaskNote() string { + if p == nil || !p.Header.FreshInstall { + return "" + } + for _, task := range p.Tasks { + if task.On == TaskOnPreDeploy { + return "preDeploy runs before other resources on a first install; the database must already be reachable." + } + } + return "" } // HasChanges reports whether applying this plan would change the cluster: diff --git a/internal/session/constants.go b/internal/session/constants.go index a71c887..316fd38 100644 --- a/internal/session/constants.go +++ b/internal/session/constants.go @@ -14,12 +14,17 @@ package session -import "time" +import ( + "time" + + "deployah.dev/deployah/internal/spec" +) // Session defaults. const ( // DefaultTimeout is the default timeout for Helm operations. - DefaultTimeout = 10 * time.Minute + // It is [spec.DefaultDeployTimeout]. + DefaultTimeout = spec.DefaultDeployTimeout // DefaultStorageDriver is the default Helm storage driver. DefaultStorageDriver = "secret" diff --git a/internal/session/session_test.go b/internal/session/session_test.go index 85dc31e..5011b81 100644 --- a/internal/session/session_test.go +++ b/internal/session/session_test.go @@ -61,7 +61,7 @@ users: // minimalSpecYAML is a self-contained spec fixture with a valid apiVersion // and a single component, reused by Spec/ParseManifest tests. -const minimalSpecYAML = `apiVersion: v1-alpha.4 +const minimalSpecYAML = `apiVersion: v1-alpha.5 project: demo components: web: diff --git a/internal/spec/constants.go b/internal/spec/constants.go index 8f7bc4f..2db9601 100644 --- a/internal/spec/constants.go +++ b/internal/spec/constants.go @@ -14,12 +14,14 @@ package spec +import "time" + // File and Path Constants const ( // CurrentManifestVersion is the manifest apiVersion written by the init - // command and expected by the current resolver. Bump this when a new - // schema version is added alongside a new schema directory. - CurrentManifestVersion = "v1-alpha.4" + // command. During alpha, only this version has an embedded schema. Bump + // it and replace the schema directory when the spec changes. + CurrentManifestVersion = "v1-alpha.5" // DefaultSpecPath is the default path for the Deployah spec file DefaultSpecPath = "deployah.yaml" @@ -43,6 +45,10 @@ const ( ConfigFileSuffix = ".yaml" ) +// SupportedManifestVersions is the apiVersion values this release will load. +// During alpha this is only [CurrentManifestVersion]. +var SupportedManifestVersions = []string{CurrentManifestVersion} + // Environment Variables const ( // EnvVarPrefix is the prefix for Deployah-specific environment variables @@ -81,6 +87,9 @@ const ( // ComponentsPrefix is the prefix for component paths in schemas ComponentsPrefix = "components." + // TasksPrefix is the prefix for task paths in schemas + TasksPrefix = "tasks." + // EnvironmentsPrefix is the prefix for environment paths in schemas EnvironmentsPrefix = "environments." @@ -152,6 +161,32 @@ const ( // worker components (terminationGracePeriodSeconds). DefaultWorkerShutdownTimeout = "60s" + // DefaultHookTaskTimeout is the default timeout for preDeploy and + // postDeploy tasks when timeout is omitted. + DefaultHookTaskTimeout = "5m" + + // DefaultDeployTimeout is the default CLI --timeout. Hook task + // timeouts must be strictly less than the session --timeout at + // deploy or run time. + DefaultDeployTimeout = 10 * time.Minute + + // DefaultBackoffLimit is the default Job retry count for tasks. + DefaultBackoffLimit = 3 + + // DefaultFanoutCount is the default fanout count when omitted. + DefaultFanoutCount = 1 + + // DefaultFanoutParallelism is the default fanout parallelism when omitted. + DefaultFanoutParallelism = 1 + + // MaxFanoutParallelism is the largest allowed fanout.parallelism. + // Kubernetes rejects Indexed Jobs when parallelism is above 10^5. + MaxFanoutParallelism = 100_000 + + // DefaultCLIJobTTLSeconds is how long CLI-triggered Jobs are kept + // after they finish (7 days). + DefaultCLIJobTTLSeconds = 7 * 24 * 60 * 60 + // DefaultMetricsPath is the default HTTP path for Prometheus metrics. DefaultMetricsPath = "/metrics" diff --git a/internal/spec/defaults.go b/internal/spec/defaults.go index 056e629..bcda14d 100644 --- a/internal/spec/defaults.go +++ b/internal/spec/defaults.go @@ -66,14 +66,14 @@ const componentsPrefixLength = len(ComponentsPrefix) // and pattern extraction operations. // // Cache keys follow the format: "{version}-{schemaType}" -// Example: "v1-alpha.4-spec", "v1-alpha.4-environments" +// Example: "v1-alpha.5-spec", "v1-alpha.5-environments" var ( // compiledSchemaCache stores compiled JSON schemas with their raw data - // Key format: "v1-alpha.4-spec" -> schemaInfo{compiled, rawData} + // Key format: "v1-alpha.5-spec" -> schemaInfo{compiled, rawData} compiledSchemaCache = make(map[string]*schemaInfo) // patternCache stores extracted component name patterns from schemas - // Key format: "v1-alpha.4" -> "^[a-zA-Z0-9_-]+$" + // Key format: "v1-alpha.5" -> "^[a-zA-Z0-9_-]+$" patternCache = make(map[string]string) // schemaMutex protects concurrent access to the caches @@ -270,7 +270,7 @@ func (w *defaultsWalker) walk(schemaData any, path string, defaults DefaultValue } // Handle map-typed additionalProperties combined with a propertyNames - // pattern (the v1-alpha.4 layout for components/environments); the + // pattern (the object-map layout for components/environments); the // pattern plays the same role as a patternProperties key. if addProps, exists := schemaMap["additionalProperties"].(map[string]any); exists { pattern := ".*" @@ -321,47 +321,51 @@ func GetDefaultValues(version string, schemaType schema.SchemaType) (DefaultValu // concrete Resources, then clears ResourcePreset so it does not conflict // with the now-populated Resources during validation. Components that // already set Resources are left untouched; components with neither -// Resources nor a preset get [ResourcePresetSmall]. Only the preset's +// Resources nor a preset get [ResourcePresetSmall]. Tasks follow the same +// rules, except a task with from and no own resources or preset is left +// empty so [Task.MergeFrom] can copy the parent. Only the preset's // "requests" values are applied; "limits" are not used. -func resolveResourcePresets(spec *Spec) error { +func resolveResourcePresets(spec *Spec) { for componentName, component := range spec.Components { - // Only resolve if ResourcePreset is set and Resources are empty - if component.ResourcePreset != "" && !component.Resources.ResourcesSet() { - if presetResources, exists := ResourcePresetMappings[component.ResourcePreset]; exists { - // Clone quantities so callers do not share mutable pointers from - // the package-level ResourcePresetMappings table. - req := presetResources["requests"] - component.Resources = Resources{ - CPU: cloneQuantity(req.CPU), - Memory: cloneQuantity(req.Memory), - EphemeralStorage: cloneQuantity(req.EphemeralStorage), - } - // Clear the resourcePreset field after converting to resources - // to avoid validation conflicts - component.ResourcePreset = "" - spec.Components[componentName] = component - continue - } + applyResourcePreset(&component.ResourcePreset, &component.Resources) + spec.Components[componentName] = component + } + for taskName, task := range spec.Tasks { + // A task with from and nothing of its own inherits the parent's + // resources in [Task.MergeFrom], so leave it empty here. + if task.From != "" && task.ResourcePreset == "" && !task.Resources.ResourcesSet() { + continue } + applyResourcePreset(&task.ResourcePreset, &task.Resources) + spec.Tasks[taskName] = task + } +} - // If neither explicit resources nor a preset is provided, apply a default preset at spec layer - if component.ResourcePreset == "" && !component.Resources.ResourcesSet() { - if presetResources, exists := ResourcePresetMappings[ResourcePresetSmall]; exists { - component.ResourcePreset = ResourcePresetSmall - req := presetResources["requests"] - component.Resources = Resources{ - CPU: cloneQuantity(req.CPU), - Memory: cloneQuantity(req.Memory), - EphemeralStorage: cloneQuantity(req.EphemeralStorage), - } - // Clear the resourcePreset field after converting to resources - // to avoid validation conflicts - component.ResourcePreset = "" - spec.Components[componentName] = component - } - } +// applyResourcePreset fills resources from preset when resources are empty, +// then clears preset so validation does not see both. An empty preset falls +// back to [ResourcePresetSmall]. Explicit resources win, and an unknown +// preset is left in place for validation to report. +func applyResourcePreset(preset *ResourcePreset, resources *Resources) { + if resources.ResourcesSet() { + return } - return nil + name := *preset + if name == "" { + name = ResourcePresetSmall + } + presetResources, exists := ResourcePresetMappings[name] + if !exists { + return + } + // Clone quantities so callers do not share mutable pointers from the + // package-level ResourcePresetMappings table. + req := presetResources["requests"] + *resources = Resources{ + CPU: cloneQuantity(req.CPU), + Memory: cloneQuantity(req.Memory), + EphemeralStorage: cloneQuantity(req.EphemeralStorage), + } + *preset = "" } // FillSpecWithDefaults fills spec with defaults from the JSON schemas for @@ -402,10 +406,19 @@ func FillSpecWithDefaults(spec *Spec, version string) error { spec.Components[componentName] = component } - // Resolve resource presets after applying schema defaults - if err = resolveResourcePresets(spec); err != nil { - return fmt.Errorf("failed to resolve resource presets: %w", err) + if spec.Tasks == nil { + spec.Tasks = make(map[string]Task) } + for taskName, task := range spec.Tasks { + if err = applyDefaultsRecursively(&task, specDefaults, "tasks."+taskName, version); err != nil { + return fmt.Errorf("failed to apply defaults to task %s: %w", taskName, err) + } + applyTaskDefaults(&task) + spec.Tasks[taskName] = task + } + + // Resolve resource presets after applying schema defaults + resolveResourcePresets(spec) // Merge spec and environment defaults for environments mergedDefaults := make(DefaultValues) @@ -870,6 +883,22 @@ func applyRoleDependentDefaults(c *Component) { } } +func applyTaskDefaults(t *Task) { + if t.Fanout.Count <= 0 { + t.Fanout.Count = DefaultFanoutCount + } + if t.Fanout.Parallelism <= 0 { + t.Fanout.Parallelism = DefaultFanoutParallelism + } + if t.BackoffLimit == nil { + n := DefaultBackoffLimit + t.BackoffLimit = &n + } + if t.On.IsHook() && t.Timeout == "" { + t.Timeout = DefaultHookTaskTimeout + } +} + // CreateSpecWithDefaults creates a minimal [Spec] for projectName and fills // it with the defaults declared by version's schema. func CreateSpecWithDefaults(projectName, version string) (*Spec, error) { diff --git a/internal/spec/defaults_test.go b/internal/spec/defaults_test.go index f89f6ea..f10a566 100644 --- a/internal/spec/defaults_test.go +++ b/internal/spec/defaults_test.go @@ -137,13 +137,13 @@ func TestGetDefaultValues(t *testing.T) { }{ { name: "valid manifest schema", - version: "v1-alpha.4", + version: CurrentManifestVersion, schemaType: schema.SchemaTypeManifest, expectErr: false, }, { name: "valid environments schema", - version: "v1-alpha.4", + version: CurrentManifestVersion, schemaType: schema.SchemaTypeEnvironments, expectErr: false, }, @@ -155,7 +155,7 @@ func TestGetDefaultValues(t *testing.T) { }, { name: "unsupported schema type", - version: "v1-alpha.4", + version: CurrentManifestVersion, schemaType: "unsupported", expectErr: true, }, @@ -172,8 +172,8 @@ func TestGetDefaultValues(t *testing.T) { } else { assert.NoError(t, err) assert.NotNil(t, defaults) - // The environments schema declares no defaults in - // v1-alpha.4; only the manifest schema must be non-empty. + // The environments schema declares no defaults; + // only the manifest schema must be non-empty. if tt.schemaType == schema.SchemaTypeManifest { assert.NotEmpty(t, defaults) } @@ -195,7 +195,7 @@ func TestFillSpecWithDefaults(t *testing.T) { { name: "valid manifest with components", manifest: &Spec{ - APIVersion: "v1-alpha.4", + APIVersion: CurrentManifestVersion, Project: "test-project", Components: map[string]Component{ "web": { @@ -203,36 +203,36 @@ func TestFillSpecWithDefaults(t *testing.T) { }, }, }, - version: "v1-alpha.4", + version: CurrentManifestVersion, expectErr: false, }, { name: "manifest with nil components", manifest: &Spec{ - APIVersion: "v1-alpha.4", + APIVersion: CurrentManifestVersion, Project: "test-project", Components: nil, }, - version: "v1-alpha.4", + version: CurrentManifestVersion, expectErr: false, }, { name: "manifest with environments", manifest: &Spec{ - APIVersion: "v1-alpha.4", + APIVersion: CurrentManifestVersion, Project: "test-project", Components: map[string]Component{}, Environments: map[string]Environment{ "production": {}, }, }, - version: "v1-alpha.4", + version: CurrentManifestVersion, expectErr: false, }, { name: "invalid version", manifest: &Spec{ - APIVersion: "v1-alpha.4", + APIVersion: CurrentManifestVersion, Project: "test-project", Components: map[string]Component{}, }, @@ -384,7 +384,7 @@ func TestApplyDefaultsRecursively(t *testing.T) { t.Run(tt.name, func(t *testing.T) { t.Parallel() - require.NoError(t, applyDefaultsRecursively(tt.obj, tt.defaults, tt.path, "v1-alpha.4")) + require.NoError(t, applyDefaultsRecursively(tt.obj, tt.defaults, tt.path, CurrentManifestVersion)) assert.Equal(t, tt.expected, tt.obj) }) } @@ -426,7 +426,7 @@ func TestApplyDefaultsToMap(t *testing.T) { t.Parallel() // This test mainly ensures the function doesn't panic - require.NoError(t, applyDefaultsToMap(tt.mapVal, tt.defaults, tt.path, "v1-alpha.4")) + require.NoError(t, applyDefaultsToMap(tt.mapVal, tt.defaults, tt.path, CurrentManifestVersion)) // No specific assertions as this is mainly testing for panics }) } @@ -465,7 +465,7 @@ func TestApplyDefaultsToSlice(t *testing.T) { t.Parallel() // This test mainly ensures the function doesn't panic - require.NoError(t, applyDefaultsToSlice(tt.sliceVal, tt.defaults, tt.path, "v1-alpha.4")) + require.NoError(t, applyDefaultsToSlice(tt.sliceVal, tt.defaults, tt.path, CurrentManifestVersion)) // No specific assertions as this is mainly testing for panics }) } @@ -659,7 +659,7 @@ func TestCreateSpecWithDefaults(t *testing.T) { { name: "valid manifest creation", projectName: "test-project", - version: "v1-alpha.4", + version: CurrentManifestVersion, expectErr: false, }, { @@ -787,7 +787,7 @@ func TestIntegration(t *testing.T) { t.Run("create manifest with defaults and verify component defaults", func(t *testing.T) { t.Parallel() - manifest, err := CreateSpecWithDefaults("test-project", "v1-alpha.4") + manifest, err := CreateSpecWithDefaults("test-project", CurrentManifestVersion) assert.NoError(t, err) assert.NotNil(t, manifest) @@ -796,7 +796,7 @@ func TestIntegration(t *testing.T) { Image: "nginx:latest", } - err = FillSpecWithDefaults(manifest, "v1-alpha.4") + err = FillSpecWithDefaults(manifest, CurrentManifestVersion) assert.NoError(t, err) webComponent := manifest.Components["web"] @@ -810,7 +810,7 @@ func TestIntegration(t *testing.T) { t.Parallel() manifest := &Spec{ - APIVersion: "v1-alpha.4", + APIVersion: CurrentManifestVersion, Project: "test-project", Components: map[string]Component{ "api": { @@ -822,7 +822,7 @@ func TestIntegration(t *testing.T) { }, } - err := FillSpecWithDefaults(manifest, "v1-alpha.4") + err := FillSpecWithDefaults(manifest, CurrentManifestVersion) assert.NoError(t, err) apiComponent := manifest.Components["api"] @@ -838,7 +838,7 @@ func TestIntegration(t *testing.T) { t.Parallel() manifest := &Spec{ - APIVersion: "v1-alpha.4", + APIVersion: CurrentManifestVersion, Project: "test-project", Components: map[string]Component{}, Environments: map[string]Environment{ @@ -846,11 +846,11 @@ func TestIntegration(t *testing.T) { }, } - err := FillSpecWithDefaults(manifest, "v1-alpha.4") + err := FillSpecWithDefaults(manifest, CurrentManifestVersion) assert.NoError(t, err) - // v1-alpha.4 declares no envFile/configFile defaults: the loader's - // convention-based lookup replaced them. + // The environments schema declares no envFile/configFile defaults; + // the loader's convention-based lookup replaced them. assert.Empty(t, manifest.Environments["production"].EnvFile) assert.Empty(t, manifest.Environments["production"].ConfigFile) }) @@ -999,7 +999,7 @@ func TestFillSpecWithDefaults_GuardClauses(t *testing.T) { version string errContains string }{ - {name: "nil spec returns error", spec: nil, version: "v1-alpha.4", errContains: "spec cannot be nil"}, + {name: "nil spec returns error", spec: nil, version: CurrentManifestVersion, errContains: "spec cannot be nil"}, {name: "empty version returns error", spec: &Spec{Project: "test"}, version: "", errContains: "version cannot be empty"}, } @@ -1074,7 +1074,7 @@ func TestApplyDefaultsToMap_EdgeCases(t *testing.T) { t.Run(tt.name, func(t *testing.T) { t.Parallel() - err := applyDefaultsToMap(tt.value, tt.defaults, tt.path, "v1-alpha.4") + err := applyDefaultsToMap(tt.value, tt.defaults, tt.path, CurrentManifestVersion) require.NoError(t, err) if tt.check != nil { tt.check(t, tt.value) @@ -1147,7 +1147,7 @@ func TestApplyDefaultsToSlice_EdgeCases(t *testing.T) { t.Run(tt.name, func(t *testing.T) { t.Parallel() - err := applyDefaultsToSlice(tt.value, tt.defaults, tt.path, "v1-alpha.4") + err := applyDefaultsToSlice(tt.value, tt.defaults, tt.path, CurrentManifestVersion) require.NoError(t, err) if tt.check != nil { tt.check(t, tt.value) @@ -1363,7 +1363,7 @@ func TestProcessStructField(t *testing.T) { t.Parallel() field, fieldType, verify := tt.setup() - err := processStructField(field, fieldType, tt.defaults, tt.path, "v1-alpha.4") + err := processStructField(field, fieldType, tt.defaults, tt.path, CurrentManifestVersion) require.NoError(t, err) verify(t) }) diff --git a/internal/spec/doc.go b/internal/spec/doc.go index e48f522..bac3767 100644 --- a/internal/spec/doc.go +++ b/internal/spec/doc.go @@ -28,6 +28,12 @@ // - [ValidateSpec]: validate spec data against a schema version // - [ValidateEnvironments]: validate environment definitions // - [ValidateSpecComponents]: check component resources and autoscaling +// - [ValidateSpecTasks]: check task names, from, on, after, and fanout +// +// # Tasks +// +// - [EffectiveTasks]: environment-scoped tasks from a spec or resolved result +// - [NewTaskJobSpec]: shared Job fields for the CLI builder and Helm values // // # Defaults // diff --git a/internal/spec/example_test.go b/internal/spec/example_test.go index 5f326fb..db0f82b 100644 --- a/internal/spec/example_test.go +++ b/internal/spec/example_test.go @@ -26,13 +26,13 @@ import ( // ExampleFillSpecWithDefaults applies schema defaults to a minimal manifest. func ExampleFillSpecWithDefaults() { m := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: "v1-alpha.5", Project: "demo", Components: map[string]spec.Component{ "web": {Image: "nginx:latest"}, }, } - if err := spec.FillSpecWithDefaults(m, "v1-alpha.4"); err != nil { + if err := spec.FillSpecWithDefaults(m, "v1-alpha.5"); err != nil { log.Fatal(err) } fmt.Println(m.Components["web"].Port) @@ -41,7 +41,7 @@ func ExampleFillSpecWithDefaults() { // ExampleLoad reads a manifest file from disk. func ExampleLoad() { - const yamlDoc = `apiVersion: v1-alpha.4 + const yamlDoc = `apiVersion: v1-alpha.5 project: demo environments: default: {} diff --git a/internal/spec/field_validation.go b/internal/spec/field_validation.go index 453c390..b1cbbb8 100644 --- a/internal/spec/field_validation.go +++ b/internal/spec/field_validation.go @@ -64,7 +64,7 @@ func initValidators() error { return fmt.Errorf("failed to extract component name pattern: %w", err) } - // Extract environment name pattern. v1-alpha.4 models "environments" as + // Extract environment name pattern. The schema models "environments" as // an object keyed by environment name, so the pattern lives on // propertyNames rather than on an array item's "name" field. envPattern, err := extractPattern(schemaData, []string{"properties", "environments", "propertyNames", "pattern"}) diff --git a/internal/spec/field_validation_test.go b/internal/spec/field_validation_test.go index 2eae0ac..711f12f 100644 --- a/internal/spec/field_validation_test.go +++ b/internal/spec/field_validation_test.go @@ -111,7 +111,7 @@ func TestValidateComponentName(t *testing.T) { } } -// TestValidateEnvName verifies ValidateEnvName rules against the v1-alpha.4 +// TestValidateEnvName verifies ValidateEnvName rules against the // object-shaped "environments" schema. Top-level environment keys never // carry a "/*" wildcard suffix; that syntax is only valid in a component's // "environments" filter list, which is a plain string array with no pattern diff --git a/internal/spec/loader.go b/internal/spec/loader.go index 15bcdb5..7bfd1ea 100644 --- a/internal/spec/loader.go +++ b/internal/spec/loader.go @@ -295,6 +295,10 @@ func Load(ctx context.Context, path, desiredEnv string, platform *PlatformConfig return nil, fmt.Errorf("validation failed: %w", err) } + if err = ValidateSpecTasks(&finalSpec); err != nil { + return nil, fmt.Errorf("validation failed: %w", err) + } + if err = FillSpecWithDefaults(&finalSpec, version); err != nil { return nil, fmt.Errorf("failed to apply defaults: %w", err) } diff --git a/internal/spec/loader_test.go b/internal/spec/loader_test.go index 87fa89f..e8b74c7 100644 --- a/internal/spec/loader_test.go +++ b/internal/spec/loader_test.go @@ -144,7 +144,7 @@ func TestLoad_NoEnvironmentsSection(t *testing.T) { dir := t.TempDir() t.Chdir(dir) path := filepath.Join(dir, "deployah.yaml") - doc := `apiVersion: v1-alpha.4 + doc := `apiVersion: v1-alpha.5 project: demo components: web: @@ -394,7 +394,7 @@ func TestParseManifest_ProfilesArray(t *testing.T) { dir := t.TempDir() path := filepath.Join(dir, "deployah.yaml") content := ` -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: web: @@ -416,7 +416,7 @@ func TestLoad_OldProfileStringRejected(t *testing.T) { dir := t.TempDir() t.Chdir(dir) content := ` -apiVersion: v1-alpha.4 +apiVersion: v1-alpha.5 project: shop components: web: diff --git a/internal/spec/platform_test.go b/internal/spec/platform_test.go index 8e8d803..3d61743 100644 --- a/internal/spec/platform_test.go +++ b/internal/spec/platform_test.go @@ -209,7 +209,7 @@ func minimalPlatform() *spec.PlatformConfig { func minimalSpec(subdomain *string) *spec.Spec { return &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -245,7 +245,7 @@ func TestResolve_FQDN(t *testing.T) { // the platform registry warn, while prefix-style entries stay warning-free. func TestResolve_UnknownEnvironmentNameWarnings(t *testing.T) { appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -382,7 +382,7 @@ func TestResolve_DomainGapError(t *testing.T) { func TestResolve_FQDNCollision(t *testing.T) { // Two components resolving to the same FQDN (apex on same domain). appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -417,7 +417,7 @@ func TestResolve_WildcardStaticSubdomainWarning(t *testing.T) { }, } appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "review": {}, @@ -456,7 +456,7 @@ func TestResolve_WildcardDynamicSubdomainNoWarning(t *testing.T) { }, } appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "review": {}, @@ -726,7 +726,7 @@ func TestResolve_ErrorCode_DomainGap(t *testing.T) { func TestResolve_ErrorCode_InvalidDNS(t *testing.T) { // Subdomain with invalid characters (not dynamic). appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -746,7 +746,7 @@ func TestResolve_ErrorCode_InvalidDNS(t *testing.T) { // TestResolve_ErrorCode_FQDNCollision verifies platform spec behavior. func TestResolve_ErrorCode_FQDNCollision(t *testing.T) { appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "production": {}, @@ -769,7 +769,7 @@ func TestResolve_DynamicSubdomainSkipsDNSValidation(t *testing.T) { // Subdomain contains ${PR_NUMBER} which is not a valid DNS label, but // the prescan marks it as dynamic so resolution should succeed. appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "review": {}, @@ -800,7 +800,7 @@ func TestResolve_DynamicSubdomainSkipsDNSValidation(t *testing.T) { func TestResolve_StaticInvalidSubdomainFailsDNS(t *testing.T) { // Same invalid subdomain but NOT marked as dynamic: should fail. appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{ "review": {}, @@ -1032,7 +1032,7 @@ func TestResolve_Profiles(t *testing.T) { { name: "domain ignored without expose", appSpec: &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{"production": {}}, Components: map[string]spec.Component{ @@ -1049,7 +1049,7 @@ func TestResolve_Profiles(t *testing.T) { { name: "storage class missing in environment", appSpec: &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{"local": {}}, Components: map[string]spec.Component{ @@ -1063,7 +1063,7 @@ func TestResolve_Profiles(t *testing.T) { { name: "storage class resolved to className", appSpec: &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{"production": {}}, Components: map[string]spec.Component{ @@ -1204,7 +1204,7 @@ func TestResolve_ComponentStorageClass(t *testing.T) { t.Run("component key wins over profile", func(t *testing.T) { t.Parallel() appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{"production": {}}, Components: map[string]spec.Component{ @@ -1234,7 +1234,7 @@ func TestResolve_ComponentStorageClass(t *testing.T) { t.Run("unknown component key errors", func(t *testing.T) { t.Parallel() appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{"production": {}}, Components: map[string]spec.Component{ @@ -1257,7 +1257,7 @@ func TestResolve_ComponentStorageClass(t *testing.T) { t.Parallel() p := minimalPlatform() appSpec := &spec.Spec{ - APIVersion: "v1-alpha.4", + APIVersion: spec.CurrentManifestVersion, Project: "shop", Environments: map[string]spec.Environment{"local": {}}, Components: map[string]spec.Component{ @@ -1296,6 +1296,7 @@ func TestResolveForDisplay(t *testing.T) { check: func(t *testing.T, appSpec *spec.Spec, resolved *spec.ResolvedSpec, _ *spec.ResolutionReport) { t.Helper() assert.Empty(t, resolved.Components) + assert.NotNil(t, resolved.Tasks, "partial result must carry the same maps as Resolve") assert.Equal(t, appSpec, resolved.Spec) }, }, @@ -1341,3 +1342,210 @@ func TestCrossCheckPlatformReferences_UnknownProfile(t *testing.T) { require.Len(t, problems, 1) assert.Contains(t, problems[0], `"missing"`) } + +func TestCrossCheckPlatformReferences_UnknownTaskProfile(t *testing.T) { + t.Parallel() + appSpec := &spec.Spec{ + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "migrate": {From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}, Profiles: []string{"missing"}}, + }, + } + platform := platformWithProfiles() + problems, _ := spec.CrossCheckPlatformReferences(appSpec, platform) + require.Len(t, problems, 1) + assert.Contains(t, problems[0], `task "migrate"`) + assert.Contains(t, problems[0], `"missing"`) +} + +// TestResolve_TasksScopedToEnvironment locks in the contract [spec.EffectiveTasks] +// relies on: a resolved spec already holds only the tasks for its environment. +func TestResolve_TasksScopedToEnvironment(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + component spec.Component + task spec.Task + environment string + wantTask bool + }{ + { + name: "no filter runs everywhere", + component: spec.Component{Image: "nginx:latest"}, + task: spec.Task{From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + environment: "local", + wantTask: true, + }, + { + name: "own filter excludes other environment", + component: spec.Component{Image: "nginx:latest"}, + task: spec.Task{From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}, Environments: []string{"production"}}, + environment: "local", + wantTask: false, + }, + { + name: "own filter includes matching environment", + component: spec.Component{Image: "nginx:latest"}, + task: spec.Task{From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}, Environments: []string{"production"}}, + environment: "production", + wantTask: true, + }, + { + name: "filter inherited from parent component excludes", + component: spec.Component{Image: "nginx:latest", Environments: []string{"production"}}, + task: spec.Task{From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + environment: "local", + wantTask: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + appSpec := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{"api": tt.component}, + Tasks: map[string]spec.Task{"migrate": tt.task}, + } + resolved, _, err := spec.Resolve( + appSpec, platformWithProfiles(), spec.NormalizeEnv(tt.environment), spec.SubstitutionReport{}, + ) + require.NoError(t, err) + _, ok := resolved.Tasks["migrate"] + assert.Equal(t, tt.wantTask, ok) + }) + } +} + +func TestResolve_TaskAppliesDefaultProfile(t *testing.T) { + t.Parallel() + appSpec := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "migrate": {From: "api", On: spec.TaskOnPreDeploy, Command: []string{"true"}}, + }, + } + platform := platformWithProfiles() + resolved, _, err := spec.Resolve(appSpec, platform, spec.NormalizeEnv("production"), spec.SubstitutionReport{}) + require.NoError(t, err) + rt, ok := resolved.Tasks["migrate"] + require.True(t, ok) + require.NotNil(t, rt.MergedProfile) + assert.Equal(t, []string{"default"}, rt.Profiles) + assert.Equal(t, "general", rt.MergedProfile.NodeSelector["workload"]) +} + +func TestResolve_UnknownTaskEnvironmentFilterWarning(t *testing.T) { + t.Parallel() + + appSpec := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "migrate": { + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"true"}, + Environments: []string{"stagign"}, + }, + }, + } + _, report, err := spec.Resolve(appSpec, minimalPlatform(), spec.NormalizeEnv("production"), spec.SubstitutionReport{}) + require.NoError(t, err) + require.NotEmpty(t, report.Warnings) + assert.Contains(t, report.Warnings[0], `task "migrate"`) + assert.Contains(t, report.Warnings[0], `"stagign"`) +} + +func TestResolve_TaskProfilesRequirePlatform(t *testing.T) { + t.Parallel() + + appSpec := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "migrate": { + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"true"}, + Profiles: []string{"batch"}, + }, + }, + } + _, _, err := spec.Resolve(appSpec, nil, spec.NormalizeEnv("dev"), spec.SubstitutionReport{}) + require.Error(t, err) + assert.Contains(t, err.Error(), "no platform file") +} + +func TestResolve_UnknownTaskProfile(t *testing.T) { + t.Parallel() + + appSpec := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "migrate": { + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"true"}, + Profiles: []string{"missing"}, + }, + }, + } + _, _, err := spec.Resolve(appSpec, platformWithProfiles(), spec.NormalizeEnv("production"), spec.SubstitutionReport{}) + require.Error(t, err) + assert.Contains(t, err.Error(), `task "migrate"`) + assert.Contains(t, err.Error(), "missing") +} + +func TestResolve_TaskCycle(t *testing.T) { + t.Parallel() + + appSpec := &spec.Spec{ + Project: "shop", + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "a": {From: "api", On: spec.TaskOnPreDeploy, After: []string{"b"}, Command: []string{"true"}}, + "b": {From: "api", On: spec.TaskOnPreDeploy, After: []string{"a"}, Command: []string{"true"}}, + }, + } + _, _, err := spec.Resolve(appSpec, platformWithProfiles(), spec.NormalizeEnv("production"), spec.SubstitutionReport{}) + require.Error(t, err) + assert.Contains(t, err.Error(), "cycle") +} + +func TestCrossCheckPlatformReferences_TaskProfilesWithoutSection(t *testing.T) { + t.Parallel() + + appSpec := &spec.Spec{ + Components: map[string]spec.Component{ + "api": {Image: "nginx:latest"}, + }, + Tasks: map[string]spec.Task{ + "migrate": { + From: "api", + On: spec.TaskOnPreDeploy, + Command: []string{"true"}, + Profiles: []string{"batch"}, + }, + }, + } + problems, _ := spec.CrossCheckPlatformReferences(appSpec, minimalPlatform()) + require.Len(t, problems, 1) + assert.Contains(t, problems[0], `task "migrate"`) + assert.Contains(t, problems[0], "no profiles section") +} diff --git a/internal/spec/profile.go b/internal/spec/profile.go index 4f41f6b..01b128c 100644 --- a/internal/spec/profile.go +++ b/internal/spec/profile.go @@ -61,7 +61,7 @@ func ResolveProfileNames(componentProfiles []string, platformProfiles map[string if platformProfiles == nil { return nil, &ResolutionError{ Code: ErrCodeProfileNotFound, - Message: "component sets profiles but the platform file has no profiles section; " + + Message: "sets profiles but the platform file has no profiles section; " + "add a root-level profiles map to deployah.platform.yaml", } } @@ -205,16 +205,77 @@ func mergePVCRetentionPolicy(base, overlay *PVCRetentionPolicy) *PVCRetentionPol return out } -// ValidateProfileAgainstComponent checks domain, storage class, and resource -// ceiling constraints from the merged profile against the component and -// target environment. -func ValidateProfileAgainstComponent( - compName string, - comp Component, +// ProfileSubject is the kind of spec object a profile is checked against. +type ProfileSubject string + +const ( + // ProfileSubjectComponent is a [Component]. + ProfileSubjectComponent ProfileSubject = "component" + // ProfileSubjectTask is a [Task]. + ProfileSubjectTask ProfileSubject = "task" +) + +// ProfileTarget is the part of a component or task that profile constraints +// apply to. +type ProfileTarget struct { + // Subject is whether the target is a component or a task. It appears in + // error messages. + Subject ProfileSubject + // Name is the component or task name. + Name string + // Resources are the explicit resource requests. Empty falls back to + // ResourcePreset. + Resources Resources + // ResourcePreset names the preset used when Resources is empty. Empty + // means [ResourcePresetSmall]. + ResourcePreset ResourcePreset + // Metrics is the component metrics block. Always nil for tasks, which + // cannot enable metrics. + Metrics *ComponentMetrics +} + +func componentProfileTarget(name string, comp Component) ProfileTarget { + return ProfileTarget{ + Subject: ProfileSubjectComponent, + Name: name, + Resources: comp.Resources, + ResourcePreset: comp.ResourcePreset, + Metrics: comp.Metrics, + } +} + +func taskProfileTarget(name string, task Task) ProfileTarget { + return ProfileTarget{ + Subject: ProfileSubjectTask, + Name: name, + Resources: task.Resources, + ResourcePreset: task.ResourcePreset, + } +} + +// newMonitorLabelsError reports metrics enabled with no monitorLabels on the +// merged profile. Shared so the with-profile and no-profile paths cannot +// drift. +func newMonitorLabelsError(subject ProfileSubject, name string) *ResolutionError { + return &ResolutionError{ + Code: ErrCodeProfileMonitorLabelsMissing, + Message: fmt.Sprintf( + "%s %q has metrics enabled but the merged profile has no metrics.monitorLabels; "+ + "set metrics.monitorLabels on a platform profile so Prometheus Operator can discover the monitor", + subject, name, + ), + } +} + +// ValidateProfile checks domain, storage class, and resource ceiling +// constraints from the merged profile against target. +func ValidateProfile( + target ProfileTarget, merged PlatformProfile, platformEnv *PlatformEnvironment, domainKey string, ) error { + subject, name := target.Subject, target.Name // Non-nil AllowedDomains (including empty) means the profile constrains // domains; nil means unconstrained. if merged.AllowedDomains != nil && domainKey != "" { @@ -222,8 +283,8 @@ func ValidateProfileAgainstComponent( return &ResolutionError{ Code: ErrCodeProfileDomainNotAllowed, Message: fmt.Sprintf( - "component %q expose.domain %q is not allowed by its profiles (allowed: %s)", - compName, domainKey, joinStrings(merged.AllowedDomains), + "%s %q expose.domain %q is not allowed by its profiles (allowed: %s)", + subject, name, domainKey, joinStrings(merged.AllowedDomains), ), } } @@ -234,8 +295,8 @@ func ValidateProfileAgainstComponent( return &ResolutionError{ Code: ErrCodeProfileStorageClassNotFound, Message: fmt.Sprintf( - "component %q profile references storageClass %q but the environment has no storageClasses", - compName, merged.StorageClass, + "%s %q profile references storageClass %q but the environment has no storageClasses", + subject, name, merged.StorageClass, ), } } @@ -244,44 +305,38 @@ func ValidateProfileAgainstComponent( return &ResolutionError{ Code: ErrCodeProfileStorageClassNotFound, Message: fmt.Sprintf( - "component %q profile references storageClass %q but environment does not define it (available: %s)", - compName, merged.StorageClass, joinStrings(available), + "%s %q profile references storageClass %q but environment does not define it (available: %s)", + subject, name, merged.StorageClass, joinStrings(available), ), } } } if merged.MaxResources != nil { - if err := checkResourceCeiling(compName, comp, merged.MaxResources); err != nil { + if err := checkResourceCeiling(target, merged.MaxResources); err != nil { return err } } - if comp.Metrics.IsEnabled() && (merged.Metrics == nil || len(merged.Metrics.MonitorLabels) == 0) { - return &ResolutionError{ - Code: ErrCodeProfileMonitorLabelsMissing, - Message: fmt.Sprintf( - "component %q has metrics enabled but the merged profile has no metrics.monitorLabels; "+ - "set metrics.monitorLabels on a platform profile so Prometheus Operator can discover the monitor", - compName, - ), - } + if target.Metrics.IsEnabled() && (merged.Metrics == nil || len(merged.Metrics.MonitorLabels) == 0) { + return newMonitorLabelsError(subject, name) } return nil } -func checkResourceCeiling(compName string, comp Component, max *ProfileMaxResources) error { +func checkResourceCeiling(target ProfileTarget, max *ProfileMaxResources) error { + subject, name := target.Subject, target.Name // Compare against effective requests (explicit resources, named preset, or // the default small preset) so validate/resolve match deploy after Load. - req := effectiveResourceRequests(comp) + req := effectiveResourceRequests(target) if quantitySet(max.CPU) && quantitySet(req.CPU) { if req.CPU.Cmp(*max.CPU) > 0 { return &ResolutionError{ Code: ErrCodeProfileResourceExceeded, Message: fmt.Sprintf( - "component %q CPU request %s exceeds profile maxResources.cpu %s", - compName, req.CPU.String(), max.CPU.String(), + "%s %q CPU request %s exceeds profile maxResources.cpu %s", + subject, name, req.CPU.String(), max.CPU.String(), ), } } @@ -291,8 +346,8 @@ func checkResourceCeiling(compName string, comp Component, max *ProfileMaxResour return &ResolutionError{ Code: ErrCodeProfileResourceExceeded, Message: fmt.Sprintf( - "component %q memory request %s exceeds profile maxResources.memory %s", - compName, req.Memory.String(), max.Memory.String(), + "%s %q memory request %s exceeds profile maxResources.memory %s", + subject, name, req.Memory.String(), max.Memory.String(), ), } } @@ -302,11 +357,11 @@ func checkResourceCeiling(compName string, comp Component, max *ProfileMaxResour // effectiveResourceRequests returns the resource requests deploy would apply: // explicit resources when set, otherwise the named or default small preset. -func effectiveResourceRequests(comp Component) Resources { - if comp.Resources.ResourcesSet() { - return comp.Resources +func effectiveResourceRequests(target ProfileTarget) Resources { + if target.Resources.ResourcesSet() { + return target.Resources } - preset := comp.ResourcePreset + preset := target.ResourcePreset if preset == "" { preset = ResourcePresetSmall } diff --git a/internal/spec/profile_test.go b/internal/spec/profile_test.go index 5008673..37321eb 100644 --- a/internal/spec/profile_test.go +++ b/internal/spec/profile_test.go @@ -370,12 +370,13 @@ func TestMergeProfiles(t *testing.T) { }) } -// TestValidateProfileAgainstComponent covers domain, storage, and ceiling -// checks. -func TestValidateProfileAgainstComponent(t *testing.T) { +// TestValidateProfile covers domain, storage, and ceiling checks. +func TestValidateProfile(t *testing.T) { t.Parallel() - comp := spec.Component{ + target := spec.ProfileTarget{ + Subject: spec.ProfileSubjectComponent, + Name: "api", Resources: spec.Resources{ CPU: spec.MustQuantity("2000m"), Memory: spec.MustQuantity("1Gi"), @@ -389,7 +390,7 @@ func TestValidateProfileAgainstComponent(t *testing.T) { t.Run("domain not allowed", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ AllowedDomains: []string{"public"}, }, env, "internal") require.Error(t, err) @@ -400,7 +401,7 @@ func TestValidateProfileAgainstComponent(t *testing.T) { t.Run("empty allowedDomains deny-all", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ AllowedDomains: []string{}, }, env, "public") require.Error(t, err) @@ -411,7 +412,7 @@ func TestValidateProfileAgainstComponent(t *testing.T) { t.Run("domain ignored without expose key", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ AllowedDomains: []string{"public"}, }, env, "") require.NoError(t, err) @@ -419,7 +420,7 @@ func TestValidateProfileAgainstComponent(t *testing.T) { t.Run("storage class missing", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ StorageClass: "missing", }, env, "") require.Error(t, err) @@ -430,18 +431,34 @@ func TestValidateProfileAgainstComponent(t *testing.T) { t.Run("cpu exceeds ceiling", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ MaxResources: &spec.ProfileMaxResources{CPU: spec.MustQuantity("1000m")}, }, env, "") require.Error(t, err) var re *spec.ResolutionError require.ErrorAs(t, err, &re) assert.Equal(t, spec.ErrCodeProfileResourceExceeded, re.Code) + assert.Contains(t, re.Error(), `component "api"`) + }) + + t.Run("task kind in ceiling error", func(t *testing.T) { + t.Parallel() + taskTarget := spec.ProfileTarget{ + Subject: spec.ProfileSubjectTask, + Name: "migrate", + Resources: target.Resources, + } + err := spec.ValidateProfile(taskTarget, spec.PlatformProfile{ + MaxResources: &spec.ProfileMaxResources{CPU: spec.MustQuantity("1000m")}, + }, env, "") + require.Error(t, err) + assert.Contains(t, err.Error(), `task "migrate"`) + assert.NotContains(t, err.Error(), "component") }) t.Run("memory exceeds ceiling", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ MaxResources: &spec.ProfileMaxResources{Memory: spec.MustQuantity("512Mi")}, }, env, "") require.Error(t, err) @@ -453,7 +470,7 @@ func TestValidateProfileAgainstComponent(t *testing.T) { t.Run("storage class with no environment storageClasses", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ StorageClass: "fast", }, &spec.PlatformEnvironment{}, "") require.Error(t, err) @@ -465,7 +482,7 @@ func TestValidateProfileAgainstComponent(t *testing.T) { t.Run("within ceiling", func(t *testing.T) { t.Parallel() - err := spec.ValidateProfileAgainstComponent("api", comp, spec.PlatformProfile{ + err := spec.ValidateProfile(target, spec.PlatformProfile{ MaxResources: &spec.ProfileMaxResources{CPU: spec.MustQuantity("2000m"), Memory: spec.MustQuantity("2Gi")}, StorageClass: "fast", AllowedDomains: []string{"public"}, @@ -516,12 +533,14 @@ func TestMergeProfiles_MonitorFields(t *testing.T) { assert.Equal(t, map[string]string{"team": "platform", "owner": "sre"}, merged.Metrics.Annotations) } -func TestValidateProfileAgainstComponent_MonitorLabelsRequired(t *testing.T) { +func TestValidateProfile_MonitorLabelsRequired(t *testing.T) { t.Parallel() - comp := spec.Component{ + target := spec.ProfileTarget{ + Subject: spec.ProfileSubjectComponent, + Name: "worker", Metrics: &spec.ComponentMetrics{Port: 9090}, } - err := spec.ValidateProfileAgainstComponent("worker", comp, spec.PlatformProfile{}, nil, "") + err := spec.ValidateProfile(target, spec.PlatformProfile{}, nil, "") require.Error(t, err) var re *spec.ResolutionError require.ErrorAs(t, err, &re) diff --git a/internal/spec/resolve.go b/internal/spec/resolve.go index 7e1674f..8d6c5b0 100644 --- a/internal/spec/resolve.go +++ b/internal/spec/resolve.go @@ -46,6 +46,7 @@ func Resolve( Spec: appSpec, Env: env, Components: make(map[string]ResolvedComponent), + Tasks: make(map[string]ResolvedTask), } // Resolve Kubernetes context from platform. @@ -134,6 +135,10 @@ func Resolve( resolved.Components[compName] = rc } + if err := resolveTasks(appSpec, env, platform, platformEnv, resolved, report); err != nil { + return nil, report, err + } + return resolved, report, nil } @@ -173,7 +178,7 @@ func resolveComponent( profileNames, err := ResolveProfileNames(comp.Profiles, platformProfiles) if err != nil { - return rc, result, err + return rc, result, fmt.Errorf("component %q: %w", name, err) } if len(profileNames) > 0 { merged, mergeErr := MergeProfiles(profileNames, platformProfiles) @@ -386,17 +391,10 @@ func validateMergedProfile( domainKey string, ) error { if merged != nil { - return ValidateProfileAgainstComponent(name, comp, *merged, platformEnv, domainKey) + return ValidateProfile(componentProfileTarget(name, comp), *merged, platformEnv, domainKey) } if comp.Metrics.IsEnabled() { - return &ResolutionError{ - Code: ErrCodeProfileMonitorLabelsMissing, - Message: fmt.Sprintf( - "component %q has metrics enabled but the merged profile has no metrics.monitorLabels; "+ - "set metrics.monitorLabels on a platform profile so Prometheus Operator can discover the monitor", - name, - ), - } + return newMonitorLabelsError(ProfileSubjectComponent, name) } return nil } @@ -502,6 +500,16 @@ func unknownEnvironmentNameWarnings(appSpec *Spec, registry []string) []string { } } } + + for _, taskName := range appSpec.TaskNames() { + for _, entry := range appSpec.Tasks[taskName].Environments { + if _, ok := matchEnvKey(entry, registry); !ok { + warnings = append(warnings, fmt.Sprintf( + "task %q environments filter entry %q matches no environment in the platform file (available: %s)", + taskName, entry, joinStrings(registry))) + } + } + } return warnings } @@ -579,6 +587,26 @@ func CrossCheckPlatformReferences(appSpec *Spec, platform *PlatformConfig) (prob } } } + + for _, name := range appSpec.TaskNames() { + task := appSpec.Tasks[name] + if len(task.Profiles) == 0 { + continue + } + if platform.Profiles == nil { + problems = append(problems, fmt.Sprintf( + "task %q sets profiles but the platform file has no profiles section", + name)) + continue + } + for _, profileName := range task.Profiles { + if _, ok := platform.Profiles[profileName]; !ok { + problems = append(problems, fmt.Sprintf( + "task %q references profile %q which is not defined in the platform file (available: %s)", + name, profileName, joinStrings(profileKeys))) + } + } + } return problems, warnings } @@ -617,6 +645,7 @@ func ResolveForDisplay( Spec: appSpec, Env: env, Components: make(map[string]ResolvedComponent), + Tasks: make(map[string]ResolvedTask), } return resolved, report, nil } @@ -638,3 +667,59 @@ func PlatformEnvContext(platform *PlatformConfig, envName string) string { } return "" } + +func resolveTasks(appSpec *Spec, env EnvIdentity, platform *PlatformConfig, platformEnv *PlatformEnvironment, resolved *ResolvedSpec, report *ResolutionReport) error { + weights, err := AssignHookWeights(appSpec.Tasks) + if err != nil { + return err + } + var platformProfiles map[string]PlatformProfile + if platform != nil { + platformProfiles = platform.Profiles + } + for _, name := range appSpec.TaskNames() { + task, ok := appSpec.MergedTask(name) + if !ok || !task.activeInEnvironment(env.Original) { + continue + } + if len(task.Profiles) > 0 && platform == nil { + return &ResolutionError{ + Code: ErrCodePlatformNotFound, + Message: fmt.Sprintf( + "task %q sets profiles but no platform file was found; "+ + "pass --platform-file or create %s", + name, DefaultPlatformPath, + ), + } + } + profileNames, profErr := ResolveProfileNames(task.Profiles, platformProfiles) + if profErr != nil { + return fmt.Errorf("task %q: %w", name, profErr) + } + rt := ResolvedTask{Task: task, HookWeight: weights[name], Profiles: profileNames} + if len(profileNames) > 0 { + merged, mergeErr := MergeProfiles(profileNames, platformProfiles) + if mergeErr != nil { + return fmt.Errorf("task %q: %w", name, mergeErr) + } + rt.MergedProfile = &merged + if profileErr := ValidateProfile(taskProfileTarget(name, task), merged, platformEnv, ""); profileErr != nil { + return profileErr + } + report.Fields = append(report.Fields, ResolvedField{ + Component: name, + Path: "profiles", + Value: strings.Join(profileNames, ", "), + Source: "platform profiles (merged left to right)", + }) + } + resolved.Tasks[name] = rt + report.Fields = append(report.Fields, ResolvedField{ + Component: name, + Path: "on", + Value: string(task.On), + Source: "spec tasks." + name + ".on", + }) + } + return nil +} diff --git a/internal/spec/resolved_spec.go b/internal/spec/resolved_spec.go index f0fc1bd..91cd3c6 100644 --- a/internal/spec/resolved_spec.go +++ b/internal/spec/resolved_spec.go @@ -37,6 +37,8 @@ type ResolvedSpec struct { KubeContext string // Components holds the per-component resolved data. Components map[string]ResolvedComponent + // Tasks holds the per-task resolved data for tasks active in Env. + Tasks map[string]ResolvedTask // Warnings is the list of non-fatal resolution warnings. Warnings []string } @@ -74,6 +76,22 @@ type ResolvedComponent struct { DomainKey string } +// ResolvedTask holds the merged, environment-filtered task used for chart +// generation and deployah run. +type ResolvedTask struct { + // Task is the task after from-merge and defaults. + Task Task + // HookWeight is the Helm hook-weight for preDeploy/postDeploy tasks. + // Zero for independent hooks; unused for manual tasks. + HookWeight int + // Profiles is the ordered list of profile names applied after default + // prepend. Empty when no profiles apply. + Profiles []string + // MergedProfile is the left-to-right merge of Profiles. Nil when no + // profiles apply. + MergedProfile *PlatformProfile +} + // ResolutionReport holds the provenance of each resolved field, enabling // the resolve command and deploy --explain to trace where each value came from. type ResolutionReport struct { @@ -91,8 +109,9 @@ type ResolutionReport struct { // ResolvedField holds provenance for a single resolved value. type ResolvedField struct { - // Component is the component name this field belongs to, or empty for - // top-level fields. + // Component is the component or task name this field belongs to, or + // empty for top-level fields. Components and tasks share one name + // pool, so the name is unambiguous. Component string // Path is the spec path, e.g. "expose.host". Path string diff --git a/internal/spec/schema/schema.go b/internal/spec/schema/schema.go index 6eb9e07..4968381 100644 --- a/internal/spec/schema/schema.go +++ b/internal/spec/schema/schema.go @@ -38,7 +38,7 @@ const ( const platformSchemaDir = "platform" var ( - // versionRegex matches version strings such as "v1-alpha.4", "v1-beta.2", + // versionRegex matches version strings such as "v1-alpha.5", "v1-beta.2", // and similar pre-release formats. versionRegex = regexp.MustCompile(`^v(\d+)(?:-(alpha|beta|rc)\.(\d+))?$`) // preReleaseOrder is a map that defines the order of pre-release types. @@ -47,7 +47,7 @@ var ( // GetManifestSchema retrieves the JSON schema for validating manifests at a // specific version. -// Version strings should follow the format "v1-alpha.4", "v1-beta.2", etc. +// Version strings should follow the format "v1-alpha.5", "v1-beta.2", etc. // The schema file must be named "manifest.json" within the version directory. func GetManifestSchema(version string) ([]byte, error) { fileName := version + "/manifest.json" @@ -58,7 +58,7 @@ func GetManifestSchema(version string) ([]byte, error) { } // GetEnvironmentsSchema returns the environments schema for the given version. -// Version strings should follow the format "v1-alpha.4", "v1-beta.2", etc. +// Version strings should follow the format "v1-alpha.5", "v1-beta.2", etc. // The schema file must be named "environments.json" within the version directory. func GetEnvironmentsSchema(version string) ([]byte, error) { fileName := version + "/environments.json" @@ -97,7 +97,7 @@ func GetManifestSchemas() (map[string][]byte, error) { // GetPlatformSchema retrieves the JSON schema for validating platform configs // at a specific version. Version strings should follow the format -// "v1-alpha.4", "v1-beta.2", etc. The schema file must be named +// "v1-alpha.5", "v1-beta.2", etc. The schema file must be named // "platform.json" within the platform/VERSION directory. func GetPlatformSchema(version string) ([]byte, error) { fileName := platformSchemaDir + "/" + version + "/" + SchemaTypePlatform.String() + ".json" diff --git a/internal/spec/schema/schema_test.go b/internal/spec/schema/schema_test.go index 4dd789b..9ab3257 100644 --- a/internal/spec/schema/schema_test.go +++ b/internal/spec/schema/schema_test.go @@ -16,7 +16,7 @@ type SchemaTestSuite struct { // TestGetManifestSchema verifies manifest schema retrieval for a version. func (s *SchemaTestSuite) TestGetManifestSchema() { - schema, err := GetManifestSchema("v1-alpha.4") + schema, err := GetManifestSchema("v1-alpha.5") s.Require().NoError(err) s.Require().NotNil(schema) } @@ -32,7 +32,7 @@ func (s *SchemaTestSuite) TestGetManifestSchema_InvalidVersion() { // TestGetEnvironmentsSchema verifies environments schema retrieval for a // version. func (s *SchemaTestSuite) TestGetEnvironmentsSchema() { - schema, err := GetEnvironmentsSchema("v1-alpha.4") + schema, err := GetEnvironmentsSchema("v1-alpha.5") s.Require().NoError(err) s.Require().NotNil(schema) } diff --git a/internal/spec/schema/v1-alpha.4/manifest.json b/internal/spec/schema/v1-alpha.4/manifest.json deleted file mode 100644 index 3765f5a..0000000 --- a/internal/spec/schema/v1-alpha.4/manifest.json +++ /dev/null @@ -1,592 +0,0 @@ -{ - "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://deployah.dev/schemas/v1-alpha.4/manifest.json", - "title": "Deployah Spec v1-alpha.4", - "description": "Deployah developer manifest (deployah.yaml). Environments are a map, context is platform-owned, expose replaces ingress. Supports role: worker and Prometheus metrics.", - "type": "object", - "additionalProperties": false, - "required": ["apiVersion", "project", "components"], - "properties": { - "apiVersion": { - "type": "string", - "title": "API Version", - "description": "Schema version. Must be 'v1-alpha.4'.", - "const": "v1-alpha.4" - }, - "project": { - "type": "string", - "title": "Project Name", - "description": "The project name used to identify deployments and prefix Kubernetes resources. Must be a valid DNS-1123 subdomain.", - "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", - "minLength": 3, - "maxLength": 64 - }, - "environments": { - "type": "object", - "title": "Environments", - "description": "Optional map of environment names to developer overrides (envFile, variables). Which environments exist is owned by the platform file when one is present. Keys support prefix-based wildcard matching.", - "propertyNames": { - "type": "string", - "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", - "minLength": 2 - }, - "additionalProperties": { - "$ref": "#/$defs/Environment" - } - }, - "components": { - "type": "object", - "title": "Components", - "description": "Map of component names to their configuration.", - "propertyNames": { - "type": "string", - "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", - "minLength": 2, - "maxLength": 63 - }, - "minProperties": 1, - "additionalProperties": { - "$ref": "#/$defs/Component" - } - } - }, - "$defs": { - "Environment": { - "type": "object", - "title": "Environment", - "description": "Developer-owned environment definition. Context is platform-owned and lives in deployah.platform.yaml.", - "additionalProperties": false, - "properties": { - "envFile": { - "type": "string", - "title": "Environment File", - "description": "Path to a dotenv file for variable substitution in this environment.", - "examples": [".env.production"] - }, - "configFile": { - "type": "string", - "title": "Configuration File", - "description": "Path to an environment-specific configuration file.", - "examples": ["config.production.yaml"] - }, - "variables": { - "type": "object", - "title": "Inline Variables", - "description": "Inline key-value variable overrides for this environment.", - "propertyNames": { - "type": "string", - "pattern": "^[A-Z0-9]+(?:_[A-Z0-9]+)*$" - }, - "additionalProperties": { - "type": ["string", "number", "boolean"] - }, - "examples": [ - { - "APP_ENV": "production", - "REPLICAS": 3, - "DEBUG": false - } - ] - } - }, - "examples": [ - {"envFile": ".env.production", "configFile": "config.production.yaml"}, - {} - ] - }, - "Component": { - "type": "object", - "title": "Component", - "description": "A deployable unit in the project.", - "additionalProperties": false, - "required": ["image"], - "properties": { - "role": { - "type": "string", - "title": "Component Role", - "description": "Deployment strategy. 'service' for HTTP services, 'worker' for background tasks, 'job' for one-off tasks.", - "default": "service", - "enum": ["service", "worker", "job"] - }, - "kind": { - "type": "string", - "title": "Component Kind", - "description": "'stateless' for interchangeable Deployment replicas. 'stateful' for StatefulSet replicas with stable network identity; optional persistence for per-pod PVCs.", - "default": "stateless", - "enum": ["stateless", "stateful"] - }, - "image": { - "type": "string", - "title": "Container Image", - "description": "Container image reference, including optional tag or digest.", - "minLength": 1, - "examples": [ - "nginx:1.28.0-alpine", - "nginx@sha256:37075895d8461222f53afa7804aec2c57d69f9842995705cc54a0c4a70d68fc9", - "myregistry.com/myapp/backend:2.0", - "wait4x/wait4x:latest" - ] - }, - "command": { - "type": "array", - "title": "Command", - "description": "Container entrypoint override.", - "items": {"type": "string"}, - "examples": [ - ["python", "app.py"], - ["bash", "-c", "echo Hello, World!"] - ] - }, - "args": { - "type": "array", - "title": "Arguments", - "description": "Container command argument override.", - "items": {"type": "string"}, - "examples": [ - ["--port", "8080"], - ["--config", "/etc/config.yaml"] - ] - }, - "port": { - "type": "integer", - "title": "Port", - "description": "Primary container port for service components. Not allowed on role: worker. Defaults to 8080 for services when omitted.", - "minimum": 1, - "maximum": 65535, - "examples": [8181, 9000] - }, - "shutdownTimeout": { - "type": "string", - "title": "Shutdown Timeout", - "description": "How long Kubernetes waits after SIGTERM before force-killing the pod (terminationGracePeriodSeconds). Defaults to 30s for services and 60s for workers.", - "pattern": "^[1-9][0-9]*(s|m|h)$", - "examples": ["30s", "60s", "2m"] - }, - "metrics": { - "title": "Metrics", - "description": "Prometheus scraping configuration. Boolean shorthand or object. Services emit a ServiceMonitor; workers emit a PodMonitor. Workers require metrics.port when enabled. Requires prometheus-operator CRDs and platform profile metrics.monitorLabels.", - "anyOf": [ - {"type": "boolean"}, - {"$ref": "#/$defs/Metrics"} - ], - "examples": [true, false, {"port": 9090, "path": "/metrics"}] - }, - "replicas": { - "type": "integer", - "title": "Replicas", - "description": "Desired replica count when autoscaling is disabled. Default is 1. Mutually exclusive with autoscaling.enabled.", - "default": 1, - "minimum": 1, - "examples": [1, 2, 3] - }, - "persistence": { - "$ref": "#/$defs/Persistence" - }, - "expose": { - "title": "Expose", - "description": "Boolean shorthand or object. 'expose: true' exposes with all defaults (the environment's default domain, the component name as subdomain, platform TLS); 'expose: false' equals omitting the block.", - "anyOf": [ - {"type": "boolean"}, - {"$ref": "#/$defs/Expose"} - ], - "examples": [true, {"subdomain": "api"}] - }, - "envFile": { - "type": "string", - "title": "Component Environment File", - "description": "Component-specific dotenv file path.", - "examples": [".env.api", ".env.api.production"] - }, - "configFile": { - "type": "string", - "title": "Component Configuration File", - "description": "Component-specific configuration file path.", - "examples": ["config.api.yaml", "config.api.production.yaml"] - }, - "environments": { - "type": "array", - "title": "Environment Filter", - "description": "Restrict this component to the listed environment names. When absent the component is active in all environments. Entries support prefix-based wildcard matching, e.g. 'review' matches an --environment value of 'review/pr-123'.", - "minItems": 1, - "items": {"type": "string"}, - "examples": [ - ["production"], - ["review", "staging"] - ] - }, - "autoscaling": { - "$ref": "#/$defs/Autoscaling" - }, - "resources": { - "$ref": "#/$defs/Resources" - }, - "resourcePreset": { - "type": "string", - "title": "Resource Preset", - "description": "Named resource profile. Cannot be combined with explicit resources.", - "enum": ["nano", "micro", "small", "medium", "large", "xlarge", "2xlarge"], - "examples": ["small", "medium", "xlarge"] - }, - "profiles": { - "type": "array", - "title": "Profiles", - "description": "Logical names of platform-defined deployment profiles. Multiple profiles are merged left to right. An empty array opts out of the default profile when no default is defined; when a default profile exists, an empty array is rejected. Requires a profiles section in the platform file.", - "items": { - "type": "string", - "minLength": 1 - }, - "examples": [["public-web"], ["public-web", "high-security"]] - }, - "env": { - "type": "object", - "title": "Environment Variables", - "description": "Static environment variables for the container.", - "propertyNames": { - "pattern": "^[A-Z_][A-Z0-9_]*$" - }, - "additionalProperties": { - "type": ["string", "number", "boolean"] - }, - "examples": [ - { - "NODE_ENV": "production", - "DEBUG": true, - "MAX_RETRIES": 5, - "TIMEOUT": 30.5 - } - ] - }, - "health": { - "$ref": "#/$defs/Health" - } - } - }, - "Persistence": { - "type": "object", - "title": "Persistence", - "description": "Durable storage configuration. Optional on kind: stateful (per-pod volumeClaimTemplates when set; omit for identity-only). Allowed on kind: stateless (shared PVC; forces Recreate strategy; rejects replicas > 1 and autoscaling).", - "additionalProperties": false, - "required": ["size", "mountPath"], - "properties": { - "size": { - "type": "string", - "title": "Size", - "description": "Requested volume size as a Kubernetes quantity.", - "pattern": "^[1-9][0-9]*(Ki|Mi|Gi|Ti|Pi|Ei)$", - "examples": ["1Gi", "20Gi", "100Gi"] - }, - "mountPath": { - "type": "string", - "title": "Mount Path", - "description": "Absolute path where the volume is mounted in the container.", - "pattern": "^/", - "minLength": 1, - "examples": ["/data", "/var/lib/postgresql/data"] - }, - "storageClass": { - "type": "string", - "title": "Storage Class", - "description": "Logical storage class key from the platform environment's storageClasses map. Overrides the profile storageClass when set.", - "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", - "minLength": 1, - "examples": ["fast", "standard"] - } - }, - "examples": [ - {"size": "20Gi", "mountPath": "/data"}, - {"size": "50Gi", "mountPath": "/var/lib/postgresql/data", "storageClass": "fast"} - ] - }, - "Expose": { - "type": "object", - "title": "Expose", - "description": "Exposes the component via an ingress rule resolved against the platform domain configuration. All fields are optional: the domain defaults to the environment's only (or default-marked) domain, and the subdomain defaults to the component name.", - "additionalProperties": false, - "properties": { - "domain": { - "type": "string", - "title": "Domain Key", - "description": "Domain key referencing an entry in the platform environment's domains map. When absent, the environment's only domain is used, or the one marked 'default: true' in the platform file.", - "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", - "minLength": 1, - "examples": ["public", "internal"] - }, - "subdomain": { - "type": "string", - "title": "Subdomain", - "description": "DNS label prepended to baseDomain to form the FQDN. When absent the component name is used. Must be a valid DNS-1123 label. Mutually exclusive with apex.", - "pattern": "^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$", - "minLength": 1, - "examples": ["api", "www"] - }, - "apex": { - "type": "boolean", - "title": "Apex", - "description": "Expose the component at the baseDomain itself instead of a subdomain. Mutually exclusive with subdomain.", - "default": false, - "examples": [true] - } - }, - "examples": [ - {"subdomain": "api"}, - {"domain": "internal"}, - {"apex": true} - ] - }, - "Autoscaling": { - "type": "object", - "title": "Autoscaling", - "description": "Horizontal pod autoscaling configuration.", - "additionalProperties": false, - "required": ["enabled", "minReplicas", "maxReplicas"], - "properties": { - "enabled": { - "type": "boolean", - "description": "Whether autoscaling is enabled.", - "default": false - }, - "minReplicas": { - "type": "integer", - "description": "Minimum number of replicas.", - "default": 2, - "minimum": 1 - }, - "maxReplicas": { - "type": "integer", - "description": "Maximum number of replicas.", - "default": 5, - "minimum": 1 - }, - "metrics": { - "type": "array", - "minItems": 1, - "items": { - "type": "object", - "additionalProperties": false, - "required": ["type", "target"], - "properties": { - "type": { - "type": "string", - "enum": ["cpu", "memory"] - }, - "target": { - "type": "integer", - "minimum": 1 - } - } - }, - "default": [{"type": "cpu", "target": 75}], - "examples": [ - [ - {"type": "cpu", "target": 70}, - {"type": "memory", "target": 80} - ] - ] - } - }, - "examples": [ - { - "enabled": true, - "minReplicas": 2, - "maxReplicas": 6, - "metrics": [{"type": "cpu", "target": 70}] - }, - { - "enabled": true, - "minReplicas": 3, - "maxReplicas": 10, - "metrics": [ - {"type": "cpu", "target": 65}, - {"type": "memory", "target": 75} - ] - } - ] - }, - "Resources": { - "type": "object", - "title": "Resources", - "description": "CPU and memory requests/limits.", - "additionalProperties": false, - "properties": { - "cpu": { - "type": "string", - "pattern": "^([1-9][0-9]*m?|[1-9][0-9]*|0m)$", - "examples": ["500m", "1"] - }, - "memory": { - "type": "string", - "pattern": "^[1-9][0-9]*(Ki|Mi|Gi|Ti|Pi|Ei)$", - "examples": ["256Mi", "1Gi"] - }, - "ephemeralStorage": { - "type": "string", - "pattern": "^[1-9][0-9]*(Ki|Mi|Gi|Ti|Pi|Ei)$", - "examples": ["500Mi", "2Gi"] - } - }, - "examples": [ - {"cpu": "500m", "memory": "512Mi"}, - {"ephemeralStorage": "2Gi"} - ] - }, - "Metrics": { - "type": "object", - "title": "Metrics Configuration", - "description": "Prometheus scrape settings. enabled defaults to true when omitted. port is required for workers; for services it defaults to the component port.", - "additionalProperties": false, - "properties": { - "enabled": { - "type": "boolean", - "title": "Enabled", - "description": "When false, disables scraping (useful for per-environment toggle via envsubst). Defaults to true when omitted.", - "default": true - }, - "port": { - "type": "integer", - "title": "Metrics Port", - "description": "Container port exposing metrics. Required for workers. Defaults to the component port for services.", - "minimum": 1, - "maximum": 65535, - "examples": [8080, 9090] - }, - "path": { - "type": "string", - "title": "Metrics Path", - "description": "HTTP path for metrics. Defaults to /metrics.", - "pattern": "^/", - "default": "/metrics", - "examples": ["/metrics", "/actuator/prometheus"] - }, - "interval": { - "type": "string", - "title": "Scrape Interval", - "description": "Override the platform profile scrape interval.", - "pattern": "^[1-9][0-9]*(s|m|h)$", - "examples": ["15s", "30s", "1m"] - }, - "scrapeTimeout": { - "type": "string", - "title": "Scrape Timeout", - "description": "Override the platform profile scrape timeout.", - "pattern": "^[1-9][0-9]*(s|m|h)$", - "examples": ["5s", "10s"] - } - }, - "examples": [ - {"port": 9090}, - {"port": 8080, "path": "/metrics", "interval": "30s"}, - {"enabled": false} - ] - }, - "Health": { - "type": "object", - "title": "Health Checks", - "description": "Health check configuration. Services support TCP/HTTP ready and alive checks. Workers support optional alive.exec only.", - "additionalProperties": false, - "properties": { - "ready": { - "oneOf": [ - {"type": "boolean", "const": false}, - { - "type": "object", - "additionalProperties": false, - "required": ["path"], - "properties": { - "path": { - "type": "string", - "pattern": "^/", - "examples": ["/health", "/ready", "/healthz"] - } - } - } - ], - "examples": [ - {"path": "/health"}, - false - ] - }, - "alive": { - "oneOf": [ - {"type": "boolean", "const": false}, - { - "type": "object", - "additionalProperties": false, - "properties": { - "path": { - "type": "string", - "pattern": "^/", - "examples": ["/livez", "/healthz", "/alive"] - }, - "exec": { - "type": "array", - "title": "Exec Command", - "description": "Command run inside the container; exit 0 means alive. Mutually exclusive with path. Available for service and worker roles.", - "minItems": 1, - "items": {"type": "string"}, - "examples": [["pgrep", "-f", "worker"], ["sh", "-c", "test -f /tmp/healthy"]] - }, - "interval": { - "type": "string", - "pattern": "^[1-9][0-9]*(s|m|h)$", - "default": "10s", - "examples": ["10s", "30s", "1m"] - }, - "restartAfter": { - "type": "string", - "pattern": "^[1-9][0-9]*(s|m|h)$", - "default": "60s", - "examples": ["60s", "2m", "5m"] - } - }, - "oneOf": [ - {"required": ["path"]}, - {"required": ["exec"]} - ] - } - ], - "examples": [ - {"path": "/livez"}, - {"path": "/livez", "interval": "10s", "restartAfter": "60s"}, - {"exec": ["pgrep", "-f", "worker"]}, - false - ] - } - }, - "examples": [ - {"ready": {"path": "/health"}}, - {"ready": {"path": "/health"}, "alive": {"path": "/livez"}}, - {"alive": {"exec": ["pgrep", "-f", "worker"]}}, - {"ready": false, "alive": false} - ] - } - }, - "examples": [ - { - "apiVersion": "v1-alpha.4", - "project": "shop", - "environments": { - "production": { - "envFile": ".env.production" - }, - "local": {} - }, - "components": { - "api": { - "image": "my-app:latest", - "port": 8080, - "expose": { - "domain": "public", - "subdomain": "api" - }, - "metrics": true - }, - "worker": { - "image": "my-worker:latest", - "role": "worker", - "command": ["./worker"], - "metrics": {"port": 9090} - } - } - } - ] -} diff --git a/internal/spec/schema/v1-alpha.4/environments.json b/internal/spec/schema/v1-alpha.5/environments.json similarity index 94% rename from internal/spec/schema/v1-alpha.4/environments.json rename to internal/spec/schema/v1-alpha.5/environments.json index 821cc5f..0c9689d 100644 --- a/internal/spec/schema/v1-alpha.4/environments.json +++ b/internal/spec/schema/v1-alpha.5/environments.json @@ -1,8 +1,8 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://deployah.dev/schemas/v1-alpha.4/environments.json", - "title": "Deployah Environments v1-alpha.4", - "description": "Schema for the environments section of the v1-alpha.4 manifest. The section is optional: which environments exist is owned by the platform file; an entry here only adds developer overrides (envFile, variables) for that environment.", + "$id": "https://deployah.dev/schemas/v1-alpha.5/environments.json", + "title": "Deployah Environments v1-alpha.5", + "description": "Schema for the environments section of the v1-alpha.5 manifest. The section is optional: which environments exist is owned by the platform file; an entry here only adds developer overrides (envFile, variables) for that environment.", "type": "object", "additionalProperties": true, "properties": { diff --git a/internal/spec/schema/v1-alpha.5/manifest.json b/internal/spec/schema/v1-alpha.5/manifest.json new file mode 100644 index 0000000..230f4f3 --- /dev/null +++ b/internal/spec/schema/v1-alpha.5/manifest.json @@ -0,0 +1,1090 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://deployah.dev/schemas/v1-alpha.5/manifest.json", + "title": "Deployah Spec v1-alpha.5", + "description": "Deployah developer manifest (deployah.yaml). Environments are a map, context is platform-owned, expose replaces ingress. Components are service or worker. Run-to-completion work lives under tasks.", + "type": "object", + "additionalProperties": false, + "required": [ + "apiVersion", + "project", + "components" + ], + "properties": { + "apiVersion": { + "type": "string", + "title": "API Version", + "description": "Schema version. Must be 'v1-alpha.5'.", + "const": "v1-alpha.5" + }, + "project": { + "type": "string", + "title": "Project Name", + "description": "The project name used to identify deployments and prefix Kubernetes resources. Must be a valid DNS-1123 subdomain.", + "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", + "minLength": 3, + "maxLength": 64 + }, + "environments": { + "type": "object", + "title": "Environments", + "description": "Optional map of environment names to developer overrides (envFile, variables). Which environments exist is owned by the platform file when one is present. Keys support prefix-based wildcard matching.", + "propertyNames": { + "type": "string", + "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", + "minLength": 2 + }, + "additionalProperties": { + "$ref": "#/$defs/Environment" + } + }, + "components": { + "type": "object", + "title": "Components", + "description": "Map of component names to their configuration.", + "propertyNames": { + "type": "string", + "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", + "minLength": 2, + "maxLength": 63 + }, + "minProperties": 1, + "additionalProperties": { + "$ref": "#/$defs/Component" + } + }, + "tasks": { + "type": "object", + "title": "Tasks", + "description": "Map of task names to run-to-completion work. Names must not collide with component names.", + "propertyNames": { + "type": "string", + "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", + "minLength": 2, + "maxLength": 63 + }, + "additionalProperties": { + "$ref": "#/$defs/Task" + } + } + }, + "$defs": { + "Environment": { + "type": "object", + "title": "Environment", + "description": "Developer-owned environment definition. Context is platform-owned and lives in deployah.platform.yaml.", + "additionalProperties": false, + "properties": { + "envFile": { + "type": "string", + "title": "Environment File", + "description": "Path to a dotenv file for variable substitution in this environment.", + "examples": [ + ".env.production" + ] + }, + "configFile": { + "type": "string", + "title": "Configuration File", + "description": "Path to an environment-specific configuration file.", + "examples": [ + "config.production.yaml" + ] + }, + "variables": { + "type": "object", + "title": "Inline Variables", + "description": "Inline key-value variable overrides for this environment.", + "propertyNames": { + "type": "string", + "pattern": "^[A-Z0-9]+(?:_[A-Z0-9]+)*$" + }, + "additionalProperties": { + "type": [ + "string", + "number", + "boolean" + ] + }, + "examples": [ + { + "APP_ENV": "production", + "REPLICAS": 3, + "DEBUG": false + } + ] + } + }, + "examples": [ + { + "envFile": ".env.production", + "configFile": "config.production.yaml" + }, + {} + ] + }, + "Component": { + "type": "object", + "title": "Component", + "description": "A deployable unit in the project.", + "additionalProperties": false, + "required": [ + "image" + ], + "properties": { + "role": { + "type": "string", + "title": "Component Role", + "description": "Deployment strategy. service for HTTP services, worker for background processes.", + "default": "service", + "enum": [ + "service", + "worker" + ] + }, + "kind": { + "type": "string", + "title": "Component Kind", + "description": "'stateless' for interchangeable Deployment replicas. 'stateful' for StatefulSet replicas with stable network identity; optional persistence for per-pod PVCs.", + "default": "stateless", + "enum": [ + "stateless", + "stateful" + ] + }, + "image": { + "type": "string", + "title": "Container Image", + "description": "Container image reference, including optional tag or digest.", + "minLength": 1, + "examples": [ + "nginx:1.28.0-alpine", + "nginx@sha256:37075895d8461222f53afa7804aec2c57d69f9842995705cc54a0c4a70d68fc9", + "myregistry.com/myapp/backend:2.0", + "wait4x/wait4x:latest" + ] + }, + "command": { + "type": "array", + "title": "Command", + "description": "Container entrypoint override.", + "items": { + "type": "string" + }, + "examples": [ + [ + "python", + "app.py" + ], + [ + "bash", + "-c", + "echo Hello, World!" + ] + ] + }, + "args": { + "type": "array", + "title": "Arguments", + "description": "Container command argument override.", + "items": { + "type": "string" + }, + "examples": [ + [ + "--port", + "8080" + ], + [ + "--config", + "/etc/config.yaml" + ] + ] + }, + "port": { + "type": "integer", + "title": "Port", + "description": "Primary container port for service components. Not allowed on role: worker. Defaults to 8080 for services when omitted.", + "minimum": 1, + "maximum": 65535, + "examples": [ + 8181, + 9000 + ] + }, + "shutdownTimeout": { + "type": "string", + "title": "Shutdown Timeout", + "description": "How long Kubernetes waits after SIGTERM before force-killing the pod (terminationGracePeriodSeconds). Defaults to 30s for services and 60s for workers.", + "pattern": "^[1-9][0-9]*(s|m|h)$", + "examples": [ + "30s", + "60s", + "2m" + ] + }, + "metrics": { + "title": "Metrics", + "description": "Prometheus scraping configuration. Boolean shorthand or object. Services emit a ServiceMonitor; workers emit a PodMonitor. Workers require metrics.port when enabled. Requires prometheus-operator CRDs and platform profile metrics.monitorLabels.", + "anyOf": [ + { + "type": "boolean" + }, + { + "$ref": "#/$defs/Metrics" + } + ], + "examples": [ + true, + false, + { + "port": 9090, + "path": "/metrics" + } + ] + }, + "replicas": { + "type": "integer", + "title": "Replicas", + "description": "Desired replica count when autoscaling is disabled. Default is 1. Mutually exclusive with autoscaling.enabled.", + "default": 1, + "minimum": 1, + "examples": [ + 1, + 2, + 3 + ] + }, + "persistence": { + "$ref": "#/$defs/Persistence" + }, + "expose": { + "title": "Expose", + "description": "Boolean shorthand or object. 'expose: true' exposes with all defaults (the environment's default domain, the component name as subdomain, platform TLS); 'expose: false' equals omitting the block.", + "anyOf": [ + { + "type": "boolean" + }, + { + "$ref": "#/$defs/Expose" + } + ], + "examples": [ + true, + { + "subdomain": "api" + } + ] + }, + "envFile": { + "type": "string", + "title": "Component Environment File", + "description": "Component-specific dotenv file path.", + "examples": [ + ".env.api", + ".env.api.production" + ] + }, + "configFile": { + "type": "string", + "title": "Component Configuration File", + "description": "Component-specific configuration file path.", + "examples": [ + "config.api.yaml", + "config.api.production.yaml" + ] + }, + "environments": { + "type": "array", + "title": "Environment Filter", + "description": "Restrict this component to the listed environment names. When absent the component is active in all environments. Entries support prefix-based wildcard matching, e.g. 'review' matches an --environment value of 'review/pr-123'.", + "minItems": 1, + "items": { + "type": "string" + }, + "examples": [ + [ + "production" + ], + [ + "review", + "staging" + ] + ] + }, + "autoscaling": { + "$ref": "#/$defs/Autoscaling" + }, + "resources": { + "$ref": "#/$defs/Resources" + }, + "resourcePreset": { + "type": "string", + "title": "Resource Preset", + "description": "Named resource profile. Cannot be combined with explicit resources.", + "enum": [ + "nano", + "micro", + "small", + "medium", + "large", + "xlarge", + "2xlarge" + ], + "examples": [ + "small", + "medium", + "xlarge" + ] + }, + "profiles": { + "type": "array", + "title": "Profiles", + "description": "Logical names of platform-defined deployment profiles. Multiple profiles are merged left to right. An empty array opts out of the default profile when no default is defined; when a default profile exists, an empty array is rejected. Requires a profiles section in the platform file.", + "items": { + "type": "string", + "minLength": 1 + }, + "examples": [ + [ + "public-web" + ], + [ + "public-web", + "high-security" + ] + ] + }, + "env": { + "type": "object", + "title": "Environment Variables", + "description": "Static environment variables for the container.", + "propertyNames": { + "pattern": "^[A-Z_][A-Z0-9_]*$" + }, + "additionalProperties": { + "type": [ + "string", + "number", + "boolean" + ] + }, + "examples": [ + { + "NODE_ENV": "production", + "DEBUG": true, + "MAX_RETRIES": 5, + "TIMEOUT": 30.5 + } + ] + }, + "health": { + "$ref": "#/$defs/Health" + } + } + }, + "Persistence": { + "type": "object", + "title": "Persistence", + "description": "Durable storage configuration. Optional on kind: stateful (per-pod volumeClaimTemplates when set; omit for identity-only). Allowed on kind: stateless (shared PVC; forces Recreate strategy; rejects replicas > 1 and autoscaling).", + "additionalProperties": false, + "required": [ + "size", + "mountPath" + ], + "properties": { + "size": { + "type": "string", + "title": "Size", + "description": "Requested volume size as a Kubernetes quantity.", + "pattern": "^[1-9][0-9]*(Ki|Mi|Gi|Ti|Pi|Ei)$", + "examples": [ + "1Gi", + "20Gi", + "100Gi" + ] + }, + "mountPath": { + "type": "string", + "title": "Mount Path", + "description": "Absolute path where the volume is mounted in the container.", + "pattern": "^/", + "minLength": 1, + "examples": [ + "/data", + "/var/lib/postgresql/data" + ] + }, + "storageClass": { + "type": "string", + "title": "Storage Class", + "description": "Logical storage class key from the platform environment's storageClasses map. Overrides the profile storageClass when set.", + "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", + "minLength": 1, + "examples": [ + "fast", + "standard" + ] + } + }, + "examples": [ + { + "size": "20Gi", + "mountPath": "/data" + }, + { + "size": "50Gi", + "mountPath": "/var/lib/postgresql/data", + "storageClass": "fast" + } + ] + }, + "Expose": { + "type": "object", + "title": "Expose", + "description": "Exposes the component via an ingress rule resolved against the platform domain configuration. All fields are optional: the domain defaults to the environment's only (or default-marked) domain, and the subdomain defaults to the component name.", + "additionalProperties": false, + "properties": { + "domain": { + "type": "string", + "title": "Domain Key", + "description": "Domain key referencing an entry in the platform environment's domains map. When absent, the environment's only domain is used, or the one marked 'default: true' in the platform file.", + "pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$", + "minLength": 1, + "examples": [ + "public", + "internal" + ] + }, + "subdomain": { + "type": "string", + "title": "Subdomain", + "description": "DNS label prepended to baseDomain to form the FQDN. When absent the component name is used. Must be a valid DNS-1123 label. Mutually exclusive with apex.", + "pattern": "^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$", + "minLength": 1, + "examples": [ + "api", + "www" + ] + }, + "apex": { + "type": "boolean", + "title": "Apex", + "description": "Expose the component at the baseDomain itself instead of a subdomain. Mutually exclusive with subdomain.", + "default": false, + "examples": [ + true + ] + } + }, + "examples": [ + { + "subdomain": "api" + }, + { + "domain": "internal" + }, + { + "apex": true + } + ] + }, + "Autoscaling": { + "type": "object", + "title": "Autoscaling", + "description": "Horizontal pod autoscaling configuration.", + "additionalProperties": false, + "required": [ + "enabled", + "minReplicas", + "maxReplicas" + ], + "properties": { + "enabled": { + "type": "boolean", + "description": "Whether autoscaling is enabled.", + "default": false + }, + "minReplicas": { + "type": "integer", + "description": "Minimum number of replicas.", + "default": 2, + "minimum": 1 + }, + "maxReplicas": { + "type": "integer", + "description": "Maximum number of replicas.", + "default": 5, + "minimum": 1 + }, + "metrics": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "type", + "target" + ], + "properties": { + "type": { + "type": "string", + "enum": [ + "cpu", + "memory" + ] + }, + "target": { + "type": "integer", + "minimum": 1 + } + } + }, + "default": [ + { + "type": "cpu", + "target": 75 + } + ], + "examples": [ + [ + { + "type": "cpu", + "target": 70 + }, + { + "type": "memory", + "target": 80 + } + ] + ] + } + }, + "examples": [ + { + "enabled": true, + "minReplicas": 2, + "maxReplicas": 6, + "metrics": [ + { + "type": "cpu", + "target": 70 + } + ] + }, + { + "enabled": true, + "minReplicas": 3, + "maxReplicas": 10, + "metrics": [ + { + "type": "cpu", + "target": 65 + }, + { + "type": "memory", + "target": 75 + } + ] + } + ] + }, + "Resources": { + "type": "object", + "title": "Resources", + "description": "CPU and memory requests/limits.", + "additionalProperties": false, + "properties": { + "cpu": { + "type": "string", + "pattern": "^([1-9][0-9]*m?|[1-9][0-9]*|0m)$", + "examples": [ + "500m", + "1" + ] + }, + "memory": { + "type": "string", + "pattern": "^[1-9][0-9]*(Ki|Mi|Gi|Ti|Pi|Ei)$", + "examples": [ + "256Mi", + "1Gi" + ] + }, + "ephemeralStorage": { + "type": "string", + "pattern": "^[1-9][0-9]*(Ki|Mi|Gi|Ti|Pi|Ei)$", + "examples": [ + "500Mi", + "2Gi" + ] + } + }, + "examples": [ + { + "cpu": "500m", + "memory": "512Mi" + }, + { + "ephemeralStorage": "2Gi" + } + ] + }, + "Metrics": { + "type": "object", + "title": "Metrics Configuration", + "description": "Prometheus scrape settings. enabled defaults to true when omitted. port is required for workers; for services it defaults to the component port.", + "additionalProperties": false, + "properties": { + "enabled": { + "type": "boolean", + "title": "Enabled", + "description": "When false, disables scraping (useful for per-environment toggle via envsubst). Defaults to true when omitted.", + "default": true + }, + "port": { + "type": "integer", + "title": "Metrics Port", + "description": "Container port exposing metrics. Required for workers. Defaults to the component port for services.", + "minimum": 1, + "maximum": 65535, + "examples": [ + 8080, + 9090 + ] + }, + "path": { + "type": "string", + "title": "Metrics Path", + "description": "HTTP path for metrics. Defaults to /metrics.", + "pattern": "^/", + "default": "/metrics", + "examples": [ + "/metrics", + "/actuator/prometheus" + ] + }, + "interval": { + "type": "string", + "title": "Scrape Interval", + "description": "Override the platform profile scrape interval.", + "pattern": "^[1-9][0-9]*(s|m|h)$", + "examples": [ + "15s", + "30s", + "1m" + ] + }, + "scrapeTimeout": { + "type": "string", + "title": "Scrape Timeout", + "description": "Override the platform profile scrape timeout.", + "pattern": "^[1-9][0-9]*(s|m|h)$", + "examples": [ + "5s", + "10s" + ] + } + }, + "examples": [ + { + "port": 9090 + }, + { + "port": 8080, + "path": "/metrics", + "interval": "30s" + }, + { + "enabled": false + } + ] + }, + "Health": { + "type": "object", + "title": "Health Checks", + "description": "Health check configuration. Services support TCP/HTTP ready and alive checks. Workers support optional alive.exec only.", + "additionalProperties": false, + "properties": { + "ready": { + "oneOf": [ + { + "type": "boolean", + "const": false + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "path" + ], + "properties": { + "path": { + "type": "string", + "pattern": "^/", + "examples": [ + "/health", + "/ready", + "/healthz" + ] + } + } + } + ], + "examples": [ + { + "path": "/health" + }, + false + ] + }, + "alive": { + "oneOf": [ + { + "type": "boolean", + "const": false + }, + { + "type": "object", + "additionalProperties": false, + "properties": { + "path": { + "type": "string", + "pattern": "^/", + "examples": [ + "/livez", + "/healthz", + "/alive" + ] + }, + "exec": { + "type": "array", + "title": "Exec Command", + "description": "Command run inside the container; exit 0 means alive. Mutually exclusive with path. Available for service and worker roles.", + "minItems": 1, + "items": { + "type": "string" + }, + "examples": [ + [ + "pgrep", + "-f", + "worker" + ], + [ + "sh", + "-c", + "test -f /tmp/healthy" + ] + ] + }, + "interval": { + "type": "string", + "pattern": "^[1-9][0-9]*(s|m|h)$", + "default": "10s", + "examples": [ + "10s", + "30s", + "1m" + ] + }, + "restartAfter": { + "type": "string", + "pattern": "^[1-9][0-9]*(s|m|h)$", + "default": "60s", + "examples": [ + "60s", + "2m", + "5m" + ] + } + }, + "oneOf": [ + { + "required": [ + "path" + ] + }, + { + "required": [ + "exec" + ] + } + ] + } + ], + "examples": [ + { + "path": "/livez" + }, + { + "path": "/livez", + "interval": "10s", + "restartAfter": "60s" + }, + { + "exec": [ + "pgrep", + "-f", + "worker" + ] + }, + false + ] + } + }, + "examples": [ + { + "ready": { + "path": "/health" + } + }, + { + "ready": { + "path": "/health" + }, + "alive": { + "path": "/livez" + } + }, + { + "alive": { + "exec": [ + "pgrep", + "-f", + "worker" + ] + } + }, + { + "ready": false, + "alive": false + } + ] + }, + "Task": { + "type": "object", + "title": "Task", + "description": "Run-to-completion work triggered on deploy or via the CLI.", + "additionalProperties": false, + "required": [ + "on" + ], + "anyOf": [ + { + "required": [ + "from" + ] + }, + { + "required": [ + "image" + ] + } + ], + "properties": { + "from": { + "type": "string", + "title": "From", + "description": "Component name to inherit env, envFile, configFile, environments, profiles, and resources from. Does not copy command, args, or service-only fields.", + "minLength": 1, + "examples": [ + "api" + ] + }, + "image": { + "type": "string", + "title": "Container Image", + "description": "Container image. When omitted, the image from from is used. When set, it replaces the parent image.", + "minLength": 1 + }, + "command": { + "type": "array", + "title": "Command", + "description": "Container entrypoint override. Required when using the parent image.", + "items": { + "type": "string" + } + }, + "args": { + "type": "array", + "title": "Arguments", + "description": "Container command argument override.", + "items": { + "type": "string" + } + }, + "on": { + "type": "string", + "title": "Trigger", + "description": "When the task runs. preDeploy and postDeploy run on every install and upgrade. manual runs only via the CLI.", + "enum": [ + "preDeploy", + "postDeploy", + "manual" + ] + }, + "after": { + "type": "array", + "title": "After", + "description": "Task names that must finish first in the same on phase. Not allowed on manual tasks.", + "items": { + "type": "string", + "minLength": 1 + } + }, + "env": { + "type": "object", + "title": "Environment Variables", + "description": "Static environment variables. Overlay the inherited map from from.", + "propertyNames": { + "pattern": "^[A-Z_][A-Z0-9_]*$" + }, + "additionalProperties": { + "type": [ + "string", + "number", + "boolean" + ] + } + }, + "envFile": { + "type": "string", + "title": "Environment File" + }, + "configFile": { + "type": "string", + "title": "Configuration File" + }, + "environments": { + "type": "array", + "title": "Environment Filter", + "minItems": 1, + "items": { + "type": "string" + } + }, + "profiles": { + "type": "array", + "title": "Profiles", + "items": { + "type": "string", + "minLength": 1 + } + }, + "resources": { + "$ref": "#/$defs/Resources" + }, + "resourcePreset": { + "type": "string", + "title": "Resource Preset", + "enum": [ + "nano", + "micro", + "small", + "medium", + "large", + "xlarge", + "2xlarge" + ] + }, + "fanout": { + "title": "Fanout", + "description": "How many indexed copies to run. Integer shorthand (count, parallelism 1) or an object. Applies to preDeploy, postDeploy, and manual.", + "oneOf": [ + { + "type": "integer", + "minimum": 1 + }, + { + "$ref": "#/$defs/Fanout" + } + ] + }, + "timeout": { + "type": "string", + "title": "Timeout", + "description": "How long a single run may take. Defaults to 5m for preDeploy and postDeploy. Hook timeout must be less than the CLI --timeout used at deploy or run time (default 10m).", + "pattern": "^[1-9][0-9]*(s|m|h)$" + }, + "backoffLimit": { + "type": "integer", + "title": "Backoff Limit", + "description": "Retries before the run is marked failed. Defaults to 3.", + "minimum": 0, + "default": 3 + }, + "ttlSecondsAfterFinished": { + "type": "integer", + "title": "TTL Seconds After Finished", + "minimum": 0 + } + } + }, + "Fanout": { + "type": "object", + "title": "Fanout", + "description": "Indexed copy count and how many may run at once.", + "additionalProperties": false, + "properties": { + "count": { + "type": "integer", + "title": "Count", + "minimum": 1 + }, + "parallelism": { + "type": "integer", + "title": "Parallelism", + "description": "How many copies may run at once. Must not exceed count. Kubernetes Indexed Jobs reject values above 100000.", + "minimum": 1, + "maximum": 100000, + "default": 1 + } + } + } + }, + "examples": [ + { + "apiVersion": "v1-alpha.5", + "project": "shop", + "environments": { + "production": { + "envFile": ".env.production" + }, + "local": {} + }, + "components": { + "api": { + "image": "my-app:latest", + "port": 8080, + "expose": { + "domain": "public", + "subdomain": "api" + }, + "metrics": true + }, + "worker": { + "image": "my-worker:latest", + "role": "worker", + "command": [ + "./worker" + ], + "metrics": { + "port": 9090 + } + } + }, + "tasks": { + "migrate": { + "from": "api", + "on": "preDeploy", + "command": [ + "migrate", + "up" + ] + } + } + } + ] +} diff --git a/internal/spec/task.go b/internal/spec/task.go new file mode 100644 index 0000000..687ad45 --- /dev/null +++ b/internal/spec/task.go @@ -0,0 +1,321 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package spec + +import ( + "encoding/json" + "fmt" + "maps" + "slices" +) + +// TaskOn is when a [Task] runs. +type TaskOn string + +const ( + // TaskOnPreDeploy runs the task before other resources on install and + // upgrade. + TaskOnPreDeploy TaskOn = "preDeploy" + // TaskOnPostDeploy runs the task after the app is ready on install and + // upgrade. + TaskOnPostDeploy TaskOn = "postDeploy" + // TaskOnManual runs the task only via the CLI. + TaskOnManual TaskOn = "manual" +) + +// IsHook reports whether o is a deploy hook (preDeploy or postDeploy). +func (o TaskOn) IsHook() bool { + return o == TaskOnPreDeploy || o == TaskOnPostDeploy +} + +// Task is run-to-completion work in a spec. +type Task struct { + // From names a component whose env, envFile, configFile, environments, + // profiles, and resources are copied. Command, args, and service-only + // fields are not copied. + From string `json:"from,omitempty" yaml:"from,omitempty"` + // Image is the container image. When empty, the image from From is + // used. When set, it replaces the parent image. + Image string `json:"image,omitempty" yaml:"image,omitempty"` + // Command overrides the container entrypoint. Required when using the + // parent image. + Command []string `json:"command,omitempty" yaml:"command,omitempty"` + // Args overrides the container command arguments. + Args []string `json:"args,omitempty" yaml:"args,omitempty"` + // On selects when the task runs. Required. + On TaskOn `json:"on" yaml:"on"` + // After lists task names that must finish first in the same On phase. + // Not allowed when On is manual. + After []string `json:"after,omitempty" yaml:"after,omitempty"` + // Env overlays inherited environment variables. + Env map[string]string `json:"env,omitempty" yaml:"env,omitempty"` + // EnvFile is a task-specific dotenv path. Replaces the inherited path + // when set. + EnvFile string `json:"envFile,omitempty" yaml:"envFile,omitempty"` + // ConfigFile is a task-specific config path. Replaces the inherited + // path when set. + ConfigFile string `json:"configFile,omitempty" yaml:"configFile,omitempty"` + // Environments limits the task to the named environments. Replaces the + // inherited filter when set. + Environments []string `json:"environments,omitempty" yaml:"environments,omitempty"` + // Profiles lists platform profile names. Replaces the inherited list + // when set. + Profiles []string `json:"profiles,omitempty" yaml:"profiles,omitempty"` + // Resources sets explicit CPU, memory, and storage requests. + Resources Resources `json:"resources" yaml:"resources,omitempty"` + // ResourcePreset selects a named resource profile when Resources is + // empty. + ResourcePreset ResourcePreset `json:"resourcePreset,omitempty" yaml:"resourcePreset,omitempty"` + // Fanout sets how many indexed copies to run. Omit for one copy. + Fanout Fanout `json:"fanout,omitzero" yaml:"fanout,omitempty"` + // Timeout is how long a single run may take (for example "5m"). + Timeout string `json:"timeout,omitempty" yaml:"timeout,omitempty"` + // BackoffLimit is how many retries are allowed before the run fails. + // Nil means [DefaultBackoffLimit]. + BackoffLimit *int `json:"backoffLimit,omitempty" yaml:"backoffLimit,omitempty"` + // TTLSecondsAfterFinished is seconds to keep a finished run. + TTLSecondsAfterFinished *int `json:"ttlSecondsAfterFinished,omitempty" yaml:"ttlSecondsAfterFinished,omitempty"` +} + +// Fanout unmarshals a YAML/JSON integer (count, parallelism 1) or +// {count, parallelism}. +type Fanout struct { + // Count is how many indexed copies to run. + Count int `json:"count,omitempty" yaml:"count,omitempty"` + // Parallelism is how many copies may run at once. Must not exceed + // Count or [MaxFanoutParallelism]. + Parallelism int `json:"parallelism,omitempty" yaml:"parallelism,omitempty"` +} + +// UnmarshalJSON handles integer and object forms: +// +// fanout: 4 +// fanout: {count: 4, parallelism: 2} +// +// A JSON null is treated as the omitted default. +func (f *Fanout) UnmarshalJSON(data []byte) error { + if string(data) == "null" { + *f = Fanout{} + return nil + } + var n int + if err := json.Unmarshal(data, &n); err == nil { + if n < 1 { + return fmt.Errorf("fanout: count must be at least 1") + } + *f = Fanout{Count: n, Parallelism: DefaultFanoutParallelism} + return nil + } + type fanoutAlias Fanout + var alias fanoutAlias + if err := json.Unmarshal(data, &alias); err != nil { + return fmt.Errorf("fanout: expected an integer or an object with count: %w", err) + } + *f = Fanout(alias) + return nil +} + +// MarshalJSON emits an integer when parallelism is 0 or 1, otherwise an +// object. The zero value is omitted by the [Task.Fanout] omitzero tag. +func (f Fanout) MarshalJSON() ([]byte, error) { + if f.Parallelism <= 1 { + return json.Marshal(f.Count) + } + type fanoutAlias Fanout + return json.Marshal(fanoutAlias(f)) +} + +// EffectiveCount returns Count, or [DefaultFanoutCount] when Count is 0. +func (f Fanout) EffectiveCount() int { + if f.Count <= 0 { + return DefaultFanoutCount + } + return f.Count +} + +// EffectiveParallelism returns Parallelism, or [DefaultFanoutParallelism] +// when Parallelism is 0. +func (f Fanout) EffectiveParallelism() int { + if f.Parallelism <= 0 { + return DefaultFanoutParallelism + } + return f.Parallelism +} + +// EffectiveImage returns the task image, or parent.Image when the task +// image is empty. +func (t Task) EffectiveImage(parent *Component) string { + if t.Image != "" { + return t.Image + } + if parent != nil { + return parent.Image + } + return "" +} + +// UsesParentImage reports whether the task runs the parent component image. +func (t Task) UsesParentImage() bool { + return t.Image == "" && t.From != "" +} + +// MergeFrom copies inheritable fields from parent when the task left them +// empty. Task env overlays parent env. Command, args, and service-only +// fields are never copied. parent may be nil when From is empty. +func (t Task) MergeFrom(parent *Component) Task { + if parent == nil { + return t + } + out := t + if out.Image == "" { + out.Image = parent.Image + } + if out.EnvFile == "" { + out.EnvFile = parent.EnvFile + } + if out.ConfigFile == "" { + out.ConfigFile = parent.ConfigFile + } + if len(out.Environments) == 0 { + out.Environments = slices.Clone(parent.Environments) + } + if len(out.Profiles) == 0 { + out.Profiles = slices.Clone(parent.Profiles) + } + if out.ResourcePreset == "" && !out.Resources.ResourcesSet() { + out.ResourcePreset = parent.ResourcePreset + if parent.Resources.ResourcesSet() { + out.Resources = Resources{ + CPU: cloneQuantity(parent.Resources.CPU), + Memory: cloneQuantity(parent.Resources.Memory), + EphemeralStorage: cloneQuantity(parent.Resources.EphemeralStorage), + } + } + } + // Always build a fresh map so the caller cannot reach back into the + // parent component or the task through the merged result. + if len(parent.Env) > 0 || len(out.Env) > 0 { + merged := make(map[string]string, len(parent.Env)+len(out.Env)) + maps.Copy(merged, parent.Env) + maps.Copy(merged, out.Env) + out.Env = merged + } + return out +} + +// HelmHookEvents returns the Helm hook event list for t.On, or empty for +// manual tasks. +func (t Task) HelmHookEvents() string { + switch t.On { + case TaskOnPreDeploy: + return "pre-install,pre-upgrade" + case TaskOnPostDeploy: + return "post-install,post-upgrade" + default: + return "" + } +} + +// TaskNames returns the sorted list of task names. Returns nil when no +// tasks are defined. +func (m *Spec) TaskNames() []string { + if m == nil || len(m.Tasks) == 0 { + return nil + } + keys := make([]string, 0, len(m.Tasks)) + for k := range m.Tasks { + keys = append(keys, k) + } + slices.Sort(keys) + return keys +} + +// MergedTask returns the named task after [Task.MergeFrom] with its +// parent component. The boolean is false when the name is unknown. +func (m *Spec) MergedTask(name string) (Task, bool) { + if m == nil { + return Task{}, false + } + task, ok := m.Tasks[name] + if !ok { + return Task{}, false + } + return task.MergeFrom(m.componentRef(task.From)), true +} + +// tasksInEnvironment returns merged tasks that apply to env. An empty +// environments list on a task means every environment. +func (m *Spec) tasksInEnvironment(env string) map[string]Task { + names := m.TaskNames() + out := make(map[string]Task, len(names)) + for _, name := range names { + task, ok := m.MergedTask(name) + if !ok || !task.activeInEnvironment(env) { + continue + } + out[name] = task + } + return out +} + +// EffectiveTasks returns the tasks that apply to environment. +// When resolved is non-nil, it returns a new map of resolved.Tasks, which +// resolution already filtered to that environment. The map is safe to add +// to or delete from; the [ResolvedTask] values still alias the resolved +// spec (env, profiles, and merged profile). When resolved is nil, it +// returns merged tasks from manifest that apply to environment, with hook +// weights from [AssignHookWeights]. +func EffectiveTasks(manifest *Spec, environment string, resolved *ResolvedSpec) (map[string]ResolvedTask, error) { + if resolved != nil { + out := make(map[string]ResolvedTask, len(resolved.Tasks)) + maps.Copy(out, resolved.Tasks) + return out, nil + } + if manifest == nil { + return map[string]ResolvedTask{}, nil + } + weights, err := AssignHookWeights(manifest.Tasks) + if err != nil { + return nil, err + } + envTasks := manifest.tasksInEnvironment(environment) + out := make(map[string]ResolvedTask, len(envTasks)) + for name, task := range envTasks { + out[name] = ResolvedTask{Task: task, HookWeight: weights[name]} + } + return out, nil +} + +func (t Task) activeInEnvironment(env string) bool { + if len(t.Environments) == 0 { + return true + } + _, ok := matchEnvKey(env, t.Environments) + return ok +} + +func (m *Spec) componentRef(name string) *Component { + if m == nil || name == "" { + return nil + } + parent, ok := m.Components[name] + if !ok { + return nil + } + cp := parent + return &cp +} + +// scheduleOnToken is rejected with a pointer at issue #35. +const scheduleOnToken = "schedule" diff --git a/internal/spec/task_after.go b/internal/spec/task_after.go new file mode 100644 index 0000000..6644f66 --- /dev/null +++ b/internal/spec/task_after.go @@ -0,0 +1,116 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package spec + +import ( + "fmt" + "maps" + "slices" +) + +// AssignHookWeights returns Helm hook-weight per task name for hook tasks. +// Manual tasks are omitted. Independent tasks (no after) share weight 0; +// Helm then runs them in name order. A task's weight is one greater than +// the maximum weight of its after dependencies. Cycles return an error. +func AssignHookWeights(tasks map[string]Task) (map[string]int, error) { + weights := make(map[string]int) + for _, on := range []TaskOn{TaskOnPreDeploy, TaskOnPostDeploy} { + phase, err := hookWeightsForPhase(tasks, on) + if err != nil { + return nil, err + } + maps.Copy(weights, phase) + } + return weights, nil +} + +func hookWeightsForPhase(tasks map[string]Task, on TaskOn) (map[string]int, error) { + names := make([]string, 0) + for name, task := range tasks { + if task.On == on { + names = append(names, name) + } + } + slices.Sort(names) + if len(names) == 0 { + return map[string]int{}, nil + } + + inPhase := make(map[string]struct{}, len(names)) + for _, n := range names { + inPhase[n] = struct{}{} + } + + indegree := make(map[string]int, len(names)) + dependents := make(map[string][]string, len(names)) + for _, n := range names { + indegree[n] = 0 + } + for _, n := range names { + for _, dep := range tasks[n].After { + if _, ok := inPhase[dep]; !ok { + continue + } + dependents[dep] = append(dependents[dep], n) + indegree[n]++ + } + } + + weights := make(map[string]int, len(names)) + ready := make([]string, 0) + for _, n := range names { + if indegree[n] == 0 { + ready = append(ready, n) + } + } + slices.Sort(ready) + + seen := 0 + for len(ready) > 0 { + n := ready[0] + ready = ready[1:] + seen++ + maxDep := -1 + for _, dep := range tasks[n].After { + if _, ok := inPhase[dep]; !ok { + continue + } + if w, ok := weights[dep]; ok && w > maxDep { + maxDep = w + } + } + weights[n] = maxDep + 1 + next := make([]string, 0) + for _, child := range dependents[n] { + indegree[child]-- + if indegree[child] == 0 { + next = append(next, child) + } + } + slices.Sort(next) + ready = append(ready, next...) + } + if seen != len(names) { + // Whatever still has an unmet dependency is the cycle plus the + // tasks waiting behind it. names is sorted, so stuck is too. + stuck := make([]string, 0, len(names)-seen) + for _, n := range names { + if indegree[n] > 0 { + stuck = append(stuck, n) + } + } + return nil, fmt.Errorf("tasks with on %s: after contains a cycle among %s", on, joinStrings(stuck)) + } + return weights, nil +} diff --git a/internal/spec/task_after_test.go b/internal/spec/task_after_test.go new file mode 100644 index 0000000..64239d1 --- /dev/null +++ b/internal/spec/task_after_test.go @@ -0,0 +1,77 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package spec + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestAssignHookWeights_IndependentShareZero(t *testing.T) { + t.Parallel() + + weights, err := AssignHookWeights(map[string]Task{ + "migrate": {On: TaskOnPreDeploy}, + "seed": {On: TaskOnPreDeploy}, + "smoke": {On: TaskOnPostDeploy}, + "backfill": {On: TaskOnManual}, + }) + require.NoError(t, err) + assert.Equal(t, 0, weights["migrate"]) + assert.Equal(t, 0, weights["seed"]) + assert.Equal(t, 0, weights["smoke"]) + _, hasManual := weights["backfill"] + assert.False(t, hasManual) +} + +func TestAssignHookWeights_ChainAndNameOrder(t *testing.T) { + t.Parallel() + + weights, err := AssignHookWeights(map[string]Task{ + "seed": {On: TaskOnPreDeploy, After: []string{"migrate"}}, + "migrate": {On: TaskOnPreDeploy}, + "grant": {On: TaskOnPreDeploy, After: []string{"seed"}}, + }) + require.NoError(t, err) + assert.Equal(t, 0, weights["migrate"]) + assert.Equal(t, 1, weights["seed"]) + assert.Equal(t, 2, weights["grant"]) +} + +func TestAssignHookWeights_Cycle(t *testing.T) { + t.Parallel() + + _, err := AssignHookWeights(map[string]Task{ + "a": {On: TaskOnPreDeploy, After: []string{"b"}}, + "b": {On: TaskOnPreDeploy, After: []string{"a"}}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "cycle") + assert.Contains(t, err.Error(), `"a", "b"`) +} + +func TestAssignHookWeights_CycleNamesOnlyStuckTasks(t *testing.T) { + t.Parallel() + + _, err := AssignHookWeights(map[string]Task{ + "migrate": {On: TaskOnPreDeploy}, + "a": {On: TaskOnPreDeploy, After: []string{"b"}}, + "b": {On: TaskOnPreDeploy, After: []string{"a"}}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), `"a", "b"`) + assert.NotContains(t, err.Error(), "migrate") +} diff --git a/internal/spec/task_job.go b/internal/spec/task_job.go new file mode 100644 index 0000000..a680744 --- /dev/null +++ b/internal/spec/task_job.go @@ -0,0 +1,111 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/License-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package spec + +import ( + "fmt" + "maps" + "math" + "slices" +) + +// TaskJobSpec is the resolved Job field set used by the CLI builder and Helm +// chart values. Completions and Parallelism are already resolved from fanout +// or caller overrides. TTLSecondsAfterFinished is nil unless the spec set +// it; the CLI applies [DefaultCLIJobTTLSeconds] itself. +type TaskJobSpec struct { + Completions int32 + Parallelism int32 + BackoffLimit int32 + ActiveDeadlineSeconds *int64 + TTLSecondsAfterFinished *int32 + Image string + Command []string + Args []string + Env map[string]string + Resources Resources +} + +// NewTaskJobSpec builds the shared Job fields from task. count and +// parallelism override fanout when greater than 0; a value of 0 means use +// the task's fanout defaults. Parallelism is rejected above +// [MaxFanoutParallelism], then clamped to count. It returns a wrapped +// error when [Task.Timeout] is not a valid duration, when parallelism is +// above [MaxFanoutParallelism], or when count, backoffLimit, or +// ttlSecondsAfterFinished do not fit in int32. +func NewTaskJobSpec(task Task, count, parallelism int) (TaskJobSpec, error) { + if count < 1 { + count = task.Fanout.EffectiveCount() + } + if parallelism < 1 { + parallelism = task.Fanout.EffectiveParallelism() + } + if parallelism > MaxFanoutParallelism { + return TaskJobSpec{}, fmt.Errorf("fanout.parallelism must be at most %d", MaxFanoutParallelism) + } + if parallelism > count { + parallelism = count + } + + backoff := DefaultBackoffLimit + if task.BackoffLimit != nil { + backoff = *task.BackoffLimit + } + + completions32, err := toInt32("fanout.count", count) + if err != nil { + return TaskJobSpec{}, err + } + backoff32, err := toInt32("backoffLimit", backoff) + if err != nil { + return TaskJobSpec{}, err + } + + out := TaskJobSpec{ + Completions: completions32, + Parallelism: int32(parallelism), //nolint:gosec // bounded by MaxFanoutParallelism + BackoffLimit: backoff32, + Image: task.Image, + Command: slices.Clone(task.Command), + Args: slices.Clone(task.Args), + Env: maps.Clone(task.Env), + Resources: Resources{ + CPU: cloneQuantity(task.Resources.CPU), + Memory: cloneQuantity(task.Resources.Memory), + EphemeralStorage: cloneQuantity(task.Resources.EphemeralStorage), + }, + } + if task.TTLSecondsAfterFinished != nil { + ttl32, ttlErr := toInt32("ttlSecondsAfterFinished", *task.TTLSecondsAfterFinished) + if ttlErr != nil { + return TaskJobSpec{}, ttlErr + } + out.TTLSecondsAfterFinished = &ttl32 + } + if task.Timeout != "" { + sec, timeoutErr := ParseDuration(task.Timeout) + if timeoutErr != nil { + return TaskJobSpec{}, fmt.Errorf("timeout: %w", timeoutErr) + } + out.ActiveDeadlineSeconds = new(int64(sec)) + } + return out, nil +} + +func toInt32(name string, n int) (int32, error) { + if n < 0 || n > math.MaxInt32 { + return 0, fmt.Errorf("%s %d is outside the int32 range", name, n) + } + return int32(n), nil +} diff --git a/internal/spec/task_job_test.go b/internal/spec/task_job_test.go new file mode 100644 index 0000000..3b68ab4 --- /dev/null +++ b/internal/spec/task_job_test.go @@ -0,0 +1,143 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package spec + +import ( + "math" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestNewTaskJobSpec(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + task Task + count int + parallelism int + want TaskJobSpec + }{ + { + name: "fanout defaults and timeout", + task: Task{ + Image: "busybox:1.36", + Command: []string{"migrate", "up"}, + Env: map[string]string{"LOG": "debug"}, + Timeout: "5m", + Fanout: Fanout{Count: 4, Parallelism: 2}, + }, + want: TaskJobSpec{ + Completions: 4, + Parallelism: 2, + BackoffLimit: int32(DefaultBackoffLimit), + ActiveDeadlineSeconds: new(int64(300)), + Image: "busybox:1.36", + Command: []string{"migrate", "up"}, + Env: map[string]string{"LOG": "debug"}, + }, + }, + { + name: "count and parallelism overrides clamp", + task: Task{Image: "busybox:1.36", Fanout: Fanout{Count: 1, Parallelism: 1}}, + count: 3, + parallelism: 8, + want: TaskJobSpec{ + Completions: 3, + Parallelism: 3, + BackoffLimit: int32(DefaultBackoffLimit), + Image: "busybox:1.36", + }, + }, + { + name: "explicit backoff and ttl", + task: Task{ + Image: "busybox:1.36", + BackoffLimit: new(5), + TTLSecondsAfterFinished: new(60), + }, + want: TaskJobSpec{ + Completions: 1, + Parallelism: 1, + BackoffLimit: 5, + TTLSecondsAfterFinished: new(int32(60)), + Image: "busybox:1.36", + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got, err := NewTaskJobSpec(tt.task, tt.count, tt.parallelism) + require.NoError(t, err) + assert.Equal(t, tt.want, got) + }) + } +} + +func TestNewTaskJobSpec_Error(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + task Task + count int + parallelism int + wantErr string + }{ + { + name: "invalid timeout", + task: Task{Image: "busybox:1.36", Timeout: "nope"}, + wantErr: "timeout", + }, + { + name: "parallelism exceeds indexed job limit", + task: Task{Image: "busybox:1.36"}, + count: MaxFanoutParallelism + 1, + parallelism: MaxFanoutParallelism + 1, + wantErr: "fanout.parallelism", + }, + { + name: "count exceeds int32", + task: Task{Image: "busybox:1.36"}, + count: math.MaxInt32 + 1, + wantErr: "int32 range", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + _, err := NewTaskJobSpec(tt.task, tt.count, tt.parallelism) + require.Error(t, err) + assert.Contains(t, err.Error(), tt.wantErr) + }) + } +} + +func TestNewTaskJobSpec_ClonesResources(t *testing.T) { + t.Parallel() + + cpu := MustQuantity("100m") + got, err := NewTaskJobSpec(Task{ + Image: "busybox:1.36", + Resources: Resources{CPU: cpu}, + }, 0, 0) + require.NoError(t, err) + require.NotNil(t, got.Resources.CPU) + assert.NotSame(t, cpu, got.Resources.CPU) + got.Resources.CPU.Add(*MustQuantity("1")) + assert.True(t, cpu.Equal(*MustQuantity("100m"))) +} diff --git a/internal/spec/task_test.go b/internal/spec/task_test.go new file mode 100644 index 0000000..65434bc --- /dev/null +++ b/internal/spec/task_test.go @@ -0,0 +1,985 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package spec + +import ( + "math" + "slices" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "sigs.k8s.io/yaml" +) + +func shopSpec(tasks map[string]Task) *Spec { + return &Spec{ + APIVersion: CurrentManifestVersion, + Project: "shop", + Components: map[string]Component{ + "api": { + Role: ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + Env: map[string]string{"DATABASE_URL": "postgres://db", "LOG": "info"}, + }, + }, + Tasks: tasks, + } +} + +func TestFanout_UnmarshalJSON(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + yaml string + want Fanout + wantErr string + }{ + { + name: "scalar integer sets count and parallelism 1", + yaml: "fanout: 4\n", + want: Fanout{Count: 4, Parallelism: DefaultFanoutParallelism}, + }, + { + name: "object form", + yaml: "fanout:\n count: 4\n parallelism: 2\n", + want: Fanout{Count: 4, Parallelism: 2}, + }, + { + name: "object omits parallelism", + yaml: "fanout:\n count: 3\n", + want: Fanout{Count: 3, Parallelism: 0}, + }, + { + name: "null is the omitted default", + yaml: "fanout: null\n", + want: Fanout{}, + }, + { + name: "zero scalar is rejected", + yaml: "fanout: 0\n", + wantErr: "at least 1", + }, + { + name: "string is rejected", + yaml: "fanout: lots\n", + wantErr: "integer or an object", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + var task Task + err := yaml.Unmarshal([]byte(tt.yaml), &task) + if tt.wantErr != "" { + require.Error(t, err) + assert.Contains(t, err.Error(), tt.wantErr) + return + } + require.NoError(t, err) + assert.Equal(t, tt.want, task.Fanout) + }) + } +} + +func TestFanout_MarshalJSON(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + in Fanout + want string + }{ + {name: "count only emits integer", in: Fanout{Count: 4, Parallelism: 1}, want: "4"}, + {name: "parallelism 0 emits integer", in: Fanout{Count: 4}, want: "4"}, + {name: "parallelism above 1 emits object", in: Fanout{Count: 4, Parallelism: 2}, want: `{"count":4,"parallelism":2}`}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got, err := tt.in.MarshalJSON() + require.NoError(t, err) + assert.JSONEq(t, tt.want, string(got)) + }) + } +} + +func TestTask_FanoutRoundTrip(t *testing.T) { + t.Parallel() + + t.Run("omitted fanout stays omitted", func(t *testing.T) { + t.Parallel() + in := Task{On: TaskOnManual, Image: "busybox:1.36"} + data, err := yaml.Marshal(in) + require.NoError(t, err) + assert.NotContains(t, string(data), "fanout") + var out Task + require.NoError(t, yaml.Unmarshal(data, &out)) + assert.Equal(t, Fanout{}, out.Fanout) + }) + + t.Run("scalar fanout round-trips", func(t *testing.T) { + t.Parallel() + in := Task{On: TaskOnManual, Image: "busybox:1.36", Fanout: Fanout{Count: 4, Parallelism: 1}} + data, err := yaml.Marshal(in) + require.NoError(t, err) + var out Task + require.NoError(t, yaml.Unmarshal(data, &out)) + assert.Equal(t, Fanout{Count: 4, Parallelism: DefaultFanoutParallelism}, out.Fanout) + }) +} + +func TestTask_MergeFrom(t *testing.T) { + t.Parallel() + + parent := Component{ + Image: "ghcr.io/acme/shop:1.2.3", + Env: map[string]string{"DATABASE_URL": "postgres://db", "LOG": "info"}, + EnvFile: ".env", + ConfigFile: "config.yaml", + Environments: []string{"prod"}, + Profiles: []string{"default"}, + ResourcePreset: ResourcePresetSmall, + Command: []string{"api"}, + Args: []string{"--serve"}, + Port: 8080, + } + + t.Run("copies inheritable fields and overlays env", func(t *testing.T) { + t.Parallel() + task := Task{ + From: "api", + On: TaskOnPreDeploy, + Command: []string{"migrate", "up"}, + Env: map[string]string{"LOG": "debug", "EXTRA": "1"}, + } + got := task.MergeFrom(&parent) + assert.Equal(t, parent.Image, got.Image) + assert.Equal(t, []string{"migrate", "up"}, got.Command) + assert.Nil(t, got.Args) + assert.Equal(t, map[string]string{ + "DATABASE_URL": "postgres://db", + "LOG": "debug", + "EXTRA": "1", + }, got.Env) + assert.Equal(t, parent.EnvFile, got.EnvFile) + assert.Equal(t, parent.ConfigFile, got.ConfigFile) + assert.Equal(t, parent.Environments, got.Environments) + assert.Equal(t, parent.Profiles, got.Profiles) + assert.Equal(t, parent.ResourcePreset, got.ResourcePreset) + }) + + t.Run("task image replaces parent image", func(t *testing.T) { + t.Parallel() + task := Task{From: "api", Image: "busybox:1.36", On: TaskOnManual} + got := task.MergeFrom(&parent) + assert.Equal(t, "busybox:1.36", got.Image) + }) + + t.Run("nil parent leaves the task unchanged", func(t *testing.T) { + t.Parallel() + task := Task{Image: "busybox:1.36", On: TaskOnManual, Command: []string{"true"}} + assert.Equal(t, task, task.MergeFrom(nil)) + }) + + t.Run("merged env does not alias the task or the parent", func(t *testing.T) { + t.Parallel() + taskEnv := map[string]string{"EXTRA": "1"} + parentEnv := map[string]string{"LOG": "info"} + got := Task{From: "api", On: TaskOnManual, Env: taskEnv}.MergeFrom(&Component{ + Image: "ghcr.io/acme/shop:1.2.3", + Env: parentEnv, + }) + got.Env["EXTRA"] = "mutated" + assert.Equal(t, map[string]string{"EXTRA": "1"}, taskEnv) + assert.Equal(t, map[string]string{"LOG": "info"}, parentEnv) + }) + + t.Run("task env is copied when the parent has none", func(t *testing.T) { + t.Parallel() + taskEnv := map[string]string{"EXTRA": "1"} + got := Task{From: "api", On: TaskOnManual, Env: taskEnv}.MergeFrom(&Component{ + Image: "ghcr.io/acme/shop:1.2.3", + }) + got.Env["EXTRA"] = "mutated" + assert.Equal(t, map[string]string{"EXTRA": "1"}, taskEnv) + }) + + t.Run("no env stays nil", func(t *testing.T) { + t.Parallel() + got := Task{From: "api", On: TaskOnManual}.MergeFrom(&Component{ + Image: "ghcr.io/acme/shop:1.2.3", + }) + assert.Nil(t, got.Env) + }) + + t.Run("parent resource quantities are cloned", func(t *testing.T) { + t.Parallel() + cpu := MustQuantity("100m") + got := Task{From: "api", On: TaskOnManual, Command: []string{"true"}}.MergeFrom(&Component{ + Image: "ghcr.io/acme/shop:1.2.3", + Resources: Resources{CPU: cpu}, + }) + require.NotNil(t, got.Resources.CPU) + assert.NotSame(t, cpu, got.Resources.CPU) + got.Resources.CPU.Add(*MustQuantity("1")) + assert.True(t, cpu.Equal(*MustQuantity("100m"))) + }) +} + +func TestMergedTask_InheritsParentResourcesAfterDefaults(t *testing.T) { + t.Parallel() + + m := &Spec{ + APIVersion: CurrentManifestVersion, + Project: "shop", + Components: map[string]Component{ + "api": { + Role: ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + ResourcePreset: ResourcePresetNano, + }, + }, + Tasks: map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"migrate"}}, + "own": { + From: "api", + On: TaskOnManual, + Command: []string{"true"}, + ResourcePreset: ResourcePresetLarge, + }, + "standalone": {Image: "busybox:1.36", On: TaskOnManual, Command: []string{"true"}}, + }, + } + require.NoError(t, FillSpecWithDefaults(m, CurrentManifestVersion)) + + migrate, ok := m.MergedTask("migrate") + require.True(t, ok) + require.NotNil(t, migrate.Resources.CPU) + assert.True(t, migrate.Resources.CPU.Equal(*MustQuantity("100m")), + "MergedTask(migrate) CPU = %s, want nano 100m", migrate.Resources.CPU.String()) + + own, ok := m.MergedTask("own") + require.True(t, ok) + require.NotNil(t, own.Resources.Memory) + assert.True(t, own.Resources.Memory.Equal(*MustQuantity("2048Mi")), + "MergedTask(own) memory = %s, want large 2048Mi", own.Resources.Memory.String()) + + standalone, ok := m.MergedTask("standalone") + require.True(t, ok) + require.NotNil(t, standalone.Resources.CPU) + assert.True(t, standalone.Resources.CPU.Equal(*MustQuantity("500m")), + "MergedTask(standalone) CPU = %s, want small 500m", standalone.Resources.CPU.String()) +} + +func TestSpec_MergedTask(t *testing.T) { + t.Parallel() + + m := shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"migrate", "up"}}, + }) + got, ok := m.MergedTask("migrate") + require.True(t, ok) + assert.Equal(t, "ghcr.io/acme/shop:1.2.3", got.Image) + assert.Equal(t, "postgres://db", got.Env["DATABASE_URL"]) + + _, ok = m.MergedTask("missing") + assert.False(t, ok) + + var nilSpec *Spec + _, ok = nilSpec.MergedTask("migrate") + assert.False(t, ok) + assert.Nil(t, nilSpec.TaskNames()) +} + +func TestSpec_TaskNames(t *testing.T) { + t.Parallel() + + assert.Nil(t, (*Spec)(nil).TaskNames()) + assert.Nil(t, (&Spec{}).TaskNames()) + m := shopSpec(map[string]Task{ + "seed": {From: "api", On: TaskOnPreDeploy, Command: []string{"true"}}, + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"true"}}, + }) + assert.Equal(t, []string{"migrate", "seed"}, m.TaskNames()) +} + +func TestTask_EffectiveImage(t *testing.T) { + t.Parallel() + + parent := &Component{Image: "ghcr.io/acme/shop:1.2.3"} + assert.Equal(t, "busybox:1.36", Task{Image: "busybox:1.36"}.EffectiveImage(parent)) + assert.Equal(t, parent.Image, Task{From: "api"}.EffectiveImage(parent)) + assert.Empty(t, Task{}.EffectiveImage(nil)) +} + +func TestTask_UsesParentImage(t *testing.T) { + t.Parallel() + + assert.True(t, Task{From: "api"}.UsesParentImage()) + assert.False(t, Task{From: "api", Image: "busybox:1.36"}.UsesParentImage()) + assert.False(t, Task{Image: "busybox:1.36"}.UsesParentImage()) +} + +func TestSpec_tasksInEnvironment(t *testing.T) { + t.Parallel() + + m := &Spec{ + APIVersion: CurrentManifestVersion, + Project: "shop", + Components: map[string]Component{ + "api": { + Role: ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + Environments: []string{"prod"}, + }, + }, + Tasks: map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"migrate"}}, + "smoke": {From: "api", On: TaskOnPostDeploy, Environments: []string{"dev", "prod"}, Command: []string{"true"}}, + "nightly": {From: "api", On: TaskOnManual, Environments: []string{"prod"}, Command: []string{"true"}}, + }, + } + + tests := []struct { + name string + env string + want []string + }{ + {name: "dev skips inherited prod-only parent", env: "dev", want: []string{"smoke"}}, + {name: "prod includes inherited and explicit", env: "prod", want: []string{"migrate", "nightly", "smoke"}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got := m.tasksInEnvironment(tt.env) + names := make([]string, 0, len(got)) + for name := range got { + names = append(names, name) + } + slices.Sort(names) + assert.Equal(t, tt.want, names) + }) + } +} + +func TestEffectiveTasks(t *testing.T) { + t.Parallel() + + manifest := &Spec{ + APIVersion: CurrentManifestVersion, + Project: "shop", + Components: map[string]Component{ + "api": { + Role: ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + Environments: []string{"prod"}, + }, + }, + Tasks: map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"migrate"}}, + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Command: []string{"seed"}}, + "smoke": {From: "api", On: TaskOnPostDeploy, Environments: []string{"dev", "prod"}, Command: []string{"true"}}, + }, + } + + tests := []struct { + name string + manifest *Spec + environment string + resolved *ResolvedSpec + wantNames []string + wantWeight map[string]int + }{ + { + name: "nil resolved filters by environment", + manifest: manifest, + environment: "dev", + wantNames: []string{"smoke"}, + wantWeight: map[string]int{"smoke": 0}, + }, + { + name: "nil resolved assigns after weights", + manifest: manifest, + environment: "prod", + wantNames: []string{"migrate", "seed", "smoke"}, + wantWeight: map[string]int{"migrate": 0, "seed": 1, "smoke": 0}, + }, + { + name: "resolved tasks win over manifest filter", + manifest: manifest, + environment: "dev", + resolved: &ResolvedSpec{ + Tasks: map[string]ResolvedTask{ + "migrate": {Task: Task{On: TaskOnPreDeploy}, HookWeight: 3}, + }, + }, + wantNames: []string{"migrate"}, + wantWeight: map[string]int{"migrate": 3}, + }, + { + name: "nil manifest and resolved is empty", + environment: "dev", + wantNames: []string{}, + wantWeight: map[string]int{}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got, err := EffectiveTasks(tt.manifest, tt.environment, tt.resolved) + require.NoError(t, err) + names := make([]string, 0, len(got)) + weights := make(map[string]int, len(got)) + for name, rt := range got { + names = append(names, name) + weights[name] = rt.HookWeight + } + slices.Sort(names) + assert.Equal(t, tt.wantNames, names) + assert.Equal(t, tt.wantWeight, weights) + }) + } +} + +func TestEffectiveTasks_Cycle(t *testing.T) { + t.Parallel() + _, err := EffectiveTasks(&Spec{ + Tasks: map[string]Task{ + "a": {On: TaskOnPreDeploy, After: []string{"b"}, Command: []string{"true"}}, + "b": {On: TaskOnPreDeploy, After: []string{"a"}, Command: []string{"true"}}, + }, + }, "dev", nil) + require.Error(t, err) + assert.Contains(t, err.Error(), "cycle") +} + +func TestEffectiveTasks_CopyDoesNotAliasResolved(t *testing.T) { + t.Parallel() + resolved := &ResolvedSpec{ + Tasks: map[string]ResolvedTask{ + "migrate": {Task: Task{On: TaskOnPreDeploy}}, + }, + } + got, err := EffectiveTasks(nil, "dev", resolved) + require.NoError(t, err) + delete(got, "migrate") + assert.Contains(t, resolved.Tasks, "migrate") +} + +func TestTask_HelmHookEvents(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + on TaskOn + want string + }{ + {name: "preDeploy", on: TaskOnPreDeploy, want: "pre-install,pre-upgrade"}, + {name: "postDeploy", on: TaskOnPostDeploy, want: "post-install,post-upgrade"}, + {name: "manual", on: TaskOnManual, want: ""}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + assert.Equal(t, tt.want, Task{On: tt.on}.HelmHookEvents()) + }) + } +} + +func TestValidateSpecTasks(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + spec *Spec + wantErr string + }{ + { + name: "valid preDeploy from parent", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"migrate", "up"}, + }, + }), + }, + { + name: "from missing component", + spec: shopSpec(map[string]Task{ + "migrate": {From: "missing", On: TaskOnPreDeploy, Command: []string{"true"}}, + }), + wantErr: "does not name a component", + }, + { + name: "name collides with a component", + spec: shopSpec(map[string]Task{ + "api": {From: "api", On: TaskOnManual, Command: []string{"true"}}, + }), + wantErr: "collides with a component", + }, + { + name: "on is required", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", Command: []string{"true"}}, + }), + wantErr: "on is required", + }, + { + name: "on schedule points at issue 35", + spec: shopSpec(map[string]Task{ + "nightly": {From: "api", On: TaskOn("schedule"), Command: []string{"true"}}, + }), + wantErr: "issues/35", + }, + { + name: "on invalid value", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOn("whenever"), Command: []string{"true"}}, + }), + wantErr: "is invalid", + }, + { + name: "after on manual is rejected", + spec: shopSpec(map[string]Task{ + "backfill": {From: "api", On: TaskOnManual, After: []string{"migrate"}, Command: []string{"true"}}, + }), + wantErr: "after is not allowed on manual tasks", + }, + { + name: "after cross-phase is rejected", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"true"}}, + "smoke": {From: "api", On: TaskOnPostDeploy, After: []string{"migrate"}, Command: []string{"true"}}, + }), + wantErr: "not in the same on phase", + }, + { + name: "after missing task", + spec: shopSpec(map[string]Task{ + "smoke": {From: "api", On: TaskOnPostDeploy, After: []string{"missing"}, Command: []string{"true"}}, + }), + wantErr: "does not name a task", + }, + { + name: "after cycle", + spec: shopSpec(map[string]Task{ + "a": {From: "api", On: TaskOnPreDeploy, After: []string{"b"}, Command: []string{"true"}}, + "b": {From: "api", On: TaskOnPreDeploy, After: []string{"a"}, Command: []string{"true"}}, + }), + wantErr: "cycle", + }, + { + name: "command required when using parent image", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy}, + }), + wantErr: "command is required when using the parent image", + }, + { + name: "command optional with custom image", + spec: shopSpec(map[string]Task{ + "tool": {Image: "busybox:1.36", On: TaskOnManual}, + }), + }, + { + name: "fanout count greater than 1 on hook is allowed", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + Fanout: Fanout{Count: 4, Parallelism: 1}, + }, + }), + }, + { + name: "parallelism greater than count", + spec: shopSpec(map[string]Task{ + "backfill": { + From: "api", + On: TaskOnManual, + Command: []string{"true"}, + Fanout: Fanout{Count: 2, Parallelism: 4}, + }, + }), + wantErr: "less than or equal to fanout.count", + }, + { + name: "parallelism exceeds indexed job limit", + spec: shopSpec(map[string]Task{ + "backfill": { + From: "api", + On: TaskOnManual, + Command: []string{"true"}, + Fanout: Fanout{Count: MaxFanoutParallelism + 1, Parallelism: MaxFanoutParallelism + 1}, + }, + }), + wantErr: "fanout.parallelism must be at most", + }, + { + name: "hook timeout equal to default deploy timeout is allowed at spec load", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + Timeout: "10m", + }, + }), + }, + { + name: "hook timeout longer than default deploy timeout is allowed at spec load", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + Timeout: "15m", + }, + }), + }, + { + name: "after dependency not active in every dependent environment", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Environments: []string{"prod"}, Command: []string{"true"}}, + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Command: []string{"true"}}, + }), + wantErr: "not active in every environment", + }, + { + name: "after dependency covers dependent environments", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"true"}}, + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Environments: []string{"prod"}, Command: []string{"true"}}, + }), + }, + { + name: "after same inherited environment filter", + spec: &Spec{ + APIVersion: CurrentManifestVersion, + Project: "shop", + Components: map[string]Component{ + "api": { + Role: ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + Environments: []string{"prod"}, + }, + }, + Tasks: map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"true"}}, + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Command: []string{"true"}}, + }, + }, + }, + { + name: "after prefix on dependency covers specific dependent", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Environments: []string{"review"}, Command: []string{"true"}}, + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Environments: []string{"review/pr-123"}, Command: []string{"true"}}, + }), + }, + { + name: "after specific dependency does not cover prefix dependent", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Environments: []string{"review/pr-123"}, Command: []string{"true"}}, + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Environments: []string{"review"}, Command: []string{"true"}}, + }), + wantErr: "not active in every environment", + }, + { + name: "after inherited prod-only vs dependent everywhere", + spec: &Spec{ + APIVersion: CurrentManifestVersion, + Project: "shop", + Components: map[string]Component{ + "api": { + Role: ComponentRoleService, + Image: "ghcr.io/acme/shop:1.2.3", + }, + }, + Tasks: map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Environments: []string{"prod"}, Command: []string{"true"}}, + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Command: []string{"true"}}, + }, + }, + wantErr: "not active in every environment", + }, + { + name: "task cannot have both resources and resourcePreset", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + ResourcePreset: ResourcePresetSmall, + Resources: Resources{CPU: MustQuantity("100m")}, + }, + }), + wantErr: "cannot have both", + }, + { + name: "fanout count exceeds int32", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + Fanout: Fanout{Count: math.MaxInt32 + 1, Parallelism: 1}, + }, + }), + wantErr: "outside the int32 range", + }, + { + name: "from or image required", + spec: shopSpec(map[string]Task{ + "orphan": {On: TaskOnManual, Command: []string{"true"}}, + }), + wantErr: "from or image is required", + }, + { + name: "invalid task name", + spec: shopSpec(map[string]Task{ + "Migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"true"}}, + }), + wantErr: "is invalid", + }, + { + name: "after contains an empty name", + spec: shopSpec(map[string]Task{ + "seed": {From: "api", On: TaskOnPreDeploy, After: []string{" "}, Command: []string{"true"}}, + }), + wantErr: "after contains an empty name", + }, + { + name: "after includes itself", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, After: []string{"migrate"}, Command: []string{"true"}}, + }), + wantErr: "after cannot include itself", + }, + { + name: "backoffLimit exceeds int32", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + BackoffLimit: new(math.MaxInt32 + 1), + }, + }), + wantErr: "outside the int32 range", + }, + { + name: "ttlSecondsAfterFinished exceeds int32", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + TTLSecondsAfterFinished: new(math.MaxInt32 + 1), + }, + }), + wantErr: "outside the int32 range", + }, + { + name: "invalid timeout", + spec: shopSpec(map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Command: []string{"true"}, Timeout: "not-a-duration"}, + }), + wantErr: "timeout", + }, + { + name: "environments slash-star suffix", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + Environments: []string{"review/*"}, + }, + }), + wantErr: `"/*" suffix is not supported`, + }, + { + name: "empty profile name", + spec: shopSpec(map[string]Task{ + "migrate": { + From: "api", + On: TaskOnPreDeploy, + Command: []string{"true"}, + Profiles: []string{" "}, + }, + }), + wantErr: "profile name must not be empty", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + err := ValidateSpecTasks(tt.spec) + if tt.wantErr == "" { + require.NoError(t, err) + return + } + require.Error(t, err) + assert.Contains(t, err.Error(), tt.wantErr) + }) + } +} + +func TestValidateSpecTasks_Nil(t *testing.T) { + t.Parallel() + + err := ValidateSpecTasks(nil) + require.Error(t, err) + assert.Contains(t, err.Error(), "spec cannot be nil") +} + +func TestCheckHookTaskTimeouts(t *testing.T) { + t.Parallel() + + tasks := map[string]Task{ + "migrate": {On: TaskOnPreDeploy, Timeout: "5m"}, + "backfill": {On: TaskOnManual, Timeout: "1h"}, + } + + require.NoError(t, CheckHookTaskTimeouts(tasks, 10*time.Minute)) + err := CheckHookTaskTimeouts(tasks, 5*time.Minute) + require.Error(t, err) + assert.Contains(t, err.Error(), "migrate") + assert.NotContains(t, err.Error(), "backfill") + + long := map[string]Task{ + "migrate": {On: TaskOnPreDeploy, Timeout: "15m"}, + } + require.NoError(t, CheckHookTaskTimeouts(long, 20*time.Minute)) + err = CheckHookTaskTimeouts(long, 10*time.Minute) + require.Error(t, err) + assert.Contains(t, err.Error(), "migrate") + + err = CheckHookTaskTimeouts(tasks, 0) + require.Error(t, err) + assert.Contains(t, err.Error(), "session timeout must be a positive duration") + + err = CheckHookTaskTimeouts(map[string]Task{ + "migrate": {On: TaskOnPreDeploy, Timeout: "not-a-duration"}, + }, 10*time.Minute) + require.Error(t, err) + assert.Contains(t, err.Error(), "timeout") + assert.NotContains(t, err.Error(), "use --detach") +} + +func TestCheckTaskTimeout(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + task Task + limit time.Duration + wantErr string + }{ + { + name: "hook under limit", + task: Task{On: TaskOnPreDeploy, Timeout: "5m"}, + limit: 10 * time.Minute, + }, + { + name: "hook at limit", + task: Task{On: TaskOnPreDeploy, Timeout: "5m"}, + limit: 5 * time.Minute, + wantErr: "use --detach or raise --timeout", + }, + { + name: "manual over limit", + task: Task{On: TaskOnManual, Timeout: "30m"}, + limit: 10 * time.Minute, + wantErr: "use --detach or raise --timeout", + }, + { + name: "manual empty timeout skipped", + task: Task{On: TaskOnManual}, + limit: 10 * time.Minute, + }, + { + name: "hook empty timeout uses default", + task: Task{On: TaskOnPreDeploy}, + limit: 5 * time.Minute, + wantErr: DefaultHookTaskTimeout, + }, + { + name: "non-positive limit", + task: Task{On: TaskOnManual, Timeout: "1m"}, + limit: 0, + wantErr: "session timeout must be a positive duration", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + err := CheckTaskTimeout("migrate", tt.task, tt.limit) + if tt.wantErr == "" { + require.NoError(t, err) + return + } + require.Error(t, err) + assert.Contains(t, err.Error(), tt.wantErr) + }) + } +} + +func TestCheckTaskTimeout_InvalidDuration(t *testing.T) { + t.Parallel() + + err := CheckTaskTimeout("migrate", Task{On: TaskOnManual, Timeout: "not-a-duration"}, time.Minute) + require.Error(t, err) + assert.Contains(t, err.Error(), "timeout") + assert.NotContains(t, err.Error(), "use --detach") +} + +func TestCheckHookTaskTimeouts_UsesEnvFilter(t *testing.T) { + t.Parallel() + + m := &Spec{ + Components: map[string]Component{ + "api": {Image: "x", Environments: []string{"prod"}}, + }, + Tasks: map[string]Task{ + "migrate": {From: "api", On: TaskOnPreDeploy, Timeout: "9m", Command: []string{"true"}}, + }, + } + require.NoError(t, CheckHookTaskTimeouts(m.tasksInEnvironment("dev"), 5*time.Minute)) + err := CheckHookTaskTimeouts(m.Tasks, 5*time.Minute) + require.Error(t, err) + assert.Contains(t, err.Error(), "migrate") +} + +func TestValidateAPIVersion_SupportedVersions(t *testing.T) { + t.Parallel() + + got, err := ValidateAPIVersion(map[string]any{"apiVersion": CurrentManifestVersion}) + require.NoError(t, err) + assert.Equal(t, CurrentManifestVersion, got) + + _, err = ValidateAPIVersion(map[string]any{"apiVersion": "v1-alpha.4"}) + require.Error(t, err) + assert.Contains(t, err.Error(), "unsupported spec schema version") + assert.Contains(t, err.Error(), "v1-alpha.4") +} diff --git a/internal/spec/task_validate.go b/internal/spec/task_validate.go new file mode 100644 index 0000000..f254c55 --- /dev/null +++ b/internal/spec/task_validate.go @@ -0,0 +1,254 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing the License. + +package spec + +import ( + "errors" + "fmt" + "maps" + "slices" + "strings" + "time" +) + +// ValidateSpecTasks validates all tasks in spec: name pool, from, on, +// after, fanout, command, timeout, and environment filter. +func ValidateSpecTasks(spec *Spec) error { + if spec == nil { + return fmt.Errorf("spec cannot be nil") + } + if len(spec.Tasks) == 0 { + return nil + } + + var errs []error + for _, name := range spec.TaskNames() { + task := spec.Tasks[name] + if err := ValidateComponentName(name); err != nil { + errs = append(errs, fmt.Errorf("task %s: %w", name, err)) + } + if _, exists := spec.Components[name]; exists { + errs = append(errs, fmt.Errorf("task %s: name collides with a component", name)) + } + if err := validateTask(name, task, spec); err != nil { + errs = append(errs, err) + } + } + if err := validateTaskAfterGraph(spec); err != nil { + errs = append(errs, err) + } + if len(errs) > 0 { + return fmt.Errorf("task validation failed: %w", errors.Join(errs...)) + } + return nil +} + +func validateTask(name string, task Task, spec *Spec) error { + var errs []error + prefix := fmt.Sprintf("task %s", name) + + if task.From == "" && task.Image == "" { + errs = append(errs, fmt.Errorf("%s: from or image is required", prefix)) + } + if task.From != "" { + if _, ok := spec.Components[task.From]; !ok { + errs = append(errs, fmt.Errorf("%s: from %q does not name a component", prefix, task.From)) + } + } + + switch task.On { + case TaskOnPreDeploy, TaskOnPostDeploy, TaskOnManual: + case TaskOn(scheduleOnToken): + errs = append(errs, fmt.Errorf("%s: on: schedule is not supported yet (see https://github.com/deployah-dev/deployah/issues/35)", prefix)) + case "": + errs = append(errs, fmt.Errorf("%s: on is required (preDeploy, postDeploy, or manual)", prefix)) + default: + errs = append(errs, fmt.Errorf("%s: on %q is invalid (preDeploy, postDeploy, or manual)", prefix, task.On)) + } + + if len(task.After) > 0 && task.On == TaskOnManual { + errs = append(errs, fmt.Errorf("%s: after is not allowed on manual tasks", prefix)) + } + for _, dep := range task.After { + if strings.TrimSpace(dep) == "" { + errs = append(errs, fmt.Errorf("%s: after contains an empty name", prefix)) + continue + } + if dep == name { + errs = append(errs, fmt.Errorf("%s: after cannot include itself", prefix)) + } + } + + usesParent := task.Image == "" && task.From != "" + if usesParent && len(task.Command) == 0 { + errs = append(errs, fmt.Errorf("%s: command is required when using the parent image", prefix)) + } + + count := task.Fanout.EffectiveCount() + parallelism := task.Fanout.EffectiveParallelism() + if count < 1 { + errs = append(errs, fmt.Errorf("%s: fanout.count must be at least 1", prefix)) + } else if _, err := toInt32("fanout.count", count); err != nil { + errs = append(errs, fmt.Errorf("%s: %w", prefix, err)) + } + if parallelism < 1 { + errs = append(errs, fmt.Errorf("%s: fanout.parallelism must be at least 1", prefix)) + } else if parallelism > MaxFanoutParallelism { + errs = append(errs, fmt.Errorf("%s: fanout.parallelism must be at most %d", prefix, MaxFanoutParallelism)) + } + if parallelism > count { + errs = append(errs, fmt.Errorf("%s: fanout.parallelism must be less than or equal to fanout.count", prefix)) + } + if task.BackoffLimit != nil { + if _, err := toInt32("backoffLimit", *task.BackoffLimit); err != nil { + errs = append(errs, fmt.Errorf("%s: %w", prefix, err)) + } + } + if task.TTLSecondsAfterFinished != nil { + if _, err := toInt32("ttlSecondsAfterFinished", *task.TTLSecondsAfterFinished); err != nil { + errs = append(errs, fmt.Errorf("%s: %w", prefix, err)) + } + } + + if err := validateResources(task.Resources, task.ResourcePreset); err != nil { + errs = append(errs, fmt.Errorf("%s: %w", prefix, err)) + } + + if err := ValidateComponentEnvironmentFilter(Component{Environments: task.Environments}); err != nil { + errs = append(errs, fmt.Errorf("%s: %w", prefix, err)) + } + if err := ValidateComponentProfiles(Component{Profiles: task.Profiles}); err != nil { + errs = append(errs, fmt.Errorf("%s: %w", prefix, err)) + } + + if task.Timeout != "" { + if _, err := ParseDuration(task.Timeout); err != nil { + errs = append(errs, fmt.Errorf("%s: timeout: %w", prefix, err)) + } + } + + if len(errs) > 0 { + return errors.Join(errs...) + } + return nil +} + +func validateTaskAfterGraph(spec *Spec) error { + var errs []error + tasks := spec.Tasks + for _, name := range slices.Sorted(maps.Keys(tasks)) { + task := tasks[name] + if !task.On.IsHook() { + continue + } + dependent, _ := spec.MergedTask(name) + for _, dep := range task.After { + other, ok := tasks[dep] + if !ok { + errs = append(errs, fmt.Errorf("task %s: after %q does not name a task", name, dep)) + continue + } + if other.On != task.On { + errs = append(errs, fmt.Errorf("task %s: after %q is not in the same on phase (%s)", name, dep, task.On)) + continue + } + depTask, _ := spec.MergedTask(dep) + if !afterCoversDependentEnvs(dependent, depTask) { + errs = append(errs, fmt.Errorf("task %s: after %q is not active in every environment where %s runs", name, dep, name)) + } + } + } + if _, err := AssignHookWeights(tasks); err != nil { + errs = append(errs, err) + } + if len(errs) > 0 { + return errors.Join(errs...) + } + return nil +} + +// afterCoversDependentEnvs reports whether dep runs in every environment +// dependent runs in. An empty environments list means every environment. +func afterCoversDependentEnvs(dependent, dep Task) bool { + if len(dep.Environments) == 0 { + return true + } + if len(dependent.Environments) == 0 { + return false + } + for _, env := range dependent.Environments { + if _, ok := matchEnvKey(env, dep.Environments); !ok { + return false + } + } + return true +} + +// CheckHookTaskTimeouts reports hook tasks whose timeout is not strictly +// less than limit (the CLI --timeout). Manual tasks are skipped. An empty +// timeout is treated as [DefaultHookTaskTimeout]. +func CheckHookTaskTimeouts(tasks map[string]Task, limit time.Duration) error { + if limit <= 0 { + return fmt.Errorf("session timeout must be a positive duration") + } + var errs []error + for _, name := range slices.Sorted(maps.Keys(tasks)) { + task := tasks[name] + if !task.On.IsHook() { + continue + } + if err := taskTimeoutAgainstLimit(name, task, limit, ""); err != nil { + errs = append(errs, err) + } + } + if len(errs) > 0 { + return errors.Join(errs...) + } + return nil +} + +// CheckTaskTimeout reports when task's timeout is not strictly less than +// limit (the CLI --timeout). An empty timeout on a hook is treated as +// [DefaultHookTaskTimeout]. An empty timeout on a manual task is skipped +// because the Job has no deadline. When the timeout exceeds limit, the +// error names --detach and --timeout so the caller can wait in the +// background or raise the session limit. +func CheckTaskTimeout(name string, task Task, limit time.Duration) error { + if limit <= 0 { + return fmt.Errorf("session timeout must be a positive duration") + } + return taskTimeoutAgainstLimit(name, task, limit, "use --detach or raise --timeout") +} + +func taskTimeoutAgainstLimit(name string, task Task, limit time.Duration, hint string) error { + timeout := task.Timeout + if timeout == "" { + if !task.On.IsHook() { + return nil + } + timeout = DefaultHookTaskTimeout + } + sec, err := ParseDuration(timeout) + if err != nil { + return fmt.Errorf("task %s: timeout: %w", name, err) + } + if time.Duration(sec)*time.Second >= limit { + msg := fmt.Sprintf("task %s: timeout %s must be less than --timeout %s", name, timeout, limit) + if hint != "" { + return fmt.Errorf("%s (%s)", msg, hint) + } + return errors.New(msg) + } + return nil +} diff --git a/internal/spec/types.go b/internal/spec/types.go index 6e2e721..9dec530 100644 --- a/internal/spec/types.go +++ b/internal/spec/types.go @@ -25,7 +25,7 @@ import ( // Spec defines the structure of the project spec. type Spec struct { - // APIVersion is the schema version of the spec (e.g., "v1-alpha.4"). + // APIVersion is the schema version of the spec (e.g., "v1-alpha.5"). APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"` // Project is the project name. Project string `json:"project" yaml:"project"` @@ -35,6 +35,8 @@ type Spec struct { Environments map[string]Environment `json:"environments,omitempty" yaml:"environments,omitempty"` // Components is a map of component names to their configuration. Components map[string]Component `json:"components" yaml:"components"` + // Tasks is a map of task names to run-to-completion work. + Tasks map[string]Task `json:"tasks,omitempty" yaml:"tasks,omitempty"` } // EnvironmentNames returns the sorted list of environment names defined in the @@ -389,12 +391,10 @@ const ( ComponentRoleService ComponentRole = "service" // ComponentRoleWorker runs background or queue-processing workloads. ComponentRoleWorker ComponentRole = "worker" - // ComponentRoleJob runs a finite batch or one-off task. - ComponentRoleJob ComponentRole = "job" ) // IsService reports whether r is the "service" role, the only role that -// listens on a port or gets exposed via an ingress rule. Worker and job +// listens on a port or gets exposed via an ingress rule. Worker // components run without inbound traffic. func (r ComponentRole) IsService() bool { return r == ComponentRoleService diff --git a/internal/spec/types_test.go b/internal/spec/types_test.go index aa7ba57..bb60582 100644 --- a/internal/spec/types_test.go +++ b/internal/spec/types_test.go @@ -38,7 +38,6 @@ func TestComponentRole_IsService(t *testing.T) { assert.True(t, ComponentRoleService.IsService()) assert.False(t, ComponentRoleWorker.IsService()) - assert.False(t, ComponentRoleJob.IsService()) } // TestComponentRole_IsWorker verifies only the "worker" role is treated @@ -48,7 +47,6 @@ func TestComponentRole_IsWorker(t *testing.T) { assert.True(t, ComponentRoleWorker.IsWorker()) assert.False(t, ComponentRoleService.IsWorker()) - assert.False(t, ComponentRoleJob.IsWorker()) } // TestComponent_ListensOnPort verifies a component listens on a port only @@ -64,7 +62,6 @@ func TestComponent_ListensOnPort(t *testing.T) { {name: "service with port", component: Component{Role: ComponentRoleService, Port: 8080}, want: true}, {name: "service without a port", component: Component{Role: ComponentRoleService, Port: 0}, want: false}, {name: "worker with a port set", component: Component{Role: ComponentRoleWorker, Port: 8080}, want: false}, - {name: "job with a port set", component: Component{Role: ComponentRoleJob, Port: 8080}, want: false}, } for _, tt := range tests { diff --git a/internal/spec/validate.go b/internal/spec/validate.go index 33ca301..64ec445 100644 --- a/internal/spec/validate.go +++ b/internal/spec/validate.go @@ -87,7 +87,7 @@ func validateYAMLAgainstSchema( } // ValidateSpec validates spec YAML against the provided JSON schema. -// version should be the version of the schema (e.g., "v1-alpha.4"). +// version should be the version of the schema (e.g., "v1-alpha.5"). // This is a strict validation: unknown fields are not allowed. func ValidateSpec(specObj map[string]any, version string) error { return validateYAMLAgainstSchema( @@ -100,7 +100,7 @@ func ValidateSpec(specObj map[string]any, version string) error { // ValidateEnvironments validates environments YAML against the provided JSON // schema file. -// version should be the version of the schema (e.g., "v1-alpha.4"). +// version should be the version of the schema (e.g., "v1-alpha.5"). // This is a strict validation: unknown fields are not allowed. func ValidateEnvironments(specObj map[string]any, version string) error { return validateYAMLAgainstSchema( @@ -112,14 +112,9 @@ func ValidateEnvironments(specObj map[string]any, version string) error { } // ValidateAPIVersion checks the spec apiVersion field for presence, type, -// and validity. +// and membership in [SupportedManifestVersions]. // Returns the apiVersion string if valid, or an error otherwise. func ValidateAPIVersion(specObj map[string]any) (string, error) { - validVersions, err := schema.GetValidManifestVersions() - if err != nil { - return "", fmt.Errorf("failed to get valid spec versions: %w", err) - } - apiVersionVal, ok := specObj["apiVersion"] if !ok { return "", fmt.Errorf("spec is missing 'apiVersion' field") @@ -130,8 +125,8 @@ func ValidateAPIVersion(specObj map[string]any) (string, error) { return "", fmt.Errorf("'apiVersion' field must be a non-empty string") } - if !slices.Contains(validVersions, apiVersionStr) { - return "", fmt.Errorf("unsupported spec schema version: %s (valid: %v)", apiVersionStr, validVersions) + if !slices.Contains(SupportedManifestVersions, apiVersionStr) { + return "", fmt.Errorf("unsupported spec schema version: %s (this release requires %s)", apiVersionStr, strings.Join(SupportedManifestVersions, ", ")) } return apiVersionStr, nil @@ -145,17 +140,27 @@ func ValidateAPIVersion(specObj map[string]any) (string, error) { // It cannot have both resources and resourcePreset, or an empty resources // object. func ValidateComponentResources(component Component) error { - hasResources := component.Resources.ResourcesSet() - hasPreset := component.ResourcePreset != "" + if err := validateResources(component.Resources, component.ResourcePreset); err != nil { + return fmt.Errorf("component %w", err) + } + return nil +} + +// validateResources reports when resources and resourcePreset are both set, +// or when a resources block is present but empty. Callers add the subject +// (component or task name) when they wrap the error. +func validateResources(resources Resources, preset ResourcePreset) error { + hasResources := resources.ResourcesSet() + hasPreset := preset != "" if hasResources && hasPreset { - return fmt.Errorf("component cannot have both 'resources' and 'resourcePreset' fields") + return fmt.Errorf("cannot have both 'resources' and 'resourcePreset' fields") } // Check if resources object is present but empty (resources: {} or // zero quantities). Pointers let us detect an explicitly set block. - if !hasResources && component.Resources.ResourcesPresent() { - return fmt.Errorf("component cannot have empty 'resources' object - either specify actual resource values or remove the resources field entirely") + if !hasResources && resources.ResourcesPresent() { + return fmt.Errorf("cannot have empty 'resources' object - either specify actual resource values or remove the resources field entirely") } // Both empty is allowed (will use defaults) @@ -221,11 +226,6 @@ func ValidateComponentHealth(component Component) error { return nil } - // Job role still rejects all health configuration. - if component.Role == ComponentRoleJob { - return fmt.Errorf("health checks are not supported for role: job components") - } - if component.Health.Ready != nil && !component.Health.Ready.Disabled { if component.Health.Ready.Path != "" && component.Health.Ready.Path[0] != '/' { return fmt.Errorf("health.ready.path must start with /") diff --git a/internal/spec/validate_test.go b/internal/spec/validate_test.go index 3bf6e4b..a586142 100644 --- a/internal/spec/validate_test.go +++ b/internal/spec/validate_test.go @@ -557,15 +557,6 @@ func TestValidateComponentHealth(t *testing.T) { expectErr: true, errMsg: "health.alive.exec[1] must not be empty", }, - { - name: "health on job role is invalid", - component: Component{ - Role: ComponentRoleJob, - Health: &Health{Ready: &HealthReady{Path: "/health"}}, - }, - expectErr: true, - errMsg: "health checks are not supported for role: job", - }, { name: "ready path without leading slash is invalid", component: Component{ diff --git a/internal/testing/plan_scenarios.go b/internal/testing/plan_scenarios.go index 3a57f27..d1adcaa 100644 --- a/internal/testing/plan_scenarios.go +++ b/internal/testing/plan_scenarios.go @@ -62,9 +62,8 @@ type PlanConfig struct { Changes []PlanConfigChange `yaml:"changes"` // Summary is the expected change tally. Summary PlanConfigSummary `yaml:"summary"` - // HooksChanged asserts [plan.Plan.HooksChanged]. Always false in this - // scenario suite today: the embedded chart defines no hooks, so this - // only documents the plumbing for when one is added. + // HooksChanged asserts [plan.Plan.HooksChanged]. True when hook tasks + // are present on a fresh install (Helm reports hook manifests). HooksChanged bool `yaml:"hooksChanged"` // Masked lists field paths (as they appear in a Changes[].Fields[] // entry's Path) that [plan.ApplyMasking] must flag as masked. Every @@ -148,6 +147,7 @@ func RunPlanScenarioTest(t *testing.T, scenario PlanTestScenario) { Namespace: current.Namespace, FreshInstall: previous.Manifest == "", } + p.Tasks = current.Tasks if cfg.Warning != "" { p.Header.Warning = cfg.Warning } @@ -169,6 +169,7 @@ type manifestSide struct { Environment string ReleaseName string Namespace string + Tasks []plan.PlannedTask } // resolvePreviousSide resolves a scenario's previous manifest: a raw @@ -269,6 +270,7 @@ func renderManifestFile(t *testing.T, dir, filename string) manifestSide { Environment: envName, ReleaseName: result.ReleaseName, Namespace: result.Namespace, + Tasks: mustPlanTasks(t, manifest, envName, resolved), } } @@ -358,3 +360,10 @@ func checkJSONGolden(t *testing.T, dir string, p *plan.Plan) { require.NoError(t, plan.RenderJSON(&buf, p)) compareOrUpdateGolden(t, goldenPath, buf.String()) } + +func mustPlanTasks(t *testing.T, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) []plan.PlannedTask { + t.Helper() + tasks, err := plan.TasksFromSpec(manifest, environment, resolved) + require.NoError(t, err) + return tasks +} diff --git a/internal/testing/types.go b/internal/testing/types.go index 188a40e..6edc405 100644 --- a/internal/testing/types.go +++ b/internal/testing/types.go @@ -263,7 +263,21 @@ func (suite *IntegrationTestSuite) renderChart(t *testing.T, testDir string, man return nil, fmt.Errorf("render chart: %w", err) } - return parseManifestYAML(result.Manifest) + manifests, parseErr := parseManifestYAML(result.Manifest) + if parseErr != nil { + return nil, parseErr + } + for _, h := range result.Hooks { + if h == nil || h.Manifest == "" { + continue + } + hookObjs, hookErr := parseManifestYAML(h.Manifest) + if hookErr != nil { + return nil, fmt.Errorf("parse hook %s: %w", h.Name, hookErr) + } + manifests = append(manifests, hookObjs...) + } + return manifests, nil } // parseManifestYAML decodes a "---"-concatenated Kubernetes YAML string @@ -325,6 +339,7 @@ func (suite *IntegrationTestSuite) validateAgainstExpected(t *testing.T, manifes // golden file per manifest. func (suite *IntegrationTestSuite) writeGoldenManifests(t *testing.T, manifests []unstructured.Unstructured, expectedDir string) { t.Helper() + require.NoError(t, os.MkdirAll(expectedDir, 0o750)) entries, err := os.ReadDir(expectedDir) require.NoError(t, err) for _, entry := range entries { diff --git a/scenarios/autoscaling-hpa/deployah.yaml b/scenarios/autoscaling-hpa/deployah.yaml index 27d5b37..bddf36e 100644 --- a/scenarios/autoscaling-hpa/deployah.yaml +++ b/scenarios/autoscaling-hpa/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: autoscaling-hpa components: api: diff --git a/scenarios/basic-web-service/deployah.yaml b/scenarios/basic-web-service/deployah.yaml index 39fe7c8..3796312 100644 --- a/scenarios/basic-web-service/deployah.yaml +++ b/scenarios/basic-web-service/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: basic-web-service components: web: diff --git a/scenarios/command-args-resources/deployah.yaml b/scenarios/command-args-resources/deployah.yaml index 8e0cedd..0bf98aa 100644 --- a/scenarios/command-args-resources/deployah.yaml +++ b/scenarios/command-args-resources/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: command-args-resources components: processor: diff --git a/scenarios/env-substitution/deployah.yaml b/scenarios/env-substitution/deployah.yaml index e7d0649..001322d 100644 --- a/scenarios/env-substitution/deployah.yaml +++ b/scenarios/env-substitution/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: env-substitution components: api: diff --git a/scenarios/error-apex-subdomain/deployah.yaml b/scenarios/error-apex-subdomain/deployah.yaml index 153f74c..ebcc81a 100644 --- a/scenarios/error-apex-subdomain/deployah.yaml +++ b/scenarios/error-apex-subdomain/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-apex-subdomain components: api: diff --git a/scenarios/error-empty-resources/deployah.yaml b/scenarios/error-empty-resources/deployah.yaml index 2bb3a08..b6611f4 100644 --- a/scenarios/error-empty-resources/deployah.yaml +++ b/scenarios/error-empty-resources/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-empty-resources components: api: diff --git a/scenarios/error-health-durations/deployah.yaml b/scenarios/error-health-durations/deployah.yaml index d398e36..218ae5a 100644 --- a/scenarios/error-health-durations/deployah.yaml +++ b/scenarios/error-health-durations/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-health-durations components: api: diff --git a/scenarios/error-health-on-worker/deployah.yaml b/scenarios/error-health-on-worker/deployah.yaml index 32a84ac..92698c6 100644 --- a/scenarios/error-health-on-worker/deployah.yaml +++ b/scenarios/error-health-on-worker/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-health-on-worker components: worker: diff --git a/scenarios/error-health-path/deployah.yaml b/scenarios/error-health-path/deployah.yaml index 3ddb341..028ca37 100644 --- a/scenarios/error-health-path/deployah.yaml +++ b/scenarios/error-health-path/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-health-path components: api: diff --git a/scenarios/error-metric-type/deployah.yaml b/scenarios/error-metric-type/deployah.yaml index 407a815..95d1e2c 100644 --- a/scenarios/error-metric-type/deployah.yaml +++ b/scenarios/error-metric-type/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-metric-type components: api: diff --git a/scenarios/error-multiple-issues/deployah.yaml b/scenarios/error-multiple-issues/deployah.yaml index 5ae76e6..c715380 100644 --- a/scenarios/error-multiple-issues/deployah.yaml +++ b/scenarios/error-multiple-issues/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-multiple-issues components: web: diff --git a/scenarios/error-profile-ceiling/deployah.yaml b/scenarios/error-profile-ceiling/deployah.yaml index b5308c1..3155d30 100644 --- a/scenarios/error-profile-ceiling/deployah.yaml +++ b/scenarios/error-profile-ceiling/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-profile-ceiling components: web: diff --git a/scenarios/error-profile-domain/deployah.yaml b/scenarios/error-profile-domain/deployah.yaml index f2bced4..1f7aeaf 100644 --- a/scenarios/error-profile-domain/deployah.yaml +++ b/scenarios/error-profile-domain/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-profile-domain components: web: diff --git a/scenarios/error-profile-unknown/deployah.yaml b/scenarios/error-profile-unknown/deployah.yaml index 1e8086d..1d1b186 100644 --- a/scenarios/error-profile-unknown/deployah.yaml +++ b/scenarios/error-profile-unknown/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-profile-unknown components: web: diff --git a/scenarios/error-replicas-autoscaling/deployah.yaml b/scenarios/error-replicas-autoscaling/deployah.yaml index 645d983..a636665 100644 --- a/scenarios/error-replicas-autoscaling/deployah.yaml +++ b/scenarios/error-replicas-autoscaling/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-replicas-autoscaling components: web: diff --git a/scenarios/error-stateless-persistence-replicas/deployah.yaml b/scenarios/error-stateless-persistence-replicas/deployah.yaml index 4b25ce3..0320bd0 100644 --- a/scenarios/error-stateless-persistence-replicas/deployah.yaml +++ b/scenarios/error-stateless-persistence-replicas/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-stateless-persistence-replicas components: web: diff --git a/scenarios/error-task-after-env/deployah.yaml b/scenarios/error-task-after-env/deployah.yaml new file mode 100644 index 0000000..5050e7f --- /dev/null +++ b/scenarios/error-task-after-env/deployah.yaml @@ -0,0 +1,21 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: error-task-after-env +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] +tasks: + migrate: + from: api + "on": preDeploy + environments: [prod] + command: ["true"] + seed: + from: api + "on": preDeploy + after: [migrate] + command: ["true"] +environments: + dev: {} diff --git a/scenarios/error-task-after-env/error-config.yaml b/scenarios/error-task-after-env/error-config.yaml new file mode 100644 index 0000000..8390ac6 --- /dev/null +++ b/scenarios/error-task-after-env/error-config.yaml @@ -0,0 +1,2 @@ +expectedErrors: + - "not active in every environment" diff --git a/scenarios/error-task-after-manual/deployah.yaml b/scenarios/error-task-after-manual/deployah.yaml new file mode 100644 index 0000000..6ea5387 --- /dev/null +++ b/scenarios/error-task-after-manual/deployah.yaml @@ -0,0 +1,16 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: error-task-after-manual +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] +tasks: + backfill: + from: api + "on": manual + after: [migrate] + command: ["true"] +environments: + dev: {} diff --git a/scenarios/error-task-after-manual/error-config.yaml b/scenarios/error-task-after-manual/error-config.yaml new file mode 100644 index 0000000..f377307 --- /dev/null +++ b/scenarios/error-task-after-manual/error-config.yaml @@ -0,0 +1,2 @@ +expectedErrors: + - "after is not allowed on manual tasks" diff --git a/scenarios/error-task-bad-on/deployah.yaml b/scenarios/error-task-bad-on/deployah.yaml new file mode 100644 index 0000000..206c856 --- /dev/null +++ b/scenarios/error-task-bad-on/deployah.yaml @@ -0,0 +1,15 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: error-task-bad-on +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] +tasks: + nightly: + from: api + "on": schedule + command: ["true"] +environments: + dev: {} diff --git a/scenarios/error-task-bad-on/error-config.yaml b/scenarios/error-task-bad-on/error-config.yaml new file mode 100644 index 0000000..1ec6076 --- /dev/null +++ b/scenarios/error-task-bad-on/error-config.yaml @@ -0,0 +1,2 @@ +expectedErrors: + - "preDeploy" diff --git a/scenarios/error-task-collision/deployah.yaml b/scenarios/error-task-collision/deployah.yaml new file mode 100644 index 0000000..5b32aed --- /dev/null +++ b/scenarios/error-task-collision/deployah.yaml @@ -0,0 +1,15 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: error-task-collision +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] +tasks: + api: + from: api + "on": preDeploy + command: ["true"] +environments: + dev: {} diff --git a/scenarios/error-task-collision/error-config.yaml b/scenarios/error-task-collision/error-config.yaml new file mode 100644 index 0000000..0571507 --- /dev/null +++ b/scenarios/error-task-collision/error-config.yaml @@ -0,0 +1,2 @@ +expectedErrors: + - "collides with a component" diff --git a/scenarios/error-task-cycle/deployah.yaml b/scenarios/error-task-cycle/deployah.yaml new file mode 100644 index 0000000..4679a2a --- /dev/null +++ b/scenarios/error-task-cycle/deployah.yaml @@ -0,0 +1,21 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: error-task-cycle +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] +tasks: + first: + from: api + "on": preDeploy + after: [second] + command: ["true"] + second: + from: api + "on": preDeploy + after: [first] + command: ["true"] +environments: + dev: {} diff --git a/scenarios/error-task-cycle/error-config.yaml b/scenarios/error-task-cycle/error-config.yaml new file mode 100644 index 0000000..3c66620 --- /dev/null +++ b/scenarios/error-task-cycle/error-config.yaml @@ -0,0 +1,2 @@ +expectedErrors: + - "cycle" diff --git a/scenarios/error-task-missing-command/deployah.yaml b/scenarios/error-task-missing-command/deployah.yaml new file mode 100644 index 0000000..23acf1d --- /dev/null +++ b/scenarios/error-task-missing-command/deployah.yaml @@ -0,0 +1,14 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: error-task-missing-command +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] +tasks: + migrate: + from: api + "on": preDeploy +environments: + dev: {} diff --git a/scenarios/error-task-missing-command/error-config.yaml b/scenarios/error-task-missing-command/error-config.yaml new file mode 100644 index 0000000..2804ab2 --- /dev/null +++ b/scenarios/error-task-missing-command/error-config.yaml @@ -0,0 +1,2 @@ +expectedErrors: + - "command is required when using the parent image" diff --git a/scenarios/error-worker-expose/deployah.yaml b/scenarios/error-worker-expose/deployah.yaml index 7c2c5ce..8e66e2d 100644 --- a/scenarios/error-worker-expose/deployah.yaml +++ b/scenarios/error-worker-expose/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-worker-expose components: worker: diff --git a/scenarios/error-worker-http-health/deployah.yaml b/scenarios/error-worker-http-health/deployah.yaml index 7a171f5..ed88e85 100644 --- a/scenarios/error-worker-http-health/deployah.yaml +++ b/scenarios/error-worker-http-health/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-worker-http-health components: worker: diff --git a/scenarios/error-worker-metrics-no-port/deployah.yaml b/scenarios/error-worker-metrics-no-port/deployah.yaml index 1cf6d8d..44386ca 100644 --- a/scenarios/error-worker-metrics-no-port/deployah.yaml +++ b/scenarios/error-worker-metrics-no-port/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-worker-metrics-no-port components: worker: diff --git a/scenarios/error-worker-port/deployah.yaml b/scenarios/error-worker-port/deployah.yaml index 0850c52..8c38901 100644 --- a/scenarios/error-worker-port/deployah.yaml +++ b/scenarios/error-worker-port/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: error-worker-port components: worker: diff --git a/scenarios/expose-apex-certmanager/deployah.yaml b/scenarios/expose-apex-certmanager/deployah.yaml index 13336b4..e2e6370 100644 --- a/scenarios/expose-apex-certmanager/deployah.yaml +++ b/scenarios/expose-apex-certmanager/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: expose-apex-certmanager components: api: diff --git a/scenarios/expose-secretname/deployah.yaml b/scenarios/expose-secretname/deployah.yaml index a457268..2aa7a18 100644 --- a/scenarios/expose-secretname/deployah.yaml +++ b/scenarios/expose-secretname/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: expose-secretname components: api: diff --git a/scenarios/expose-selfsigned/deployah.yaml b/scenarios/expose-selfsigned/deployah.yaml index ef5f8da..eeab4b0 100644 --- a/scenarios/expose-selfsigned/deployah.yaml +++ b/scenarios/expose-selfsigned/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: expose-selfsigned components: web: diff --git a/scenarios/extras-env-and-crds/deployah.yaml b/scenarios/extras-env-and-crds/deployah.yaml index 820c937..a3c766a 100644 --- a/scenarios/extras-env-and-crds/deployah.yaml +++ b/scenarios/extras-env-and-crds/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: extras-env-and-crds components: web: diff --git a/scenarios/extras-manifest/deployah.yaml b/scenarios/extras-manifest/deployah.yaml index d43e2dc..e499097 100644 --- a/scenarios/extras-manifest/deployah.yaml +++ b/scenarios/extras-manifest/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: extras-manifest components: web: diff --git a/scenarios/health-check-http/deployah.yaml b/scenarios/health-check-http/deployah.yaml index 54b7240..1f0cd63 100644 --- a/scenarios/health-check-http/deployah.yaml +++ b/scenarios/health-check-http/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: health-check-http components: api: diff --git a/scenarios/health-disabled/deployah.yaml b/scenarios/health-disabled/deployah.yaml index 41e1631..9b0817d 100644 --- a/scenarios/health-disabled/deployah.yaml +++ b/scenarios/health-disabled/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: health-disabled components: api: diff --git a/scenarios/invalid-manifest/deployah.yaml b/scenarios/invalid-manifest/deployah.yaml index 089cb86..5f82021 100644 --- a/scenarios/invalid-manifest/deployah.yaml +++ b/scenarios/invalid-manifest/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: invalid-manifest components: web: diff --git a/scenarios/multi-component/deployah.yaml b/scenarios/multi-component/deployah.yaml index f047b64..40f9cab 100644 --- a/scenarios/multi-component/deployah.yaml +++ b/scenarios/multi-component/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: multi-component components: web: diff --git a/scenarios/multi-env/deployah.yaml b/scenarios/multi-env/deployah.yaml index 370ca36..bf3f9a8 100644 --- a/scenarios/multi-env/deployah.yaml +++ b/scenarios/multi-env/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: multi-env components: web: diff --git a/scenarios/plan-after-failed-upgrade/deployah.yaml b/scenarios/plan-after-failed-upgrade/deployah.yaml index 25ba371..54bf703 100644 --- a/scenarios/plan-after-failed-upgrade/deployah.yaml +++ b/scenarios/plan-after-failed-upgrade/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-after-failed-upgrade components: web: diff --git a/scenarios/plan-command-change/before.yaml b/scenarios/plan-command-change/before.yaml index bbca0d1..4f03ac8 100644 --- a/scenarios/plan-command-change/before.yaml +++ b/scenarios/plan-command-change/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-command-change components: api: diff --git a/scenarios/plan-command-change/deployah.yaml b/scenarios/plan-command-change/deployah.yaml index 78d349b..7d52c3d 100644 --- a/scenarios/plan-command-change/deployah.yaml +++ b/scenarios/plan-command-change/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-command-change components: api: diff --git a/scenarios/plan-extras-fresh-install/deployah.yaml b/scenarios/plan-extras-fresh-install/deployah.yaml index c0fdd2a..f187dce 100644 --- a/scenarios/plan-extras-fresh-install/deployah.yaml +++ b/scenarios/plan-extras-fresh-install/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-extras-fresh-install components: web: diff --git a/scenarios/plan-fresh-install/deployah.yaml b/scenarios/plan-fresh-install/deployah.yaml index 57b51c1..f169776 100644 --- a/scenarios/plan-fresh-install/deployah.yaml +++ b/scenarios/plan-fresh-install/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-fresh-install components: web: diff --git a/scenarios/plan-hpa-change/before.yaml b/scenarios/plan-hpa-change/before.yaml index a3f41ea..8e589f8 100644 --- a/scenarios/plan-hpa-change/before.yaml +++ b/scenarios/plan-hpa-change/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-hpa-change components: api: diff --git a/scenarios/plan-hpa-change/deployah.yaml b/scenarios/plan-hpa-change/deployah.yaml index 078736a..3caf963 100644 --- a/scenarios/plan-hpa-change/deployah.yaml +++ b/scenarios/plan-hpa-change/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-hpa-change components: api: diff --git a/scenarios/plan-image-bump/before.yaml b/scenarios/plan-image-bump/before.yaml index 2ef7b9e..f06cf82 100644 --- a/scenarios/plan-image-bump/before.yaml +++ b/scenarios/plan-image-bump/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-image-bump components: web: diff --git a/scenarios/plan-image-bump/deployah.yaml b/scenarios/plan-image-bump/deployah.yaml index fa9dc5c..a131f71 100644 --- a/scenarios/plan-image-bump/deployah.yaml +++ b/scenarios/plan-image-bump/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-image-bump components: web: diff --git a/scenarios/plan-ingress-added/before.yaml b/scenarios/plan-ingress-added/before.yaml index a145d19..aceba65 100644 --- a/scenarios/plan-ingress-added/before.yaml +++ b/scenarios/plan-ingress-added/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-ingress-added components: api: diff --git a/scenarios/plan-ingress-added/deployah.yaml b/scenarios/plan-ingress-added/deployah.yaml index d5ea140..8a8f050 100644 --- a/scenarios/plan-ingress-added/deployah.yaml +++ b/scenarios/plan-ingress-added/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-ingress-added components: api: diff --git a/scenarios/plan-mixed-changes/before.yaml b/scenarios/plan-mixed-changes/before.yaml index d7950b4..8b5e6b3 100644 --- a/scenarios/plan-mixed-changes/before.yaml +++ b/scenarios/plan-mixed-changes/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-mixed-changes components: web: diff --git a/scenarios/plan-mixed-changes/deployah.yaml b/scenarios/plan-mixed-changes/deployah.yaml index 5ffd246..87b1273 100644 --- a/scenarios/plan-mixed-changes/deployah.yaml +++ b/scenarios/plan-mixed-changes/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-mixed-changes components: web: diff --git a/scenarios/plan-no-changes/before.yaml b/scenarios/plan-no-changes/before.yaml index 3767119..a84f0ef 100644 --- a/scenarios/plan-no-changes/before.yaml +++ b/scenarios/plan-no-changes/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-no-changes components: web: diff --git a/scenarios/plan-no-changes/deployah.yaml b/scenarios/plan-no-changes/deployah.yaml index 3767119..a84f0ef 100644 --- a/scenarios/plan-no-changes/deployah.yaml +++ b/scenarios/plan-no-changes/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-no-changes components: web: diff --git a/scenarios/plan-resource-added/before.yaml b/scenarios/plan-resource-added/before.yaml index f056c82..c83b20c 100644 --- a/scenarios/plan-resource-added/before.yaml +++ b/scenarios/plan-resource-added/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-resource-added components: web: diff --git a/scenarios/plan-resource-added/deployah.yaml b/scenarios/plan-resource-added/deployah.yaml index 583e556..26061ba 100644 --- a/scenarios/plan-resource-added/deployah.yaml +++ b/scenarios/plan-resource-added/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-resource-added components: web: diff --git a/scenarios/plan-resource-removed/before.yaml b/scenarios/plan-resource-removed/before.yaml index 9bbea9b..8664da1 100644 --- a/scenarios/plan-resource-removed/before.yaml +++ b/scenarios/plan-resource-removed/before.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-resource-removed components: web: diff --git a/scenarios/plan-resource-removed/deployah.yaml b/scenarios/plan-resource-removed/deployah.yaml index b4a9d3d..d45124e 100644 --- a/scenarios/plan-resource-removed/deployah.yaml +++ b/scenarios/plan-resource-removed/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: plan-resource-removed components: web: diff --git a/scenarios/plan-tasks-section/deployah.yaml b/scenarios/plan-tasks-section/deployah.yaml new file mode 100644 index 0000000..510ddfa --- /dev/null +++ b/scenarios/plan-tasks-section/deployah.yaml @@ -0,0 +1,24 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: plan-tasks-section +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small +tasks: + migrate: + from: api + "on": preDeploy + command: ["migrate", "up"] + smoke: + from: api + "on": postDeploy + command: ["curl", "-f", "http://api/health"] + backfill: + from: api + "on": manual + command: ["backfill"] +environments: + dev: {} diff --git a/scenarios/plan-tasks-section/golden.txt b/scenarios/plan-tasks-section/golden.txt new file mode 100644 index 0000000..e1eb057 --- /dev/null +++ b/scenarios/plan-tasks-section/golden.txt @@ -0,0 +1,20 @@ +Project: plan-tasks-section +Environment: dev +Release: plan-tasks-section-dev (fresh install) +Namespace: default + +Tasks: + preDeploy + migrate (timeout 5m) weight 0 + postDeploy + smoke (timeout 5m) weight 0 + manual (CLI only) + backfill +Note: preDeploy runs before other resources on a first install; the database must already be reachable. + ++ Deployment/plan-tasks-section-dev-api ++ Service/plan-tasks-section-dev-api + +Note: Helm hooks changed for this release (not shown above). + +Plan: 2 to add, 0 to change, 0 to destroy. diff --git a/scenarios/plan-tasks-section/plan-config.yaml b/scenarios/plan-tasks-section/plan-config.yaml new file mode 100644 index 0000000..1a3172a --- /dev/null +++ b/scenarios/plan-tasks-section/plan-config.yaml @@ -0,0 +1,15 @@ +freshInstall: true +hooksChanged: true + +changes: + - action: add + kind: Deployment + name: plan-tasks-section-dev-api + - action: add + kind: Service + name: plan-tasks-section-dev-api + +summary: + add: 2 + change: 0 + destroy: 0 diff --git a/scenarios/profile-basic/deployah.yaml b/scenarios/profile-basic/deployah.yaml index f7e9081..5e0dc84 100644 --- a/scenarios/profile-basic/deployah.yaml +++ b/scenarios/profile-basic/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: profile-basic components: web: diff --git a/scenarios/profile-merge/deployah.yaml b/scenarios/profile-merge/deployah.yaml index f570719..06b6525 100644 --- a/scenarios/profile-merge/deployah.yaml +++ b/scenarios/profile-merge/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: profile-merge components: api: diff --git a/scenarios/service-metrics-dedicated/deployah.yaml b/scenarios/service-metrics-dedicated/deployah.yaml index 8e8bb3e..90c08f3 100644 --- a/scenarios/service-metrics-dedicated/deployah.yaml +++ b/scenarios/service-metrics-dedicated/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: service-metrics-dedicated components: api: diff --git a/scenarios/service-metrics/deployah.yaml b/scenarios/service-metrics/deployah.yaml index 909a8d6..16cea79 100644 --- a/scenarios/service-metrics/deployah.yaml +++ b/scenarios/service-metrics/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: service-metrics components: api: diff --git a/scenarios/stateful-basic/deployah.yaml b/scenarios/stateful-basic/deployah.yaml index 0643c3e..35accfa 100644 --- a/scenarios/stateful-basic/deployah.yaml +++ b/scenarios/stateful-basic/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: stateful-basic components: db: diff --git a/scenarios/stateful-hpa/deployah.yaml b/scenarios/stateful-hpa/deployah.yaml index 74acc54..727a194 100644 --- a/scenarios/stateful-hpa/deployah.yaml +++ b/scenarios/stateful-hpa/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: stateful-hpa components: cache: diff --git a/scenarios/stateful-identity/deployah.yaml b/scenarios/stateful-identity/deployah.yaml index 3d622cd..71f61b2 100644 --- a/scenarios/stateful-identity/deployah.yaml +++ b/scenarios/stateful-identity/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: stateful-identity components: peer: diff --git a/scenarios/stateful-profile-retention/deployah.yaml b/scenarios/stateful-profile-retention/deployah.yaml index b5d0587..0b40525 100644 --- a/scenarios/stateful-profile-retention/deployah.yaml +++ b/scenarios/stateful-profile-retention/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: stateful-profile-retention components: db: diff --git a/scenarios/stateless-persistence/deployah.yaml b/scenarios/stateless-persistence/deployah.yaml index 5c43c3b..af68c50 100644 --- a/scenarios/stateless-persistence/deployah.yaml +++ b/scenarios/stateless-persistence/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: stateless-persistence components: web: diff --git a/scenarios/task-after-order/deployah.yaml b/scenarios/task-after-order/deployah.yaml new file mode 100644 index 0000000..c413cd4 --- /dev/null +++ b/scenarios/task-after-order/deployah.yaml @@ -0,0 +1,21 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: task-after-order +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small +tasks: + migrate: + from: api + "on": preDeploy + command: ["migrate", "up"] + seed: + from: api + "on": preDeploy + after: [migrate] + command: ["seed"] +environments: + dev: {} diff --git a/scenarios/task-after-order/expected/deployment-task-after-order-dev-api.yaml b/scenarios/task-after-order/expected/deployment-task-after-order-dev-api.yaml new file mode 100644 index 0000000..731cca9 --- /dev/null +++ b/scenarios/task-after-order/expected/deployment-task-after-order-dev-api.yaml @@ -0,0 +1,82 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: task-after-order + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-after-order + helm.sh/chart: api-0.1.0 + name: task-after-order-dev-api + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/name: api + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-after-order + helm.sh/chart: api-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/name: api + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: api + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default + terminationGracePeriodSeconds: 30 diff --git a/scenarios/task-after-order/expected/job-task-after-order-dev-migrate.yaml b/scenarios/task-after-order/expected/job-task-after-order-dev-migrate.yaml new file mode 100644 index 0000000..221a68e --- /dev/null +++ b/scenarios/task-after-order/expected/job-task-after-order-dev-migrate.yaml @@ -0,0 +1,51 @@ +apiVersion: batch/v1 +kind: Job +metadata: + annotations: + deployah.dev/project: task-after-order + deployah.dev/source: spec + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded + helm.sh/hook-weight: "0" + labels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-after-order + helm.sh/chart: migrate-0.1.0 + name: task-after-order-dev-migrate + namespace: default +spec: + activeDeadlineSeconds: 300 + backoffLimit: 3 + completionMode: Indexed + completions: 1 + parallelism: 1 + template: + metadata: + labels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-after-order + helm.sh/chart: migrate-0.1.0 + spec: + automountServiceAccountToken: false + containers: + - command: + - migrate + - up + image: docker.io/library/nginx:latest + imagePullPolicy: Always + name: migrate + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + restartPolicy: OnFailure diff --git a/scenarios/task-after-order/expected/job-task-after-order-dev-seed.yaml b/scenarios/task-after-order/expected/job-task-after-order-dev-seed.yaml new file mode 100644 index 0000000..570e773 --- /dev/null +++ b/scenarios/task-after-order/expected/job-task-after-order-dev-seed.yaml @@ -0,0 +1,50 @@ +apiVersion: batch/v1 +kind: Job +metadata: + annotations: + deployah.dev/project: task-after-order + deployah.dev/source: spec + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded + helm.sh/hook-weight: "1" + labels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: seed + deployah.dev/component: seed + deployah.dev/environment: dev + deployah.dev/project: task-after-order + helm.sh/chart: seed-0.1.0 + name: task-after-order-dev-seed + namespace: default +spec: + activeDeadlineSeconds: 300 + backoffLimit: 3 + completionMode: Indexed + completions: 1 + parallelism: 1 + template: + metadata: + labels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: seed + deployah.dev/component: seed + deployah.dev/environment: dev + deployah.dev/project: task-after-order + helm.sh/chart: seed-0.1.0 + spec: + automountServiceAccountToken: false + containers: + - command: + - seed + image: docker.io/library/nginx:latest + imagePullPolicy: Always + name: seed + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + restartPolicy: OnFailure diff --git a/scenarios/task-after-order/expected/service-task-after-order-dev-api.yaml b/scenarios/task-after-order/expected/service-task-after-order-dev-api.yaml new file mode 100644 index 0000000..d09daf7 --- /dev/null +++ b/scenarios/task-after-order/expected/service-task-after-order-dev-api.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: task-after-order + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-after-order + helm.sh/chart: api-0.1.0 + name: task-after-order-dev-api + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: task-after-order-dev + app.kubernetes.io/name: api + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/task-fanout-hook/deployah.yaml b/scenarios/task-fanout-hook/deployah.yaml new file mode 100644 index 0000000..4934729 --- /dev/null +++ b/scenarios/task-fanout-hook/deployah.yaml @@ -0,0 +1,17 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: task-fanout-hook +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small +tasks: + migrate: + from: api + "on": preDeploy + command: ["true"] + fanout: 4 +environments: + dev: {} diff --git a/scenarios/task-fanout-hook/expected/deployment-task-fanout-hook-dev-api.yaml b/scenarios/task-fanout-hook/expected/deployment-task-fanout-hook-dev-api.yaml new file mode 100644 index 0000000..8b02822 --- /dev/null +++ b/scenarios/task-fanout-hook/expected/deployment-task-fanout-hook-dev-api.yaml @@ -0,0 +1,82 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: task-fanout-hook + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-fanout-hook + helm.sh/chart: api-0.1.0 + name: task-fanout-hook-dev-api + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/name: api + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-fanout-hook + helm.sh/chart: api-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/name: api + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: api + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default + terminationGracePeriodSeconds: 30 diff --git a/scenarios/task-fanout-hook/expected/job-task-fanout-hook-dev-migrate.yaml b/scenarios/task-fanout-hook/expected/job-task-fanout-hook-dev-migrate.yaml new file mode 100644 index 0000000..8fec93e --- /dev/null +++ b/scenarios/task-fanout-hook/expected/job-task-fanout-hook-dev-migrate.yaml @@ -0,0 +1,50 @@ +apiVersion: batch/v1 +kind: Job +metadata: + annotations: + deployah.dev/project: task-fanout-hook + deployah.dev/source: spec + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded + helm.sh/hook-weight: "0" + labels: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-fanout-hook + helm.sh/chart: migrate-0.1.0 + name: task-fanout-hook-dev-migrate + namespace: default +spec: + activeDeadlineSeconds: 300 + backoffLimit: 3 + completionMode: Indexed + completions: 4 + parallelism: 1 + template: + metadata: + labels: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-fanout-hook + helm.sh/chart: migrate-0.1.0 + spec: + automountServiceAccountToken: false + containers: + - command: + - "true" + image: docker.io/library/nginx:latest + imagePullPolicy: Always + name: migrate + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + restartPolicy: OnFailure diff --git a/scenarios/task-fanout-hook/expected/service-task-fanout-hook-dev-api.yaml b/scenarios/task-fanout-hook/expected/service-task-fanout-hook-dev-api.yaml new file mode 100644 index 0000000..939b213 --- /dev/null +++ b/scenarios/task-fanout-hook/expected/service-task-fanout-hook-dev-api.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: task-fanout-hook + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-fanout-hook + helm.sh/chart: api-0.1.0 + name: task-fanout-hook-dev-api + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: task-fanout-hook-dev + app.kubernetes.io/name: api + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/task-fanout-manual/deployah.yaml b/scenarios/task-fanout-manual/deployah.yaml new file mode 100644 index 0000000..0f150ab --- /dev/null +++ b/scenarios/task-fanout-manual/deployah.yaml @@ -0,0 +1,19 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: task-fanout-manual +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small +tasks: + backfill: + from: api + "on": manual + command: ["backfill"] + fanout: + count: 4 + parallelism: 2 +environments: + dev: {} diff --git a/scenarios/task-fanout-manual/expected/deployment-task-fanout-manual-dev-api.yaml b/scenarios/task-fanout-manual/expected/deployment-task-fanout-manual-dev-api.yaml new file mode 100644 index 0000000..7e7ea72 --- /dev/null +++ b/scenarios/task-fanout-manual/expected/deployment-task-fanout-manual-dev-api.yaml @@ -0,0 +1,82 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: task-fanout-manual + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-fanout-manual-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-fanout-manual + helm.sh/chart: api-0.1.0 + name: task-fanout-manual-dev-api + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: task-fanout-manual-dev + app.kubernetes.io/name: api + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: task-fanout-manual-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-fanout-manual + helm.sh/chart: api-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: task-fanout-manual-dev + app.kubernetes.io/name: api + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: api + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default + terminationGracePeriodSeconds: 30 diff --git a/scenarios/task-fanout-manual/expected/service-task-fanout-manual-dev-api.yaml b/scenarios/task-fanout-manual/expected/service-task-fanout-manual-dev-api.yaml new file mode 100644 index 0000000..ee99659 --- /dev/null +++ b/scenarios/task-fanout-manual/expected/service-task-fanout-manual-dev-api.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: task-fanout-manual + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-fanout-manual-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-fanout-manual + helm.sh/chart: api-0.1.0 + name: task-fanout-manual-dev-api + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: task-fanout-manual-dev + app.kubernetes.io/name: api + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/task-from-env-overlay/deployah.yaml b/scenarios/task-from-env-overlay/deployah.yaml new file mode 100644 index 0000000..7ba9571 --- /dev/null +++ b/scenarios/task-from-env-overlay/deployah.yaml @@ -0,0 +1,22 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: task-from-env-overlay +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small + env: + DATABASE_URL: postgres://db + LOG: info +tasks: + migrate: + from: api + "on": preDeploy + command: ["migrate", "up"] + env: + LOG: debug + EXTRA: "1" +environments: + dev: {} diff --git a/scenarios/task-from-env-overlay/expected/deployment-task-from-env-overlay-dev-api.yaml b/scenarios/task-from-env-overlay/expected/deployment-task-from-env-overlay-dev-api.yaml new file mode 100644 index 0000000..d088923 --- /dev/null +++ b/scenarios/task-from-env-overlay/expected/deployment-task-from-env-overlay-dev-api.yaml @@ -0,0 +1,82 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: task-from-env-overlay + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-from-env-overlay + helm.sh/chart: api-0.1.0 + name: task-from-env-overlay-dev-api + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/name: api + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-from-env-overlay + helm.sh/chart: api-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/name: api + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: api + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default + terminationGracePeriodSeconds: 30 diff --git a/scenarios/task-from-env-overlay/expected/job-task-from-env-overlay-dev-migrate.yaml b/scenarios/task-from-env-overlay/expected/job-task-from-env-overlay-dev-migrate.yaml new file mode 100644 index 0000000..9e911a8 --- /dev/null +++ b/scenarios/task-from-env-overlay/expected/job-task-from-env-overlay-dev-migrate.yaml @@ -0,0 +1,58 @@ +apiVersion: batch/v1 +kind: Job +metadata: + annotations: + deployah.dev/project: task-from-env-overlay + deployah.dev/source: spec + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded + helm.sh/hook-weight: "0" + labels: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-from-env-overlay + helm.sh/chart: migrate-0.1.0 + name: task-from-env-overlay-dev-migrate + namespace: default +spec: + activeDeadlineSeconds: 300 + backoffLimit: 3 + completionMode: Indexed + completions: 1 + parallelism: 1 + template: + metadata: + labels: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-from-env-overlay + helm.sh/chart: migrate-0.1.0 + spec: + automountServiceAccountToken: false + containers: + - command: + - migrate + - up + env: + - name: DATABASE_URL + value: postgres://db + - name: EXTRA + value: "1" + - name: LOG + value: debug + image: docker.io/library/nginx:latest + imagePullPolicy: Always + name: migrate + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + restartPolicy: OnFailure diff --git a/scenarios/task-from-env-overlay/expected/service-task-from-env-overlay-dev-api.yaml b/scenarios/task-from-env-overlay/expected/service-task-from-env-overlay-dev-api.yaml new file mode 100644 index 0000000..2aadfad --- /dev/null +++ b/scenarios/task-from-env-overlay/expected/service-task-from-env-overlay-dev-api.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: task-from-env-overlay + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-from-env-overlay + helm.sh/chart: api-0.1.0 + name: task-from-env-overlay-dev-api + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: task-from-env-overlay-dev + app.kubernetes.io/name: api + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/task-manual-omitted/deployah.yaml b/scenarios/task-manual-omitted/deployah.yaml new file mode 100644 index 0000000..ae6f830 --- /dev/null +++ b/scenarios/task-manual-omitted/deployah.yaml @@ -0,0 +1,16 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: task-manual-omitted +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small +tasks: + backfill: + from: api + "on": manual + command: ["backfill"] +environments: + dev: {} diff --git a/scenarios/task-manual-omitted/expected/deployment-task-manual-omitted-dev-api.yaml b/scenarios/task-manual-omitted/expected/deployment-task-manual-omitted-dev-api.yaml new file mode 100644 index 0000000..d347f58 --- /dev/null +++ b/scenarios/task-manual-omitted/expected/deployment-task-manual-omitted-dev-api.yaml @@ -0,0 +1,82 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: task-manual-omitted + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-manual-omitted-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-manual-omitted + helm.sh/chart: api-0.1.0 + name: task-manual-omitted-dev-api + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: task-manual-omitted-dev + app.kubernetes.io/name: api + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: task-manual-omitted-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-manual-omitted + helm.sh/chart: api-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: task-manual-omitted-dev + app.kubernetes.io/name: api + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: api + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default + terminationGracePeriodSeconds: 30 diff --git a/scenarios/task-manual-omitted/expected/service-task-manual-omitted-dev-api.yaml b/scenarios/task-manual-omitted/expected/service-task-manual-omitted-dev-api.yaml new file mode 100644 index 0000000..c3a3848 --- /dev/null +++ b/scenarios/task-manual-omitted/expected/service-task-manual-omitted-dev-api.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: task-manual-omitted + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-manual-omitted-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-manual-omitted + helm.sh/chart: api-0.1.0 + name: task-manual-omitted-dev-api + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: task-manual-omitted-dev + app.kubernetes.io/name: api + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/task-pre-post-hooks/deployah.yaml b/scenarios/task-pre-post-hooks/deployah.yaml new file mode 100644 index 0000000..0f9c6e7 --- /dev/null +++ b/scenarios/task-pre-post-hooks/deployah.yaml @@ -0,0 +1,22 @@ +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 +project: task-pre-post-hooks +components: + api: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small + env: + DATABASE_URL: postgres://db +tasks: + migrate: + from: api + "on": preDeploy + command: ["migrate", "up"] + smoke: + from: api + "on": postDeploy + command: ["curl", "-f", "http://api/health"] +environments: + dev: {} diff --git a/scenarios/task-pre-post-hooks/expected/deployment-task-pre-post-hooks-dev-api.yaml b/scenarios/task-pre-post-hooks/expected/deployment-task-pre-post-hooks-dev-api.yaml new file mode 100644 index 0000000..26f8cd0 --- /dev/null +++ b/scenarios/task-pre-post-hooks/expected/deployment-task-pre-post-hooks-dev-api.yaml @@ -0,0 +1,82 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: task-pre-post-hooks + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-pre-post-hooks + helm.sh/chart: api-0.1.0 + name: task-pre-post-hooks-dev-api + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/name: api + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-pre-post-hooks + helm.sh/chart: api-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/name: api + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: api + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default + terminationGracePeriodSeconds: 30 diff --git a/scenarios/task-pre-post-hooks/expected/job-task-pre-post-hooks-dev-migrate.yaml b/scenarios/task-pre-post-hooks/expected/job-task-pre-post-hooks-dev-migrate.yaml new file mode 100644 index 0000000..ffe9fb0 --- /dev/null +++ b/scenarios/task-pre-post-hooks/expected/job-task-pre-post-hooks-dev-migrate.yaml @@ -0,0 +1,54 @@ +apiVersion: batch/v1 +kind: Job +metadata: + annotations: + deployah.dev/project: task-pre-post-hooks + deployah.dev/source: spec + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded + helm.sh/hook-weight: "0" + labels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-pre-post-hooks + helm.sh/chart: migrate-0.1.0 + name: task-pre-post-hooks-dev-migrate + namespace: default +spec: + activeDeadlineSeconds: 300 + backoffLimit: 3 + completionMode: Indexed + completions: 1 + parallelism: 1 + template: + metadata: + labels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: migrate + deployah.dev/component: migrate + deployah.dev/environment: dev + deployah.dev/project: task-pre-post-hooks + helm.sh/chart: migrate-0.1.0 + spec: + automountServiceAccountToken: false + containers: + - command: + - migrate + - up + env: + - name: DATABASE_URL + value: postgres://db + image: docker.io/library/nginx:latest + imagePullPolicy: Always + name: migrate + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + restartPolicy: OnFailure diff --git a/scenarios/task-pre-post-hooks/expected/job-task-pre-post-hooks-dev-smoke.yaml b/scenarios/task-pre-post-hooks/expected/job-task-pre-post-hooks-dev-smoke.yaml new file mode 100644 index 0000000..df4bbc3 --- /dev/null +++ b/scenarios/task-pre-post-hooks/expected/job-task-pre-post-hooks-dev-smoke.yaml @@ -0,0 +1,55 @@ +apiVersion: batch/v1 +kind: Job +metadata: + annotations: + deployah.dev/project: task-pre-post-hooks + deployah.dev/source: spec + helm.sh/hook: post-install,post-upgrade + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded + helm.sh/hook-weight: "0" + labels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: smoke + deployah.dev/component: smoke + deployah.dev/environment: dev + deployah.dev/project: task-pre-post-hooks + helm.sh/chart: smoke-0.1.0 + name: task-pre-post-hooks-dev-smoke + namespace: default +spec: + activeDeadlineSeconds: 300 + backoffLimit: 3 + completionMode: Indexed + completions: 1 + parallelism: 1 + template: + metadata: + labels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: smoke + deployah.dev/component: smoke + deployah.dev/environment: dev + deployah.dev/project: task-pre-post-hooks + helm.sh/chart: smoke-0.1.0 + spec: + automountServiceAccountToken: false + containers: + - command: + - curl + - -f + - http://api/health + env: + - name: DATABASE_URL + value: postgres://db + image: docker.io/library/nginx:latest + imagePullPolicy: Always + name: smoke + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + restartPolicy: OnFailure diff --git a/scenarios/task-pre-post-hooks/expected/service-task-pre-post-hooks-dev-api.yaml b/scenarios/task-pre-post-hooks/expected/service-task-pre-post-hooks-dev-api.yaml new file mode 100644 index 0000000..66d5a90 --- /dev/null +++ b/scenarios/task-pre-post-hooks/expected/service-task-pre-post-hooks-dev-api.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: task-pre-post-hooks + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: api + deployah.dev/component: api + deployah.dev/environment: dev + deployah.dev/project: task-pre-post-hooks + helm.sh/chart: api-0.1.0 + name: task-pre-post-hooks-dev-api + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: task-pre-post-hooks-dev + app.kubernetes.io/name: api + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/worker-exec-health/deployah.yaml b/scenarios/worker-exec-health/deployah.yaml index 3977511..aad21c7 100644 --- a/scenarios/worker-exec-health/deployah.yaml +++ b/scenarios/worker-exec-health/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: worker-exec-health components: worker: diff --git a/scenarios/worker-hpa/deployah.yaml b/scenarios/worker-hpa/deployah.yaml index 494eb89..5725472 100644 --- a/scenarios/worker-hpa/deployah.yaml +++ b/scenarios/worker-hpa/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: worker-hpa components: worker: diff --git a/scenarios/worker-metrics/deployah.yaml b/scenarios/worker-metrics/deployah.yaml index 9678e6d..b1ef359 100644 --- a/scenarios/worker-metrics/deployah.yaml +++ b/scenarios/worker-metrics/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: worker-metrics components: worker: diff --git a/scenarios/worker-stateful/deployah.yaml b/scenarios/worker-stateful/deployah.yaml index 3a7c03d..52e6c2f 100644 --- a/scenarios/worker-stateful/deployah.yaml +++ b/scenarios/worker-stateful/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: worker-stateful components: worker: diff --git a/scenarios/worker-stateless/deployah.yaml b/scenarios/worker-stateless/deployah.yaml index ab9995d..a57b092 100644 --- a/scenarios/worker-stateless/deployah.yaml +++ b/scenarios/worker-stateless/deployah.yaml @@ -1,5 +1,5 @@ -# $schema: ../../internal/spec/schema/v1-alpha.4/manifest.json -apiVersion: v1-alpha.4 +# $schema: ../../internal/spec/schema/v1-alpha.5/manifest.json +apiVersion: v1-alpha.5 project: worker-stateless components: worker: