From fe406fdc7e4ee39d974bd439529191c7598b3954 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 21 Jul 2026 10:28:58 -0400 Subject: [PATCH 01/17] docs(deploy): two-phase deploy_component, revert_component, stage/activate CLI Document the two-phase deploy behavior and the new surface from HarperFast/harper#1849: - deploy_component: cluster-wide stage -> barrier -> activate; new properties activate:false (stage-and-stop, returns a staged deployment_id), deployment_id (activate a previously-staged deployment), revert_on_failure, two_phase, and the ignore_replication_errors / deployment_timeout knobs. - revert_component: fast cluster-wide rollback to the retained previous version. - Deployment lifecycle: stage/activate phase names, staged/staging/activating/ rolled_back statuses, and staged-build retention (deployment_stagingRetention_maxCount). - CLI: harper stage / activate / revert verbs (aliases + examples). Companion to HarperFast/harper#1849 (still a draft). Co-Authored-By: Claude Opus 4.8 --- reference/cli/operations-api-commands.md | 173 +++++++++++++---------- reference/operations-api/operations.md | 99 +++++++++---- 2 files changed, 171 insertions(+), 101 deletions(-) diff --git a/reference/cli/operations-api-commands.md b/reference/cli/operations-api-commands.md index e893f42ec..7c9255308 100644 --- a/reference/cli/operations-api-commands.md +++ b/reference/cli/operations-api-commands.md @@ -34,80 +34,84 @@ The following operations are available through the CLI. Operations that require This is just a brief overview of all operations available as CLI commands. Review the respective operation documentation for more information on available arguments and expected behavior. Keep in mind that all operations options are converted to CLI arguments in the same way (using `snake_case`). ::: -| Operation | Description | Category | Available Since | -| -------------------------------- | ------------------------------------- | ---------------------------------------------------------------------- | --------------- | -| `describe_table` | Describe table structure and metadata | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `describe_all` | Describe all databases and tables | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `describe_database` | Describe database structure | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_database` | Create a new database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_database` | Delete a database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_table` | Create a new table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_table` | Delete a table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_attribute` | Create a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_attribute` | Delete a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `search_by_id` | Search records by ID | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `search_by_value` | Search records by attribute value | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `insert` | Insert new records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `update` | Update existing records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `upsert` | Insert or update records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `delete` | Delete records | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `sql` | Execute SQL queries | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `csv_file_load` | Load data from CSV file | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `csv_url_load` | Load data from CSV URL | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `list_users` | List all users | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `add_user` | Create a new user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `alter_user` | Modify user properties | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `drop_user` | Delete a user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `list_roles` | List all roles | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `drop_role` | Delete a role | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `create_csr` | Create certificate signing request | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `sign_certificate` | Sign a certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `list_certificates` | List SSL/TLS certificates | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `add_certificate` | Add SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `remove_certificate` | Remove SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `add_ssh_key` | Add SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `get_ssh_key` | Get SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.7.2 | -| `update_ssh_key` | Update SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `delete_ssh_key` | Delete SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `list_ssh_keys` | List all SSH keys | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `set_ssh_known_hosts` | Set SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `get_ssh_known_hosts` | Get SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `cluster_get_routes` | Get cluster routing information | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `cluster_network` | Get cluster network status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `cluster_status` | Get cluster status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `remove_node` | Remove node from cluster | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `add_component` | Add a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `deploy_component` | Deploy a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `deploy` (alias) | Alias for `deploy_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `package_component` | Package a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `package` (alias) | Alias for `package_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `drop_component` | Remove a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `get_components` | List all components | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `get_component_file` | Get component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `set_component_file` | Set component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `install_node_modules` | Install Node.js dependencies | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `set_configuration` | Update configuration settings | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | -| `get_configuration` | Get current configuration | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | -| `create_authentication_tokens` | Create authentication tokens | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | -| `refresh_operation_token` | Refresh operation token | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | -| `restart_service` | Restart Harper service | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `restart` | Restart Harper instance | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `system_information` | Get system information | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `registration_info` | Get registration information | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `get_fingerprint` | Get instance fingerprint | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `set_license` | Set license key | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `get_usage_licenses` | Get usage and license info | [Licensing](../operations-api/operations.md#registration--licensing) | v4.7.3 | -| `get_job` | Get job status | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | -| `search_jobs_by_start_date` | Search jobs by start date | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | -| `read_log` | Read application logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `read_transaction_log` | Read transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `read_audit_log` | Read audit logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `delete_transaction_logs_before` | Delete old transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `purge_stream` | Purge streaming data | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | -| `delete_records_before` | Delete old records | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | -| `get_status` | Get custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | -| `set_status` | Set custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | -| `clear_status` | Clear custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| Operation | Description | Category | Available Since | +| -------------------------------- | --------------------------------------------- | ---------------------------------------------------------------------- | --------------- | +| `describe_table` | Describe table structure and metadata | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `describe_all` | Describe all databases and tables | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `describe_database` | Describe database structure | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_database` | Create a new database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_database` | Delete a database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_table` | Create a new table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_table` | Delete a table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_attribute` | Create a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_attribute` | Delete a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `search_by_id` | Search records by ID | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `search_by_value` | Search records by attribute value | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `insert` | Insert new records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `update` | Update existing records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `upsert` | Insert or update records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `delete` | Delete records | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `sql` | Execute SQL queries | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `csv_file_load` | Load data from CSV file | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `csv_url_load` | Load data from CSV URL | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `list_users` | List all users | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | +| `add_user` | Create a new user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | +| `alter_user` | Modify user properties | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | +| `drop_user` | Delete a user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | +| `list_roles` | List all roles | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | +| `drop_role` | Delete a role | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | +| `create_csr` | Create certificate signing request | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `sign_certificate` | Sign a certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `list_certificates` | List SSL/TLS certificates | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `add_certificate` | Add SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `remove_certificate` | Remove SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `add_ssh_key` | Add SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `get_ssh_key` | Get SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.7.2 | +| `update_ssh_key` | Update SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `delete_ssh_key` | Delete SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `list_ssh_keys` | List all SSH keys | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `set_ssh_known_hosts` | Set SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `get_ssh_known_hosts` | Get SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `cluster_get_routes` | Get cluster routing information | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `cluster_network` | Get cluster network status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `cluster_status` | Get cluster status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `remove_node` | Remove node from cluster | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `add_component` | Add a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `deploy_component` | Deploy a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `deploy` (alias) | Alias for `deploy_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `stage` (alias) | Stage a component without going live | [Components](../operations-api/operations.md#deploy_component) | v5.2.0 | +| `activate` (alias) | Activate a previously-staged deploy | [Components](../operations-api/operations.md#deploy_component) | v5.2.0 | +| `revert_component` | Roll a component back to its previous version | [Components](../operations-api/operations.md#revert_component) | v5.2.0 | +| `revert` (alias) | Alias for `revert_component` | [Components](../operations-api/operations.md#revert_component) | v5.2.0 | +| `package_component` | Package a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `package` (alias) | Alias for `package_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `drop_component` | Remove a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `get_components` | List all components | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `get_component_file` | Get component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `set_component_file` | Set component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `install_node_modules` | Install Node.js dependencies | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `set_configuration` | Update configuration settings | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | +| `get_configuration` | Get current configuration | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | +| `create_authentication_tokens` | Create authentication tokens | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | +| `refresh_operation_token` | Refresh operation token | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | +| `restart_service` | Restart Harper service | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `restart` | Restart Harper instance | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `system_information` | Get system information | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `registration_info` | Get registration information | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `get_fingerprint` | Get instance fingerprint | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `set_license` | Set license key | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `get_usage_licenses` | Get usage and license info | [Licensing](../operations-api/operations.md#registration--licensing) | v4.7.3 | +| `get_job` | Get job status | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | +| `search_jobs_by_start_date` | Search jobs by start date | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | +| `read_log` | Read application logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `read_transaction_log` | Read transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `read_audit_log` | Read audit logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `delete_transaction_logs_before` | Delete old transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `purge_stream` | Purge streaming data | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | +| `delete_records_before` | Delete old records | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | +| `get_status` | Get custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| `set_status` | Set custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| `clear_status` | Clear custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | ### Command Aliases @@ -115,6 +119,9 @@ The following aliases are available for convenience: - `deploy` → `deploy_component` - `package` → `package_component` +- `stage` → `deploy_component` with `activate=false` (build + upload the component cluster-wide without going live; prints a `deployment_id`) +- `activate` → `deploy_component` with a `deployment_id` (take a previously staged deployment live; requires `deployment_id`) +- `revert` → `revert_component` For detailed parameter information for each operation, see the [Operations API documentation](../operations-api/operations.md). @@ -212,6 +219,24 @@ harper get_components harper deploy project=my-app package=https://github.com/user/repo ``` +**Stage a component without going live** (builds + uploads across the cluster and prints a `deployment_id`): + +```bash +harper stage project=my-app +``` + +**Activate a previously-staged deployment** (takes it live; requires the `deployment_id` from `stage`): + +```bash +harper activate project=my-app deployment_id=a3f8c2d1... +``` + +**Roll a component back to its previous version**: + +```bash +harper revert project=my-app +``` + :::tip For more information on components and applications, see the [Components Reference](../components/overview.md). ::: diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index e958e8817..7b23b684c 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -602,12 +602,25 @@ Detailed documentation: [Components Overview](../components/overview.md) Deploys a component. The `package` option accepts any valid NPM reference including GitHub repos (`HarperDB/app#semver:v1.0.0`), tarballs, or NPM packages. The `payload` option accepts a base64-encoded tar string from `package_component`. Supports `"replicated": true` and `"restart": true` or `"restart": "rolling"`. +Across a cluster, `deploy_component` runs in two phases so a deploy is all-or-nothing at go-live: + +1. **Stage** — the incoming version is downloaded/packed, extracted, and `npm install`ed into a hidden staging directory on **every** node, without touching the live component. +2. **Activate** — only after every node reports a successful stage does any node atomically swap the staged copy into the live path and restart. + +If a node can't fetch the package or fails `npm install`, it fails during staging and the live component is left untouched on every node, rather than leaving part of the cluster half-updated. The request and response shape are unchanged; the two phases are internal. + Additional parameters: - `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`) - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) +- `activate` — set to `false` to **stage only** and stop before go-live. The build is prepared and verified on every node and the response returns a `deployment_id` in a `staged` state; nothing goes live. Activate it later by calling `deploy_component` again with that `deployment_id` (see below). Useful for pre-staging a release and flipping it live in a separate, fast step. +- `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). No new payload is fetched or installed; the already-staged build is swapped live cluster-wide. `project` is still required. +- `revert_on_failure` — if the activate phase fails on some nodes (leaving the cluster split across versions), automatically swap the nodes that did activate back to their previous version so the cluster reconverges. Off by default. +- `ignore_replication_errors` — treat replication/peer failures as non-fatal (best-effort deploy to a partially-available cluster). This also opts out of the stage barrier. +- `deployment_timeout` — per-deploy budget (ms) for peers to receive the replicated deployment row; defaults to 120000. +- `two_phase` — set to `false` to force the legacy single-phase (in-place) deploy instead of stage-then-activate. -The response includes a `deployment_id` that can be used to query the deployment record: +A normal deploy (stage + activate): ```json { @@ -624,26 +637,58 @@ Response: ```json { "deployment_id": "a3f8c2d1...", - "message": "Component deployed successfully" + "message": "Successfully deployed: my-app" } ``` +Stage now, activate later: + +```json +{ "operation": "deploy_component", "project": "my-app", "package": "my-org/my-app#semver:v1.2.3", "activate": false } +``` + +```json +{ "deployment_id": "a3f8c2d1...", "project": "my-app", "staged": true, "message": "Staged component: my-app" } +``` + +```json +{ "operation": "deploy_component", "project": "my-app", "deployment_id": "a3f8c2d1...", "restart": "rolling" } +``` + +### `revert_component` + +Swaps a component's live version back to its **retained previous version** across the cluster, then restarts. Every `deploy_component` activation retains the version it replaced (one previous version is kept per component), so `revert_component` is a fast rollback that does not re-fetch or re-install. The swap is bidirectional — reverting a revert rolls forward again. + +This supports customer-driven rollback: deploy a new version, run your own health checks against it, and revert if you are not happy — even when the cluster otherwise looks healthy. + +```json +{ + "operation": "revert_component", + "project": "my-app", + "restart": "rolling" +} +``` + +`revert_component` fails with "no previous version is retained" for a component that has only ever been deployed once (nothing to revert to). + ### Deployment Operations -Harper records every `deploy_component` call in the `system.hdb_deployment` table, capturing the full lifecycle of a deployment including phase transitions (prepare → load → replicate → restart → success/failed), per-node outcomes, and a bounded event log of install output. +Harper records every `deploy_component` call in the `system.hdb_deployment` table, capturing the full lifecycle of a deployment including phase transitions (`stage` → `activate` → `restart` → `success`/`failed`, or `prepare` → `replicate` → `restart` on the legacy single-phase path), per-node outcomes, and a bounded event log of install output. + +**Staged-build retention.** Deployments staged with `activate: false` leave their built files on disk until they are activated. Harper keeps only the most recent staged builds per component (default 5, configurable via the `deployment_stagingRetention_maxCount` configuration option); older not-yet-activated staged builds are evicted automatically when a new stage lands. Activating a `deployment_id` that has aged out of this window fails with "no staged build found." The `hdb_deployment` records themselves are retained as an audit trail; large payload tarballs are reclaimed automatically after a successful deploy (see `deployment_payloadRetention_maxSize`). ### `list_deployments` Returns a list of deployment records, newest first. All filter parameters are optional. -| Parameter | Type | Description | -| --------- | ------ | ------------------------------------------------ | -| `project` | string | Filter to a specific component project | -| `status` | string | Filter by status: `pending`, `success`, `failed` | -| `since` | number | Start of time range (Unix timestamp ms) | -| `until` | number | End of time range (Unix timestamp ms) | -| `limit` | number | Maximum number of results (default: 100) | -| `offset` | number | Pagination offset | +| Parameter | Type | Description | +| --------- | ------ | ------------------------------------------------------------------------- | +| `project` | string | Filter to a specific component project | +| `status` | string | Filter by status: `pending`, `staged`, `success`, `failed`, `rolled_back` | +| `since` | number | Start of time range (Unix timestamp ms) | +| `until` | number | End of time range (Unix timestamp ms) | +| `limit` | number | Maximum number of results (default: 100) | +| `offset` | number | Pagination offset | ```json { @@ -669,22 +714,22 @@ Returns a single deployment record by `deployment_id`. When called on an in-prog The deployment record includes: -| Field | Description | -| -------------------- | ----------------------------------------------------------------------- | -| `deployment_id` | Unique identifier (content hash) | -| `project` | Component project name | -| `package_identifier` | Package reference or `payload` for tar uploads | -| `status` | `pending`, `success`, `failed`, or `rolled_back` | -| `phase` | Current lifecycle phase: `prepare`, `load`, `replicate`, `restart` | -| `event_log` | Bounded log of install output and phase transitions (up to 200 entries) | -| `peer_results` | Per-node outcome map for replicated deployments | -| `payload_hash` | SHA-256 hash of the deployment tarball | -| `payload_size` | Byte size of the deployment tarball | -| `started_at` | Timestamp when deployment began | -| `completed_at` | Timestamp when deployment finished | -| `user` | User who initiated the deployment | -| `rollback_of` | `deployment_id` of the deployment this rolls back, if applicable | -| `error` | Error message for failed deployments | +| Field | Description | +| -------------------- | -------------------------------------------------------------------------------------------------- | +| `deployment_id` | Unique identifier (content hash) | +| `project` | Component project name | +| `package_identifier` | Package reference or `payload` for tar uploads | +| `status` | `pending`, `staging`, `staged`, `activating`, `success`, `failed`, or `rolled_back` | +| `phase` | Current lifecycle phase: `stage`, `load`, `activate`, `restart` (or legacy `prepare`, `replicate`) | +| `event_log` | Bounded log of install output and phase transitions (up to 200 entries) | +| `peer_results` | Per-node outcome map for replicated deployments | +| `payload_hash` | SHA-256 hash of the deployment tarball | +| `payload_size` | Byte size of the deployment tarball | +| `started_at` | Timestamp when deployment began | +| `completed_at` | Timestamp when deployment finished | +| `user` | User who initiated the deployment | +| `rollback_of` | `deployment_id` of the deployment this rolls back, if applicable | +| `error` | Error message for failed deployments | ### `get_deployment_payload` From acbb3af4ce155d31f9a206d00c441252a168625c Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 21 Jul 2026 10:38:57 -0400 Subject: [PATCH 02/17] =?UTF-8?q?docs(deploy):=20address=20review=20?= =?UTF-8?q?=E2=80=94=20category=20links=20to=20#components,=20label=20stag?= =?UTF-8?q?e/activate=20example?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- reference/cli/operations-api-commands.md | 8 ++++---- reference/operations-api/operations.md | 6 ++++++ 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/reference/cli/operations-api-commands.md b/reference/cli/operations-api-commands.md index 7c9255308..032d45ae4 100644 --- a/reference/cli/operations-api-commands.md +++ b/reference/cli/operations-api-commands.md @@ -79,10 +79,10 @@ This is just a brief overview of all operations available as CLI commands. Revie | `add_component` | Add a component | [Components](../operations-api/operations.md#components) | v4.3.0 | | `deploy_component` | Deploy a component | [Components](../operations-api/operations.md#components) | v4.3.0 | | `deploy` (alias) | Alias for `deploy_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `stage` (alias) | Stage a component without going live | [Components](../operations-api/operations.md#deploy_component) | v5.2.0 | -| `activate` (alias) | Activate a previously-staged deploy | [Components](../operations-api/operations.md#deploy_component) | v5.2.0 | -| `revert_component` | Roll a component back to its previous version | [Components](../operations-api/operations.md#revert_component) | v5.2.0 | -| `revert` (alias) | Alias for `revert_component` | [Components](../operations-api/operations.md#revert_component) | v5.2.0 | +| `stage` (alias) | Stage a component without going live | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `activate` (alias) | Activate a previously-staged deploy | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `revert_component` | Roll a component back to its previous version | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `revert` (alias) | Alias for `revert_component` | [Components](../operations-api/operations.md#components) | v5.2.0 | | `package_component` | Package a component | [Components](../operations-api/operations.md#components) | v4.3.0 | | `package` (alias) | Alias for `package_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | | `drop_component` | Remove a component | [Components](../operations-api/operations.md#components) | v4.3.0 | diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 7b23b684c..a428bfaaa 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -643,14 +643,20 @@ Response: Stage now, activate later: +**Stage request** (`activate: false`): + ```json { "operation": "deploy_component", "project": "my-app", "package": "my-org/my-app#semver:v1.2.3", "activate": false } ``` +**Stage response** (nothing is live yet; note the `staged` marker and the `deployment_id`): + ```json { "deployment_id": "a3f8c2d1...", "project": "my-app", "staged": true, "message": "Staged component: my-app" } ``` +**Activation request** (take the staged build live by passing its `deployment_id`): + ```json { "operation": "deploy_component", "project": "my-app", "deployment_id": "a3f8c2d1...", "restart": "rolling" } ``` From 4da3f9b06f846fc01e257507b34d37cf12dce5bd Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Wed, 29 Jul 2026 14:00:39 -0400 Subject: [PATCH 03/17] docs: add_ssh_key server-side keygen, payload retention count, status values MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documents harper-pro#594 and syncs with harper#1849's latest behavior. add_ssh_key `generate: true` (harper-pro#594): - Harper mints the ed25519 keypair on the node and returns only `public_key`, so the private key is never carried in a request body, shell history, or CI log. Includes the response shape and the `harper:` comment. - `key` and `generate` are mutually exclusive; ssh-keygen must be on PATH. - Notes that `public_key` is returned ONLY on the generating call — Harper does not retain it, and update_ssh_key requires a key you supply (it cannot mint one), so the recovery path is delete_ssh_key + add_ssh_key generate again. - CLI example added alongside the deploy/stage/activate/revert examples. Payload retention (harper#1849): - Splits the old one-line aside into its own subsection with a table covering both bounds: payloadRetention_maxSize (10 MiB, bounds a single payload) and the new payloadRetention_maxCount (default 1, bounds how many are kept per project). Explains why the default is conservative, that rows are never deleted, and that a reclaimed payload reports payload_blob_present: false. Also: - revert_on_failure / ignore_replication_errors now note they apply to a `deployment_id` activate too, matching the peer-failure gate added to that path. - list_deployments `status` listed only 5 of 13 real values; replaced with terminal vs in-flight groupings. - New note on restartRequired: deploying a brand-new component without a restart marks one as required (per node), while redeploying a live component does not. --- reference/cli/operations-api-commands.md | 6 ++ reference/operations-api/operations.md | 75 ++++++++++++++++++++---- 2 files changed, 71 insertions(+), 10 deletions(-) diff --git a/reference/cli/operations-api-commands.md b/reference/cli/operations-api-commands.md index 032d45ae4..d44b74cfa 100644 --- a/reference/cli/operations-api-commands.md +++ b/reference/cli/operations-api-commands.md @@ -237,6 +237,12 @@ harper activate project=my-app deployment_id=a3f8c2d1... harper revert project=my-app ``` +**Mint a deploy key for a private repository** (Harper generates the ed25519 keypair server-side and returns only the public half — register it with your git host): + +```bash +harper add_ssh_key name=my-key generate=true host=my-key.github.com hostname=github.com +``` + :::tip For more information on components and applications, see the [Components Reference](../components/overview.md). ::: diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 89dfeac4d..a52e28156 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -609,6 +609,10 @@ Across a cluster, `deploy_component` runs in two phases so a deploy is all-or-no If a node can't fetch the package or fails `npm install`, it fails during staging and the live component is left untouched on every node, rather than leaving part of the cluster half-updated. The request and response shape are unchanged; the two phases are internal. +:::note +Deploying a **brand-new** component without a restart (`"restart": false`, or omitting `restart`) marks a restart as required — `get_status` reports `restartRequired: true`, and requests to the new component's routes return an actionable 404 explaining that a restart is needed. A never-loaded component can't serve its routes until Harper restarts, so this makes that state visible instead of silent. Each node reports this for itself, since whether the component was already active can differ per node. Redeploying a component that is **already** live does not set the flag: that component's own file watcher requests a restart only if the update actually needs one. +::: + Additional parameters: - `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`) @@ -730,20 +734,31 @@ This supports customer-driven rollback: deploy a new version, run your own healt Harper records every `deploy_component` call in the `system.hdb_deployment` table, capturing the full lifecycle of a deployment including phase transitions (`stage` → `activate` → `restart` → `success`/`failed`, or `prepare` → `replicate` → `restart` on the legacy single-phase path), per-node outcomes, and a bounded event log of install output. -**Staged-build retention.** Deployments staged with `activate: false` leave their built files on disk until they are activated. Harper keeps only the most recent staged builds per component (default 5, configurable via the `deployment_stagingRetention_maxCount` configuration option); older not-yet-activated staged builds are evicted automatically when a new stage lands. Activating a `deployment_id` that has aged out of this window fails with "no staged build found." The `hdb_deployment` records themselves are retained as an audit trail; large payload tarballs are reclaimed automatically after a successful deploy (see `deployment_payloadRetention_maxSize`). +**Staged-build retention.** Deployments staged with `activate: false` leave their built files on disk until they are activated. Harper keeps only the most recent staged builds per component (default 5, configurable via the `deployment_stagingRetention_maxCount` configuration option); older not-yet-activated staged builds are evicted automatically when a new stage lands. Activating a `deployment_id` that has aged out of this window fails with "no staged build found." + +**Payload retention.** The `hdb_deployment` records themselves are always retained as the audit trail — retention only ever reclaims the stored tarball (`payload_blob`), never the row. Two configuration options bound it, and they answer different questions: + +| Option | Default | Effect | +| -------------------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | +| `deployment_payloadRetention_maxSize` | 10 MiB | Reclaims **this** deploy's tarball right after it succeeds, if the tarball was larger than this. Bounds any single payload. | +| `deployment_payloadRetention_maxCount` | 1 | Keeps at most this many stored tarballs **per project**, newest first, dropping the rest after a successful deploy. Bounds total. | + +The default of `maxCount: 1` means only the current version's tarball is kept. It is deliberately conservative: retained payloads share the instance's disk with your own data, so several copies of a large application payload can quietly consume quota. Raise it if you want a wider window of deployments whose payload is still downloadable; set it to `0` to keep none. + +Pruning is automatic and best-effort — it never fails a deploy — and is skipped when a peer failed, since the older payloads are still the retry artifact in that case. A deployment whose payload has been reclaimed (automatically, or explicitly via [`delete_deployment_payload`](#delete_deployment_payload)) reports `payload_blob_present: false` and can no longer serve [`get_deployment_payload`](#get_deployment_payload); everything else about the record stays intact. ### `list_deployments` Returns a list of deployment records, newest first. All filter parameters are optional. -| Parameter | Type | Description | -| --------- | ------ | ------------------------------------------------------------------------- | -| `project` | string | Filter to a specific component project | -| `status` | string | Filter by status: `pending`, `staged`, `success`, `failed`, `rolled_back` | -| `since` | number | Start of time range (Unix timestamp ms) | -| `until` | number | End of time range (Unix timestamp ms) | -| `limit` | number | Maximum number of results (default: 100) | -| `offset` | number | Pagination offset | +| Parameter | Type | Description | +| --------- | ------ | ---------------------------------------- | +| `project` | string | Filter to a specific component project | +| `status` | string | Filter by status (see below) | +| `since` | number | Start of time range (Unix timestamp ms) | +| `until` | number | End of time range (Unix timestamp ms) | +| `limit` | number | Maximum number of results (default: 100) | +| `offset` | number | Pagination offset | ```json { @@ -756,6 +771,11 @@ Returns a list of deployment records, newest first. All filter parameters are op Response includes a `deployments` array and a `total` count. The `payload_blob` field is stripped from list responses for size; use `get_deployment_payload` to retrieve the tarball. +Deployment statuses fall into two groups: + +- **Terminal** — `success`, `failed`, `rolled_back`, and `staged` (an `activate: false` stage-and-stop resting until it is activated or ages out of the staging-retention window). +- **In flight** — `pending`, `extracting`, `installing`, `staging`, `loading`, `replicating`, `activating`, `reverting`, `restarting`. These are the phase the deployment is currently in; a record only stays in one of them while the deploy is running. + ### `get_deployment` Returns a single deployment record by `deployment_id`. When called on an in-progress deployment via a request that accepts `text/event-stream`, the response streams live phase events and install output as Server-Sent Events, replaying the buffered event log then tailing until the deployment reaches a terminal status. @@ -826,7 +846,11 @@ The deployment must be in a terminal status (`success`, `failed`, or `rolled_bac ### `add_ssh_key` -Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. +Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. Supply the private key with `key`, or omit it and pass `generate: true` to have Harper mint the keypair itself. + +The stored private key is encrypted at rest and only ever crosses the cluster as ciphertext; `list_ssh_keys` and the logs never return key material. + +Adding an existing key: ```json { @@ -838,6 +862,37 @@ Adds an SSH key (must be ed25519) for authenticating deployments from private re } ``` +#### Server-side key generation (`generate`) + +With `generate: true`, Harper mints an ed25519 keypair on the node handling the request and returns only the **public** half. The private key is created inside the cluster and never travels from a client, so it can't be captured in a shell history, CI log, or request body on the way in: + +```json +{ + "operation": "add_ssh_key", + "name": "my-key", + "generate": true, + "host": "my-key.github.com", + "hostname": "github.com" +} +``` + +Response: + +```json +{ + "message": "Added ssh key: my-key", + "public_key": "ssh-ed25519 AAAAC3Nza... harper:my-key" +} +``` + +Register that `public_key` with your git host (e.g. as a GitHub deploy key) to authorize the deploy. The generated key is commented `harper:` so it's identifiable in the host's key list. + +`key` and `generate` are mutually exclusive — sending both is rejected. Generation runs `ssh-keygen` on the node; if it isn't available on the PATH the operation fails with an actionable error rather than storing a partial key. + +:::note +`public_key` is returned **only** on the generating call — that response is the one time the public half is handed back. Harper stores the sealed private key and the host config; it does not retain the public key for later retrieval, and `update_ssh_key` requires a key you supply (it can't mint one). So capture `public_key` from this response — if you lose it, `delete_ssh_key` then `add_ssh_key` with `generate: true` again to mint a fresh pair, and re-register the new public key with your git host. +::: + --- ## Secrets From eb5437c736b21b00940a6e6431d7f501822662b9 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Wed, 29 Jul 2026 14:04:25 -0400 Subject: [PATCH 04/17] docs(cli): point the ops-table links at the sections that actually document them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The SSH rows linked to #certificate-management, but the SSH operations are documented under ## Components — so every one of those seven links landed the reader in the wrong section. Repointed them to #components (the convention the other 15 component rows already use). Auditing the same block turned up the identical defect on the six user/role rows (list_users, add_user, alter_user, drop_user, list_roles, drop_role), which are documented under ## Users & Roles. Repointed those to #users--roles, matching the existing link to that section elsewhere in the docs. The five genuine certificate rows (create_csr, sign_certificate, list_certificates, add_certificate, remove_certificate) were already correct and are unchanged. Category labels updated to match their new targets. Verified every operations.md anchor referenced from this table against the real headings — all 11 resolve. --- reference/cli/operations-api-commands.md | 26 ++++++++++++------------ 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/reference/cli/operations-api-commands.md b/reference/cli/operations-api-commands.md index d44b74cfa..27fa04fef 100644 --- a/reference/cli/operations-api-commands.md +++ b/reference/cli/operations-api-commands.md @@ -54,24 +54,24 @@ This is just a brief overview of all operations available as CLI commands. Revie | `sql` | Execute SQL queries | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | | `csv_file_load` | Load data from CSV file | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | | `csv_url_load` | Load data from CSV URL | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `list_users` | List all users | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `add_user` | Create a new user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `alter_user` | Modify user properties | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `drop_user` | Delete a user | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `list_roles` | List all roles | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | -| `drop_role` | Delete a role | [Security](../operations-api/operations.md#certificate-management) | v4.3.0 | +| `list_users` | List all users | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `add_user` | Create a new user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `alter_user` | Modify user properties | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `drop_user` | Delete a user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `list_roles` | List all roles | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `drop_role` | Delete a role | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | | `create_csr` | Create certificate signing request | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | | `sign_certificate` | Sign a certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | | `list_certificates` | List SSL/TLS certificates | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | | `add_certificate` | Add SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | | `remove_certificate` | Remove SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `add_ssh_key` | Add SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `get_ssh_key` | Get SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.7.2 | -| `update_ssh_key` | Update SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `delete_ssh_key` | Delete SSH key | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `list_ssh_keys` | List all SSH keys | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `set_ssh_known_hosts` | Set SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `get_ssh_known_hosts` | Get SSH known hosts | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `add_ssh_key` | Add SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `get_ssh_key` | Get SSH key | [Components](../operations-api/operations.md#components) | v4.7.2 | +| `update_ssh_key` | Update SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `delete_ssh_key` | Delete SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `list_ssh_keys` | List all SSH keys | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `set_ssh_known_hosts` | Set SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `get_ssh_known_hosts` | Get SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | | `cluster_get_routes` | Get cluster routing information | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | | `cluster_network` | Get cluster network status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | | `cluster_status` | Get cluster status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | From 40f0e69c34b7d34d5af0b02f8438f0cfc6ec0bd1 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Wed, 29 Jul 2026 14:37:49 -0400 Subject: [PATCH 05/17] docs(deploy): by-reference deploys, sealed credentials, CI token auth MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Builds on the two-phase deploy docs (#599) to cover the rest of the deploy-by-reference workflow. reference/components/applications.md - "Deploying by Reference" — `harper deploy by_ref=true`, `ref=`, `credential=`. Explains why a reference pins to a resolved SHA rather than the tag or branch name typed: peers resolve the package independently, so a tag that moves mid-deploy could otherwise leave nodes running different code. Notes that the cluster clones the pushed remote, and that by-reference means the cluster builds from source — so an app whose build can't run on the node should stay on payload deploys. - "Provisioning a Deploy Credential" — `harper deploy setup=true`, the client-side sealing flow, and that reverting to the previous version needs no credential at all. reference/cli/authentication.md - Documents HARPER_CLI_REFRESH_TOKEN / HARPER_CLI_OPERATION_TOKEN and adds "Token credentials for CI/CD", so the CI guidance is no longer "put an admin password in your pipeline". Covers precedence, in-memory-only refresh, and token lifetimes (1d / 30d defaults). - Warns that a user holds only ONE valid refresh token at a time: issuing a new one invalidates the last, so a routine local `harper login` silently breaks a pipeline using the same account. Hence the recommendation to give CI its own user. (Verified against security/tokenAuthentication.ts, which stores a single hashed refresh_token per user record.) reference/cli/commands.md - `harper login --for-ci`, including the stdout/stderr split that makes `| gh secret set --env-file -` work without displaying the token. reference/security/secrets.md - Points the existing private-source deploy credentials section at the CLI flow that automates it. Documents HarperFast/harper#1850, #1851, and #1876. Verified every anchor link in the changed files resolves (16 checked). Co-Authored-By: Claude Opus 4.8 --- reference/cli/authentication.md | 49 +++++++++++++++++++ reference/cli/commands.md | 15 ++++++ reference/components/applications.md | 73 ++++++++++++++++++++++++++++ reference/security/secrets.md | 2 + 4 files changed, 139 insertions(+) diff --git a/reference/cli/authentication.md b/reference/cli/authentication.md index c2d9487ac..f24f54bc4 100644 --- a/reference/cli/authentication.md +++ b/reference/cli/authentication.md @@ -71,6 +71,10 @@ The CLI supports loading environment variables from your shell environment (or o - `HARPER_CLI_TARGET` (or `CLI_TARGET`) - Sets the default `target` for CLI commands. - `HARPER_CLI_USERNAME` (or `CLI_TARGET_USERNAME`) - Harper admin username for the target. - `HARPER_CLI_PASSWORD` (or `CLI_TARGET_PASSWORD`) - Harper admin password for the target. +- `HARPER_CLI_REFRESH_TOKEN` (or `CLI_TARGET_REFRESH_TOKEN`) - Long-lived token the CLI exchanges for an operation token on each run. Available since v5.2.0. +- `HARPER_CLI_OPERATION_TOKEN` (or `CLI_TARGET_OPERATION_TOKEN`) - Short-lived operation token, if you would rather supply one directly. Available since v5.2.0. + +**Prefer tokens over a password in CI.** A refresh token is scoped to authentication, can be revoked without changing the account password, and — unlike `HARPER_CLI_PASSWORD` — cannot be reused to log in interactively. See [Token credentials for CI/CD](#token-credentials-for-cicd) below. **Example `.env` file**: @@ -129,6 +133,51 @@ harper deploy target=https://prod-server.com:9925 replicated=true harper restart target=https://prod-server.com:9925 replicated=true ``` +##### Token credentials for CI/CD + +Available since: v5.2.0 + +Rather than storing an admin password in your CI provider, log in once locally and hand CI a **refresh token**. The CLI mints a fresh, short-lived operation token from it on every run, so the only durable secret CI holds is a revocable token. + +`harper login --for-ci` writes the two variables to **stdout** in `.env` format — and nothing else, so it pipes cleanly. Everything a human reads (the banner, prompts, status) goes to stderr: + +```bash +# Set both GitHub Actions secrets in one command — the token is never displayed +harper login --for-ci | gh secret set --env-file - + +# Or copy them to the clipboard to paste in by hand +harper login --for-ci | pbcopy +``` + +The block it emits: + +```bash +HARPER_CLI_TARGET=https://example.com:9925/ +HARPER_CLI_REFRESH_TOKEN=eyJhbGciOi... +``` + +Because stdout carries only these lines, the token never appears on screen or in your shell history — which is not true of copying it out of terminal output by hand. + +Expose the two values to the deploy step and no other credentials are needed: + +```yaml +- name: Deploy + run: harper deploy project=my-app restart=true replicated=true + env: + HARPER_CLI_TARGET: ${{ secrets.HARPER_CLI_TARGET }} + HARPER_CLI_REFRESH_TOKEN: ${{ secrets.HARPER_CLI_REFRESH_TOKEN }} +``` + +**Precedence**: an explicitly supplied username (arguments or `HARPER_CLI_USERNAME`/`HARPER_CLI_PASSWORD`) wins; otherwise env-var tokens are used; otherwise the token saved by `harper login`. Env-var tokens deliberately outrank the saved credentials file so a runner that has both behaves predictably. A token refreshed from an env var is held in memory for that invocation only — nothing is written to `~/.harperdb/credentials.json`. + +**Lifetimes**: operation tokens expire after `authentication.operationTokenTimeout` (default `1d`) and refresh tokens after `authentication.refreshTokenTimeout` (default `30d`). CI needs re-provisioning when the refresh token expires. + +:::warning +**Each user has only one valid refresh token at a time.** Logging in again as that user issues a new one and invalidates the previous one, so a routine local `harper login` will break a pipeline holding an older token for the same account. + +Create a **dedicated CI user** and run `harper login --for-ci` as that user. This also scopes the pipeline's permissions and lets you revoke its access — by logging in again as that user, or by changing its password — without disturbing anyone else. +::: + #### Method 3: Command Parameters Provide credentials directly as command parameters: diff --git a/reference/cli/commands.md b/reference/cli/commands.md index 24cfbc57b..d216a125a 100644 --- a/reference/cli/commands.md +++ b/reference/cli/commands.md @@ -155,6 +155,7 @@ harper login **Optional Parameters**: - `` - The URL of the Harper instance to log in to. +- `--for-ci` - Print CI/CD credentials to stdout after logging in. Available since v5.2.0. **Prompts**: @@ -164,6 +165,20 @@ You'll be asked to type in the following information: - `` - Harper admin username. - `` - Harper admin password. +**`--for-ci`**: + +Prints `HARPER_CLI_TARGET` and `HARPER_CLI_REFRESH_TOKEN` to **stdout** in `.env` format — and nothing else, so the output pipes directly into a secret store without the token being displayed. Everything else (banner, prompts, status) goes to stderr: + +```bash +# Set both GitHub Actions secrets in one command +harper login --for-ci | gh secret set --env-file - + +# Or copy them to paste in by hand +harper login --for-ci | pbcopy +``` + +See [Token credentials for CI/CD](authentication.md#token-credentials-for-cicd) for how the CLI consumes these variables, and for why a pipeline should use a dedicated user. + ### `harper logout` Available since: v5.0.0 diff --git a/reference/components/applications.md b/reference/components/applications.md index dfc70cd33..95882a494 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -130,6 +130,79 @@ Harper generates a `package.json` from component configurations and uses a form For SSH-based private repos, use the [Add SSH Key](#add_ssh_key) operation to register keys first. +### Deploying by Reference + +Available since: v5.2.0 + +Omitting `package` uploads a snapshot of your working directory. The result is an anonymous artifact: nothing records _which_ commit it came from, so reproducing it later — or stepping back to a previous release — means finding those exact files again. + +Deploying by **reference** sends a pinned git reference instead, and the cluster fetches that exact commit. Redeploying the same reference is an exact redeploy, and rolling back is deploying an older one. + +`harper deploy by_ref=true` builds that reference from the local git repository, so you don't assemble the URL yourself: + +```sh +harper deploy by_ref=true restart=true replicated=true +``` + +This resolves the repository's `origin` remote and the current commit, then deploys `package=git+https://github.com//.git#`. + +**Parameters**: + +- `by_ref` - Build the package reference from the local repository. +- `ref` _(optional)_ - Deploy a specific commit, tag, or branch instead of `HEAD`. Implies `by_ref`. +- `credential` _(optional)_ - Git host whose stored credential authenticates the clone, e.g. `github.com`. Omit for public repositories. + +```sh +# Deploy a specific tag +harper deploy ref=v1.2.0 restart=true replicated=true + +# Roll back by deploying an older commit +harper deploy ref=9f8c2a1 restart=true replicated=true +``` + +**A reference is pinned to a SHA, not to the name you typed.** Tags and branches are resolved locally and the full commit SHA is what ships. This matters on a cluster: peers resolve the package independently, so a tag that moves mid-deploy — or a branch that advances — could otherwise leave nodes running different code. + +**Commit and push first.** The cluster clones from the remote, so it only sees commits that have been pushed. `by_ref` warns when the working tree is dirty, since uncommitted changes won't be part of the deploy. + +#### Private repositories + +Pass `credential=` for a private repository. The CLI attaches a `credentials` reference naming a secret that the cluster resolves in memory at clone time, so no token travels in the operation body or lands on disk: + +```sh +harper deploy by_ref=true credential=github.com restart=true replicated=true +``` + +Provision that credential once with [`harper deploy setup=true`](#provisioning-a-deploy-credential). See [Private-source deploy credentials](../security/secrets.md#private-source-deploy-credentials) for how the secret is named and resolved, and [`add_ssh_key`](#add_ssh_key) for the SSH-key alternative. + +:::note +Deploying by reference means the **cluster** installs and builds the component from source. If your application needs a build step that can't run on the node, keep shipping the built output as a payload deploy instead. +::: + +### Provisioning a Deploy Credential + +Available since: v5.2.0 + +`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source: + +```sh +harper deploy setup=true +``` + +It asks which private source needs a credential (a GitHub repository or an npm registry) and sources a token — from your `gh` CLI session, or one you paste — then: + +1. Fetches the cluster's public key with `get_secrets_public_key`. +2. **Encrypts the token locally** into an `enc:v1:` envelope. +3. Stores only the ciphertext via `set_secret`, granted to the component. +4. Prints the `credentials` reference for the deploy to use. + +The plaintext never leaves your machine: the operations API, its logs, and replication only ever carry the envelope, and the cluster decrypts it in memory at deploy time. This requires a cluster with secrets custody (Harper Pro / Fabric) — see [Client-side encryption](../security/secrets.md#client-side-encryption-encrypt-before-it-leaves-the-client). + +Because the stored token is durable, later deploys — including re-fetching an older reference — reuse it without re-entering anything. + +:::note +Rolling back to the **immediately previous** version needs no credential at all: [`revert_component`](../operations-api/operations.md#revert_component) swaps in the retained previous build without re-fetching from the source. +::: + ## Dependency Management Harper uses `npm` and `package.json` for dependency management. diff --git a/reference/security/secrets.md b/reference/security/secrets.md index 2e2a0a842..9b0ac045d 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -289,6 +289,8 @@ function encryptSecret(plaintext, publicKeyPem, kid) { `deploy_component` accepts a `credentials` array so a component installed from a private **npm registry** or private **git repository** can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body, persisting as a plaintext `.npmrc`, or being written to disk for git — so package-reference deploys survive rollback, reboot, and new peers joining. Ingested tokens are stored under a derived name (`deploy..` or `deploy..git.`) granted to the component. See [`deploy_component`](../operations-api/operations.md#deploy_component). +To provision one of these without handing the cluster a plaintext token at all, `harper deploy setup=true` (v5.2.0+) runs the [client-side encryption](#client-side-encryption-encrypt-before-it-leaves-the-client) flow above for you: it fetches the public key, seals the token locally into an `enc:v1:` envelope, stores only the ciphertext under that same derived name, and prints the `credentials` reference for the deploy to use. See [Provisioning a Deploy Credential](../components/applications.md#provisioning-a-deploy-credential). + ## Threat model **Protects against:** theft of on-disk config/`.env` files, the editor/operations read surface, secrets appearing in operations logs and replication payloads, and an operator observing traffic at the TLS-terminating layer. Client-side encryption additionally keeps plaintext off the operations API entirely. From 9415237892b8f325640e2e3f9ef3f122b277c1f5 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Thu, 30 Jul 2026 09:27:44 -0400 Subject: [PATCH 06/17] docs: correct deployment status buckets, rolling-restart response, revert caveat Addresses cb1kenobi's review on #599. - "Terminal" wrongly included `staged`. The implementation's TERMINAL_STATUSES is {success, failed, rolled_back} and that set gates get_deployment_payload and payload-pruning eligibility, so listing `staged` as terminal contradicted the get_deployment_payload section. Split into Terminal / Resting / In flight, and noted why a resting payload is deliberately still held. - get_deployment's `status` row listed 7 of 13 values; now lists all, pointing at the grouped explanation. - The `restart: "rolling"` example's response showed the no-restart message. Corrected to the rolling shape (restartJobId + ", restarting Harper") and noted what the no-restart response looks like instead. - revert_component: added a caution that reverting swaps live directories and does NOT rewrite the stored `package:` reference, so a node provisioned after a revert installs the reverted-away version. Verified in code: revertComponent never calls writeComponentRootConfig, and installApplications() installs from root config. - deployment_id activate: documents that the staged package identifier and credential references are recovered and persisted at activation (fixed in harper#1849), so `package` need not be repeated on the activate call. --- reference/operations-api/operations.md | 54 +++++++++++++++----------- 1 file changed, 32 insertions(+), 22 deletions(-) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index a52e28156..72ff9e415 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -618,7 +618,7 @@ Additional parameters: - `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`) - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) - `activate` — set to `false` to **stage only** and stop before go-live. The build is prepared and verified on every node and the response returns a `deployment_id` in a `staged` state; nothing goes live. Activate it later by calling `deploy_component` again with that `deployment_id` (see below). Useful for pre-staging a release and flipping it live in a separate, fast step. -- `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). No new payload is fetched or installed; the already-staged build is swapped live cluster-wide. `project` is still required. +- `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). No new payload is fetched or installed; the already-staged build is swapped live cluster-wide. `project` is still required. For a `package` deploy you do not need to repeat `package` here: the identifier and credential references recorded when it was staged are recovered from the deployment and persisted to root config at activation, on every node — so a later restart or a newly joined peer reinstalls the version you activated. - `revert_on_failure` — if the activate phase fails on some nodes (leaving the cluster split across versions), automatically swap the nodes that did activate back to their previous version so the cluster reconverges. Off by default. Applies to both a full deploy and a `deployment_id` activate. - `ignore_replication_errors` — treat replication/peer failures as non-fatal (best-effort deploy to a partially-available cluster). This also opts out of the stage barrier. Applies to both a full deploy and a `deployment_id` activate. - `deployment_timeout` — per-deploy budget (ms) for peers to receive the replicated deployment row; defaults to 120000. @@ -685,15 +685,18 @@ A normal deploy (stage + activate): } ``` -Response: +Response — a rolling restart is driven by a separate replicated job, so its id comes back as `restartJobId`: ```json { "deployment_id": "a3f8c2d1...", - "message": "Successfully deployed: my-app" + "restartJobId": "b7d41e09...", + "message": "Successfully deployed: my-app, restarting Harper" } ``` +Without a restart (`"restart": false`, or omitted) the response carries no `restartJobId` and the message is just `Successfully deployed: my-app`. + Stage now, activate later: **Stage request** (`activate: false`): @@ -730,6 +733,12 @@ This supports customer-driven rollback: deploy a new version, run your own healt `revert_component` fails with "no previous version is retained" for a component that has only ever been deployed once (nothing to revert to). +:::caution +Reverting swaps the **live directories** on the nodes currently in the cluster; it does not rewrite the component's stored `package:` reference in `harperdb-config.yaml`. For a `package` deploy that means a revert is not a config-level rollback: a node provisioned _after_ the revert — a newly joined peer, or an existing node whose components directory is rebuilt — installs from the stored package reference at boot, so it comes up on the version you reverted away from rather than the one every other node is running. + +To roll back durably for a `package` deploy, deploy the older version explicitly (`deploy_component` with the previous `package` reference) instead of, or after, reverting. `revert_component` is the fast live-instance swap; an explicit deploy is what changes what a future node will install. +::: + ### Deployment Operations Harper records every `deploy_component` call in the `system.hdb_deployment` table, capturing the full lifecycle of a deployment including phase transitions (`stage` → `activate` → `restart` → `success`/`failed`, or `prepare` → `replicate` → `restart` on the legacy single-phase path), per-node outcomes, and a bounded event log of install output. @@ -771,10 +780,11 @@ Returns a list of deployment records, newest first. All filter parameters are op Response includes a `deployments` array and a `total` count. The `payload_blob` field is stripped from list responses for size; use `get_deployment_payload` to retrieve the tarball. -Deployment statuses fall into two groups: +Deployment statuses fall into three groups: -- **Terminal** — `success`, `failed`, `rolled_back`, and `staged` (an `activate: false` stage-and-stop resting until it is activated or ages out of the staging-retention window). -- **In flight** — `pending`, `extracting`, `installing`, `staging`, `loading`, `replicating`, `activating`, `reverting`, `restarting`. These are the phase the deployment is currently in; a record only stays in one of them while the deploy is running. +- **Terminal** — `success`, `failed`, `rolled_back`. The deploy is over. Only these count as terminal internally, which is what gates `get_deployment_payload` and makes a payload eligible for retention pruning. +- **Resting** — `staged`. An `activate: false` stage-and-stop, waiting to be activated or to age out of the staging-retention window. It is finished but not terminal, so its payload is deliberately still held: it is the source the pending activation needs. +- **In flight** — `pending`, `extracting`, `installing`, `staging`, `loading`, `replicating`, `activating`, `reverting`, `restarting`. The phase the deployment is currently in; a record only stays in one of these while the deploy is running. ### `get_deployment` @@ -789,22 +799,22 @@ Returns a single deployment record by `deployment_id`. When called on an in-prog The deployment record includes: -| Field | Description | -| -------------------- | -------------------------------------------------------------------------------------------------- | -| `deployment_id` | Unique identifier (content hash) | -| `project` | Component project name | -| `package_identifier` | Package reference or `payload` for tar uploads | -| `status` | `pending`, `staging`, `staged`, `activating`, `success`, `failed`, or `rolled_back` | -| `phase` | Current lifecycle phase: `stage`, `load`, `activate`, `restart` (or legacy `prepare`, `replicate`) | -| `event_log` | Bounded log of install output and phase transitions (up to 200 entries) | -| `peer_results` | Per-node outcome map for replicated deployments | -| `payload_hash` | SHA-256 hash of the deployment tarball | -| `payload_size` | Byte size of the deployment tarball | -| `started_at` | Timestamp when deployment began | -| `completed_at` | Timestamp when deployment finished | -| `user` | User who initiated the deployment | -| `rollback_of` | `deployment_id` of the deployment this rolls back, if applicable | -| `error` | Error message for failed deployments | +| Field | Description | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `deployment_id` | Unique identifier (content hash) | +| `project` | Component project name | +| `package_identifier` | Package reference or `payload` for tar uploads | +| `status` | Any of the values listed under [`list_deployments`](#list_deployments) — `pending`, `extracting`, `installing`, `staging`, `staged`, `loading`, `replicating`, `activating`, `reverting`, `restarting`, `success`, `failed`, or `rolled_back` | +| `phase` | Current lifecycle phase: `stage`, `load`, `activate`, `restart` (or legacy `prepare`, `replicate`) | +| `event_log` | Bounded log of install output and phase transitions (up to 200 entries) | +| `peer_results` | Per-node outcome map for replicated deployments | +| `payload_hash` | SHA-256 hash of the deployment tarball | +| `payload_size` | Byte size of the deployment tarball | +| `started_at` | Timestamp when deployment began | +| `completed_at` | Timestamp when deployment finished | +| `user` | User who initiated the deployment | +| `rollback_of` | `deployment_id` of the deployment this rolls back, if applicable | +| `error` | Error message for failed deployments | ### `get_deployment_payload` From f9ad59b59c17e4825e433e58751b8c47a48c9207 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Wed, 12 Aug 2026 11:42:47 -0400 Subject: [PATCH 07/17] docs(deploy): add_ssh_key generate no longer needs ssh-keygen on the host harper-pro#594 moved generation in-process (node:crypto), so the PATH requirement and its error path no longer exist. The response shape and the harper: comment convention documented above are unchanged. Co-Authored-By: Claude Opus 5 --- reference/operations-api/operations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 72ff9e415..1608587fe 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -897,7 +897,7 @@ Response: Register that `public_key` with your git host (e.g. as a GitHub deploy key) to authorize the deploy. The generated key is commented `harper:` so it's identifiable in the host's key list. -`key` and `generate` are mutually exclusive — sending both is rejected. Generation runs `ssh-keygen` on the node; if it isn't available on the PATH the operation fails with an actionable error rather than storing a partial key. +`key` and `generate` are mutually exclusive — sending both is rejected. Generation happens in-process, so it requires no `ssh-keygen` binary on the host and the minted private key is never written to a temporary file on its way into storage. :::note `public_key` is returned **only** on the generating call — that response is the one time the public half is handed back. Harper stores the sealed private key and the host config; it does not retain the public key for later retrieval, and `update_ssh_key` requires a key you supply (it can't mint one). So capture `public_key` from this response — if you lose it, `delete_ssh_key` then `add_ssh_key` with `generate: true` again to mint a fresh pair, and re-register the new public key with your git host. From d000da293c48f32cde26d1003e6593c3ebab648e Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Wed, 12 Aug 2026 12:29:59 -0400 Subject: [PATCH 08/17] docs(deploy): revert_component is addressed, idempotent, and config-level MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Brings the revert documentation in line with HarperFast/harper#1849 after review. The behavior changed in three ways that each invalidated something documented here. - `revert_on_failure` is gone — it is now rejected outright, so the bullet is removed. Replaced with what actually happens on a partial activation (the deployment stays `activating`; recover by rolling forward or by an explicit revert) and why there is no automatic rollback: past the activation barrier, a peer reporting failure does not prove it did not activate, so auto-reverting the failed nodes can split the cluster three ways instead of converging it. - `revert_component` now takes a **required** `to_deployment_id` and is idempotent rather than a bidirectional toggle. Documented as such, with the reason the target is mandatory (a retry after a lost response must not flip the rejected release back in), a parameter table, the response fields including `from_deployment_id`/`rollback_of`, and the one-activation reach of retention. - The `:::caution` saying a revert is not a config-level rollback is **no longer true** and is replaced by the opposite: the revert rewrites the stored `package:` reference and the boot-time application lock as part of the same operation, so a newly joined peer or a rebuilt components directory installs the version the cluster is actually running. Reverting away from a `package` deploy to a payload-deployed version removes the reference entirely. CLI: `harper revert` now requires `to_deployment_id`, and the example and command table say so. The wider CLI table diff is Prettier realigning column widths after the description text changed. Version badges still carry the placeholder release, per this PR's existing note. --- reference/cli/operations-api-commands.md | 162 ++++++++++++----------- reference/components/applications.md | 2 +- reference/operations-api/operations.md | 29 ++-- 3 files changed, 104 insertions(+), 89 deletions(-) diff --git a/reference/cli/operations-api-commands.md b/reference/cli/operations-api-commands.md index 27fa04fef..eebe24e95 100644 --- a/reference/cli/operations-api-commands.md +++ b/reference/cli/operations-api-commands.md @@ -34,84 +34,84 @@ The following operations are available through the CLI. Operations that require This is just a brief overview of all operations available as CLI commands. Review the respective operation documentation for more information on available arguments and expected behavior. Keep in mind that all operations options are converted to CLI arguments in the same way (using `snake_case`). ::: -| Operation | Description | Category | Available Since | -| -------------------------------- | --------------------------------------------- | ---------------------------------------------------------------------- | --------------- | -| `describe_table` | Describe table structure and metadata | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `describe_all` | Describe all databases and tables | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `describe_database` | Describe database structure | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_database` | Create a new database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_database` | Delete a database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_table` | Create a new table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_table` | Delete a table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `create_attribute` | Create a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `drop_attribute` | Delete a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | -| `search_by_id` | Search records by ID | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `search_by_value` | Search records by attribute value | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `insert` | Insert new records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `update` | Update existing records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `upsert` | Insert or update records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | -| `delete` | Delete records | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `sql` | Execute SQL queries | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `csv_file_load` | Load data from CSV file | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `csv_url_load` | Load data from CSV URL | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | -| `list_users` | List all users | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | -| `add_user` | Create a new user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | -| `alter_user` | Modify user properties | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | -| `drop_user` | Delete a user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | -| `list_roles` | List all roles | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | -| `drop_role` | Delete a role | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | -| `create_csr` | Create certificate signing request | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `sign_certificate` | Sign a certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `list_certificates` | List SSL/TLS certificates | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `add_certificate` | Add SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `remove_certificate` | Remove SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | -| `add_ssh_key` | Add SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | -| `get_ssh_key` | Get SSH key | [Components](../operations-api/operations.md#components) | v4.7.2 | -| `update_ssh_key` | Update SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | -| `delete_ssh_key` | Delete SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | -| `list_ssh_keys` | List all SSH keys | [Components](../operations-api/operations.md#components) | v4.4.0 | -| `set_ssh_known_hosts` | Set SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | -| `get_ssh_known_hosts` | Get SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | -| `cluster_get_routes` | Get cluster routing information | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `cluster_network` | Get cluster network status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `cluster_status` | Get cluster status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `remove_node` | Remove node from cluster | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | -| `add_component` | Add a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `deploy_component` | Deploy a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `deploy` (alias) | Alias for `deploy_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `stage` (alias) | Stage a component without going live | [Components](../operations-api/operations.md#components) | v5.2.0 | -| `activate` (alias) | Activate a previously-staged deploy | [Components](../operations-api/operations.md#components) | v5.2.0 | -| `revert_component` | Roll a component back to its previous version | [Components](../operations-api/operations.md#components) | v5.2.0 | -| `revert` (alias) | Alias for `revert_component` | [Components](../operations-api/operations.md#components) | v5.2.0 | -| `package_component` | Package a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `package` (alias) | Alias for `package_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `drop_component` | Remove a component | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `get_components` | List all components | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `get_component_file` | Get component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `set_component_file` | Set component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `install_node_modules` | Install Node.js dependencies | [Components](../operations-api/operations.md#components) | v4.3.0 | -| `set_configuration` | Update configuration settings | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | -| `get_configuration` | Get current configuration | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | -| `create_authentication_tokens` | Create authentication tokens | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | -| `refresh_operation_token` | Refresh operation token | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | -| `restart_service` | Restart Harper service | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `restart` | Restart Harper instance | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `system_information` | Get system information | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `registration_info` | Get registration information | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `get_fingerprint` | Get instance fingerprint | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `set_license` | Set license key | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | -| `get_usage_licenses` | Get usage and license info | [Licensing](../operations-api/operations.md#registration--licensing) | v4.7.3 | -| `get_job` | Get job status | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | -| `search_jobs_by_start_date` | Search jobs by start date | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | -| `read_log` | Read application logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `read_transaction_log` | Read transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `read_audit_log` | Read audit logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `delete_transaction_logs_before` | Delete old transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | -| `purge_stream` | Purge streaming data | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | -| `delete_records_before` | Delete old records | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | -| `get_status` | Get custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | -| `set_status` | Set custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | -| `clear_status` | Clear custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| Operation | Description | Category | Available Since | +| -------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------- | --------------- | +| `describe_table` | Describe table structure and metadata | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `describe_all` | Describe all databases and tables | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `describe_database` | Describe database structure | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_database` | Create a new database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_database` | Delete a database | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_table` | Create a new table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_table` | Delete a table | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `create_attribute` | Create a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `drop_attribute` | Delete a table attribute | [Database](../operations-api/operations.md#databases--tables) | v4.3.0 | +| `search_by_id` | Search records by ID | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `search_by_value` | Search records by attribute value | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `insert` | Insert new records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `update` | Update existing records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `upsert` | Insert or update records | [Data](../operations-api/operations.md#nosql-operations) | v4.4.9 | +| `delete` | Delete records | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `sql` | Execute SQL queries | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `csv_file_load` | Load data from CSV file | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `csv_url_load` | Load data from CSV URL | [Data](../operations-api/operations.md#nosql-operations) | v4.3.0 | +| `list_users` | List all users | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `add_user` | Create a new user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `alter_user` | Modify user properties | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `drop_user` | Delete a user | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `list_roles` | List all roles | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `drop_role` | Delete a role | [Users & Roles](../operations-api/operations.md#users--roles) | v4.3.0 | +| `create_csr` | Create certificate signing request | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `sign_certificate` | Sign a certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `list_certificates` | List SSL/TLS certificates | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `add_certificate` | Add SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `remove_certificate` | Remove SSL/TLS certificate | [Security](../operations-api/operations.md#certificate-management) | v4.4.0 | +| `add_ssh_key` | Add SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `get_ssh_key` | Get SSH key | [Components](../operations-api/operations.md#components) | v4.7.2 | +| `update_ssh_key` | Update SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `delete_ssh_key` | Delete SSH key | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `list_ssh_keys` | List all SSH keys | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `set_ssh_known_hosts` | Set SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `get_ssh_known_hosts` | Get SSH known hosts | [Components](../operations-api/operations.md#components) | v4.4.0 | +| `cluster_get_routes` | Get cluster routing information | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `cluster_network` | Get cluster network status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `cluster_status` | Get cluster status | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `remove_node` | Remove node from cluster | [Clustering](../operations-api/operations.md#replication--clustering) | v4.3.0 | +| `add_component` | Add a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `deploy_component` | Deploy a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `deploy` (alias) | Alias for `deploy_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `stage` (alias) | Stage a component without going live | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `activate` (alias) | Activate a previously-staged deploy | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `revert_component` | Put a component's retained previous version back in service | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `revert` (alias) | Alias for `revert_component` | [Components](../operations-api/operations.md#components) | v5.2.0 | +| `package_component` | Package a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `package` (alias) | Alias for `package_component` | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `drop_component` | Remove a component | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `get_components` | List all components | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `get_component_file` | Get component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `set_component_file` | Set component file contents | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `install_node_modules` | Install Node.js dependencies | [Components](../operations-api/operations.md#components) | v4.3.0 | +| `set_configuration` | Update configuration settings | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | +| `get_configuration` | Get current configuration | [Configuration](../operations-api/operations.md#configuration) | v4.3.0 | +| `create_authentication_tokens` | Create authentication tokens | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | +| `refresh_operation_token` | Refresh operation token | [Authentication](../operations-api/operations.md#token-authentication) | v4.3.0 | +| `restart_service` | Restart Harper service | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `restart` | Restart Harper instance | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `system_information` | Get system information | [System](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `registration_info` | Get registration information | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `get_fingerprint` | Get instance fingerprint | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `set_license` | Set license key | [Licensing](../operations-api/operations.md#registration--licensing) | v4.3.0 | +| `get_usage_licenses` | Get usage and license info | [Licensing](../operations-api/operations.md#registration--licensing) | v4.7.3 | +| `get_job` | Get job status | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | +| `search_jobs_by_start_date` | Search jobs by start date | [Jobs](../operations-api/operations.md#jobs) | v4.3.0 | +| `read_log` | Read application logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `read_transaction_log` | Read transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `read_audit_log` | Read audit logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `delete_transaction_logs_before` | Delete old transaction logs | [Logging](../operations-api/operations.md#logs) | v4.3.0 | +| `purge_stream` | Purge streaming data | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | +| `delete_records_before` | Delete old records | [Maintenance](../operations-api/operations.md#jobs) | v4.3.0 | +| `get_status` | Get custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| `set_status` | Set custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | +| `clear_status` | Clear custom status information | [Status](../operations-api/operations.md#registration--licensing) | v4.6.0 | ### Command Aliases @@ -231,12 +231,14 @@ harper stage project=my-app harper activate project=my-app deployment_id=a3f8c2d1... ``` -**Roll a component back to its previous version**: +**Put a component's retained previous version back in service** (requires the deployment you want live again; `list_deployments` reports it): ```bash -harper revert project=my-app +harper revert project=my-app to_deployment_id=a3f8c2d1... ``` +Naming the target makes the command safe to re-run: if that version is already live, it reports so and changes nothing. + **Mint a deploy key for a private repository** (Harper generates the ed25519 keypair server-side and returns only the public half — register it with your git host): ```bash diff --git a/reference/components/applications.md b/reference/components/applications.md index 95882a494..d04a5dd80 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -200,7 +200,7 @@ The plaintext never leaves your machine: the operations API, its logs, and repli Because the stored token is durable, later deploys — including re-fetching an older reference — reuse it without re-entering anything. :::note -Rolling back to the **immediately previous** version needs no credential at all: [`revert_component`](../operations-api/operations.md#revert_component) swaps in the retained previous build without re-fetching from the source. +Rolling back to the **immediately previous** version needs no credential at all: [`revert_component`](../operations-api/operations.md#revert_component) puts the retained previous build back in service without re-fetching from the source, and updates the stored `package:` reference to match so a rebuilt node installs the version the cluster is running. It takes a required `to_deployment_id`, so re-running it is safe. ::: ## Dependency Management diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 1608587fe..ec5d18865 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -619,9 +619,11 @@ Additional parameters: - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) - `activate` — set to `false` to **stage only** and stop before go-live. The build is prepared and verified on every node and the response returns a `deployment_id` in a `staged` state; nothing goes live. Activate it later by calling `deploy_component` again with that `deployment_id` (see below). Useful for pre-staging a release and flipping it live in a separate, fast step. - `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). No new payload is fetched or installed; the already-staged build is swapped live cluster-wide. `project` is still required. For a `package` deploy you do not need to repeat `package` here: the identifier and credential references recorded when it was staged are recovered from the deployment and persisted to root config at activation, on every node — so a later restart or a newly joined peer reinstalls the version you activated. -- `revert_on_failure` — if the activate phase fails on some nodes (leaving the cluster split across versions), automatically swap the nodes that did activate back to their previous version so the cluster reconverges. Off by default. Applies to both a full deploy and a `deployment_id` activate. - `ignore_replication_errors` — treat replication/peer failures as non-fatal (best-effort deploy to a partially-available cluster). This also opts out of the stage barrier. Applies to both a full deploy and a `deployment_id` activate. - `deployment_timeout` — per-deploy budget (ms) for peers to receive the replicated deployment row; defaults to 120000. + +If the activate phase fails on some nodes after others have already gone live, the deploy reports the split nodes and the deployment stays in an `activating` state rather than being rolled back automatically. Recover by rolling forward (stage and activate a known-good version) or by rolling back explicitly with [`revert_component`](#revert_component). There is deliberately no automatic rollback: once a node is past the activation barrier, a peer reporting failure does not prove that peer did not activate — it can complete its swap and then fail, or die before replying — so automatically reverting "the failed nodes" risks rolling an untouched node an extra version back and leaving the cluster split three ways instead of converging it. + - `two_phase` — set to `false` to force the legacy single-phase (in-place) deploy instead of stage-then-activate. - `credentials` — credentials for installing a component from a private npm registry or private git repository (see below) @@ -719,25 +721,36 @@ Stage now, activate later: ### `revert_component` -Swaps a component's live version back to its **retained previous version** across the cluster, then restarts. Every `deploy_component` activation retains the version it replaced (one previous version is kept per component), so `revert_component` is a fast rollback that does not re-fetch or re-install. The swap is bidirectional — reverting a revert rolls forward again. +Puts a component's **retained previous version** back in service across the cluster, then restarts. Every `deploy_component` activation retains the version it replaced, so this is a fast rollback that resolves no package, decrypts no secret, downloads no artifact and runs no install — every node already has the bytes, and the swap is a single atomic directory rename per node. + +This is the rollback for the bad release you just shipped: deploy a new version, run your own health checks against it, and put the old one back if you are not happy — even when the cluster otherwise looks healthy. -This supports customer-driven rollback: deploy a new version, run your own health checks against it, and revert if you are not happy — even when the cluster otherwise looks healthy. +`to_deployment_id` is **required**, and names the deployment you expect to be live once the call returns: ```json { "operation": "revert_component", "project": "my-app", + "to_deployment_id": "a3f8c2d1-...", "restart": "rolling" } ``` -`revert_component` fails with "no previous version is retained" for a component that has only ever been deployed once (nothing to revert to). +Naming the target is what makes the operation safe to retry. If that version is **already live**, the call succeeds without changing anything — so a client that loses the response and retries cannot flip the rejected release back in. If it matches the retained previous version, the swap happens and the version it displaced becomes the new retained previous, so an explicitly targeted revert of a revert rolls forward again. -:::caution -Reverting swaps the **live directories** on the nodes currently in the cluster; it does not rewrite the component's stored `package:` reference in `harperdb-config.yaml`. For a `package` deploy that means a revert is not a config-level rollback: a node provisioned _after_ the revert — a newly joined peer, or an existing node whose components directory is rebuilt — installs from the stored package reference at boot, so it comes up on the version you reverted away from rather than the one every other node is running. +| Parameter | Description | +| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `project` | **Required.** The component to revert. | +| `to_deployment_id` | **Required.** The deployment you expect to be live afterwards. `list_deployments` reports it, and `deploy_component` returns it. | +| `restart` | `true` to restart immediately, or `"rolling"` for a rolling restart. | +| `ignore_replication_errors` | Treat peer failures as non-fatal. | +| `deployment_timeout` | Per-operation budget (ms) for peers. | -To roll back durably for a `package` deploy, deploy the older version explicitly (`deploy_component` with the previous `package` reference) instead of, or after, reverting. `revert_component` is the fast live-instance swap; an explicit deploy is what changes what a future node will install. -::: +The response reports `reverted` (`false` when the target was already live), `to_deployment_id`, and `from_deployment_id` — the version taken out of service, which is also recorded as `rollback_of` on the new `hdb_deployment` row for the audit trail. + +**Only the immediately previous version is retained**, so `revert_component` reaches back exactly one activation. It fails for a component that has only ever been deployed once ("no previous version is retained"), and refuses a `to_deployment_id` that is neither live nor the retained previous, naming what the component can actually be reverted to. To return to an older version, redeploy it with `deploy_component` — that is a deploy, not a revert. + +Reverting also rewrites the component's stored `package:` reference in `harperdb-config.yaml` and its entry in the boot-time application lock, as part of the same operation. So a revert is a config-level rollback too: a node provisioned _after_ the revert — a newly joined peer, or an existing node whose components directory is rebuilt — installs the version the cluster is actually running, not the one you reverted away from. Reverting away from a `package` deploy to a payload-deployed version removes the package reference entirely, for the same reason. ### Deployment Operations From d66bcae8d07a0a199f2005e46ace6d2f42d1daed Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Mon, 17 Aug 2026 16:03:41 -0400 Subject: [PATCH 09/17] docs(deploy): move the deploy surface to v5.3.0 and badge it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The v5.2.0 placeholder this branch has carried since July is wrong, and now demonstrably so rather than provisionally. harper `main` is at **v5.2.2** (tagged 2026-08-13) and by_ref merged 2026-08-14, after that tag and in no release. The v5.3.0 upgrade directive on harper#2173 confirms which release is next. So none of this surface can ship in a 5.2.x patch: | Surface | Upstream | Ships in | | --- | --- | --- | | two-phase deploy, `revert_component`, staging retention | harper#1849 | v5.3.0 | | `deploy by_ref=true` | harper#1850 (merged, untagged) | v5.3.0 | | `deploy setup=true` | harper#1851 | v5.3.0 | | `add_ssh_key generate: true` | harper-pro#594 | v5.3.0 | The one piece that genuinely shipped in v5.2.0 — `login --for-ci` and the token env vars (harper#1876) — left this branch for #630, so nothing here is v5.2.0 any more. Addresses @kriszyp's two versioning findings. Both plain-text `Available since` lines are now ``, standalone under their headings per CONTRIBUTING.md, and the API reference no longer reads as though these behaviors exist throughout v5: - `deploy_component` — changed badge; it is existing surface whose cluster behavior changed. - `revert_component` — added badge, and the row it was missing from the Components summary table. - `add_ssh_key`'s `generate` subsection — added badge. Release-notes coverage, the other half of those findings, follows in a separate commit once the OIDC surface is in. --- reference/components/applications.md | 4 ++-- reference/operations-api/operations.md | 7 +++++++ reference/security/secrets.md | 2 +- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/reference/components/applications.md b/reference/components/applications.md index b9ef5bac3..da362964f 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -138,7 +138,7 @@ For SSH-based private repos, use the [Add SSH Key](#add_ssh_key) operation to re ### Deploying by Reference -Available since: v5.2.0 + Omitting `package` uploads a snapshot of your working directory. The result is an anonymous artifact: nothing records _which_ commit it came from, so reproducing it later — or stepping back to a previous release — means finding those exact files again. @@ -186,7 +186,7 @@ Deploying by reference means the **cluster** installs and builds the component f ### Provisioning a Deploy Credential -Available since: v5.2.0 + `harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source: diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 1d951c880..3bbefbae2 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -572,6 +572,7 @@ Detailed documentation: [Components Overview](../components/overview.md) | --------------------------- | ----------------------------------------------------------------------- | ------------- | | `add_component` | Creates a new component project from a template | super_user | | `deploy_component` | Deploys a component via payload (tar) or package reference (NPM/GitHub) | super_user | +| `revert_component` | Puts a component's retained previous version back in service | super_user | | `package_component` | Packages a component project into a base64-encoded tar | super_user | | `drop_component` | Deletes a component or a file within a component | super_user | | `get_components` | Lists all component files and config | super_user | @@ -591,6 +592,8 @@ Detailed documentation: [Components Overview](../components/overview.md) ### `deploy_component` + + Deploys a component. The `package` option accepts any valid NPM reference including GitHub repos (`HarperDB/app#semver:v1.0.0`), tarballs, or NPM packages. The `payload` option accepts a base64-encoded tar string from `package_component`. Supports `"replicated": true` and `"restart": true` or `"restart": "rolling"`. Across a cluster, `deploy_component` runs in two phases so a deploy is all-or-nothing at go-live: @@ -715,6 +718,8 @@ Stage now, activate later: ### `revert_component` + + Puts a component's **retained previous version** back in service across the cluster, then restarts. Every `deploy_component` activation retains the version it replaced, so this is a fast rollback that resolves no package, decrypts no secret, downloads no artifact and runs no install — every node already has the bytes, and the swap is a single atomic directory rename per node. This is the rollback for the bad release you just shipped: deploy a new version, run your own health checks against it, and put the old one back if you are not happy — even when the cluster otherwise looks healthy. @@ -881,6 +886,8 @@ Adding an existing key: #### Server-side key generation (`generate`) + + With `generate: true`, Harper mints an ed25519 keypair on the node handling the request and returns only the **public** half. The private key is created inside the cluster and never travels from a client, so it can't be captured in a shell history, CI log, or request body on the way in: ```json diff --git a/reference/security/secrets.md b/reference/security/secrets.md index 9b0ac045d..073f74526 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -289,7 +289,7 @@ function encryptSecret(plaintext, publicKeyPem, kid) { `deploy_component` accepts a `credentials` array so a component installed from a private **npm registry** or private **git repository** can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body, persisting as a plaintext `.npmrc`, or being written to disk for git — so package-reference deploys survive rollback, reboot, and new peers joining. Ingested tokens are stored under a derived name (`deploy..` or `deploy..git.`) granted to the component. See [`deploy_component`](../operations-api/operations.md#deploy_component). -To provision one of these without handing the cluster a plaintext token at all, `harper deploy setup=true` (v5.2.0+) runs the [client-side encryption](#client-side-encryption-encrypt-before-it-leaves-the-client) flow above for you: it fetches the public key, seals the token locally into an `enc:v1:` envelope, stores only the ciphertext under that same derived name, and prints the `credentials` reference for the deploy to use. See [Provisioning a Deploy Credential](../components/applications.md#provisioning-a-deploy-credential). +To provision one of these without handing the cluster a plaintext token at all, `harper deploy setup=true` (v5.3.0+) runs the [client-side encryption](#client-side-encryption-encrypt-before-it-leaves-the-client) flow above for you: it fetches the public key, seals the token locally into an `enc:v1:` envelope, stores only the ciphertext under that same derived name, and prints the `credentials` reference for the deploy to use. See [Provisioning a Deploy Credential](../components/applications.md#provisioning-a-deploy-credential). ## Threat model From c6f5d85e619e99b7667b6fbe9fbede2abbddcaa2 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Mon, 17 Aug 2026 16:09:45 -0400 Subject: [PATCH 10/17] docs(deploy): address the review findings on two-phase deploy and revert MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @kriszyp's eight findings, re-checked against the current state of harper#1849 and harper-pro#594 rather than the state they were filed against. Four had been fixed at the source in the meantime; four needed the docs changed. **"All-or-nothing at go-live" was wrong, and self-contradictory.** The page claimed it in the opening paragraph and then correctly described a version split twenty lines later. Reframed around what the two phases actually buy: an all-nodes staging barrier, which eliminates the common failure (a node that can't fetch or install) but not a per-node activation failure. The activation-failure paragraph is now its own `#### Activation failures` section instead of sitting mid-parameter-list, and the opening paragraph links to it. **The deploy modes are now documented as a matrix**, which core made possible by adding the validation @kriszyp asked for. `activate: false` and `deployment_id` are rejected outright when combined with `two_phase: false`, `replicated: false`, or a cluster whose `system` database isn't replicated, so a request that says stage-only can no longer activate. `two_phase: true` is likewise rejected without the replication to support it. `revert_on_failure` is refused rather than accepted and ignored. **The `system`-replication precondition is stated** — @Ethan-Arrowood's second question. A plain deploy on a cluster with `system` excluded from replication still silently takes the one-shot path, and that is the case worth warning about, because it also means no rollback target. **`revert_component`'s guarantees are scoped to the two-phase path.** The retained `.deploy-previous` copy is created by activation, so a component deployed one-shot has nothing to revert to however many times it has been deployed — the docs implied repeated deploys build up a rollback target. Also documents that `restart` is optional (a reverted component keeps serving the rolled-away-from code until something restarts it) and adds the `force` parameter the validator accepts. **The SSH-key encryption guarantee is qualified.** `sealSSHKey` in harper-pro still falls back to storing and replicating the private key in plaintext with only a WARN when no secret custody is registered, and that fallback is deliberate — SSH keys predate custody. So encryption at rest is a property of the deployment's configuration, not of the operation, and `generate: true` in particular reads as though the key could never be exposed. Custody is present by default, so the warning says that too rather than implying the risk is likely. Fixed at the source, no doc change needed: - **Explicit refs now resolve to a SHA** (harper 5c85714b8, 26cce986d, 21043f994) — the guarantee the page claimed is now real. The fuller treatment, including remote resolution and failing closed, is in #617. - **`auth_*` vs. ordinary `username=` precedence** — `main` grew a canonical numbered precedence section covering exactly this, and #630 folds the token rules into it. - **Version badges** — previous commit. --- reference/operations-api/operations.md | 67 +++++++++++++++++++++----- 1 file changed, 56 insertions(+), 11 deletions(-) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 3bbefbae2..c47abab50 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -596,12 +596,20 @@ Detailed documentation: [Components Overview](../components/overview.md) Deploys a component. The `package` option accepts any valid NPM reference including GitHub repos (`HarperDB/app#semver:v1.0.0`), tarballs, or NPM packages. The `payload` option accepts a base64-encoded tar string from `package_component`. Supports `"replicated": true` and `"restart": true` or `"restart": "rolling"`. -Across a cluster, `deploy_component` runs in two phases so a deploy is all-or-nothing at go-live: +Across a cluster, `deploy_component` runs in two phases separated by an **all-nodes staging barrier**: 1. **Stage** — the incoming version is downloaded/packed, extracted, and `npm install`ed into a hidden staging directory on **every** node, without touching the live component. -2. **Activate** — only after every node reports a successful stage does any node atomically swap the staged copy into the live path and restart. +2. **Activate** — only after every node reports a successful stage does any node atomically swap the staged copy into the live path. -If a node can't fetch the package or fails `npm install`, it fails during staging and the live component is left untouched on every node, rather than leaving part of the cluster half-updated. The request and response shape are unchanged; the two phases are internal. +The barrier is what the two phases buy you: if a node can't fetch the package or fails `npm install`, it fails during staging and the live component is left untouched on **every** node, rather than leaving part of the cluster half-updated. That is the class of failure — by far the most common one — that a two-phase deploy eliminates. + +It is not, however, all-or-nothing at go-live. Activation still happens per node, so a swap that fails on one node after others have already gone live leaves the cluster running mixed versions until you resolve it, and there is deliberately no automatic rollback — see [activation failures](#activation-failures) below. The request and response shape are unchanged; the two phases are internal. + +:::note +Two-phase deploy requires operation replication **and** replication of the `system` database, because the staging barrier is coordinated through a replicated deployment row. + +A deploy on a cluster where `system` is excluded from replication silently takes the legacy one-shot path instead: no staging barrier, and no retained previous version for [`revert_component`](#revert_component) to roll back to. The staged-phase parameters are not silently downgraded that way — `activate: false` and `deployment_id` are **rejected** with an explanatory error rather than going live unexpectedly, as is `two_phase: true` itself. +::: :::note Deploying a **brand-new** component without a restart (`"restart": false`, or omitting `restart`) marks a restart as required — `get_status` reports `restartRequired: true`, and requests to the new component's routes return an actionable 404 explaining that a restart is needed. A never-loaded component can't serve its routes until Harper restarts, so this makes that state visible instead of silent. Each node reports this for itself, since whether the component was already active can differ per node. Redeploying a component that is **already** live does not set the flag: that component's own file watcher requests a restart only if the update actually needs one. @@ -616,14 +624,34 @@ Additional parameters: - `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). No new payload is fetched or installed; the already-staged build is swapped live cluster-wide. `project` is still required. For a `package` deploy you do not need to repeat `package` here: the identifier and credential references recorded when it was staged are recovered from the deployment and persisted to root config at activation, on every node — so a later restart or a newly joined peer reinstalls the version you activated. - `ignore_replication_errors` — treat replication/peer failures as non-fatal (best-effort deploy to a partially-available cluster). This also opts out of the stage barrier. Applies to both a full deploy and a `deployment_id` activate. - `deployment_timeout` — per-deploy budget (ms) for peers to receive the replicated deployment row; defaults to 120000. - -If the activate phase fails on some nodes after others have already gone live, the deploy reports the split nodes and the deployment stays in an `activating` state rather than being rolled back automatically. Recover by rolling forward (stage and activate a known-good version) or by rolling back explicitly with [`revert_component`](#revert_component). There is deliberately no automatic rollback: once a node is past the activation barrier, a peer reporting failure does not prove that peer did not activate — it can complete its swap and then fail, or die before replying — so automatically reverting "the failed nodes" risks rolling an untouched node an extra version back and leaving the cluster split three ways instead of converging it. - - `two_phase` — set to `false` to force the legacy single-phase (in-place) deploy instead of stage-then-activate. - `credentials` — credentials for installing a component from a private npm registry or private git repository (see below) `urlPath` and `host` both require `package` and are rejected on a payload-only deploy. To mount a payload-deployed component, add `host`/`urlPath` to its entry in the root `harper-config.yaml` instead. +#### Deploy modes + +`activate`, `deployment_id`, `two_phase`, and `replicated` are not independent knobs — a request that asks for a staged phase without the machinery to support it is rejected rather than quietly doing something else. The valid combinations: + +| Request | Result | +| --- | --- | +| No mode parameters, `system` replicated | Two-phase stage → barrier → activate | +| No mode parameters, `system` **not** replicated | Legacy one-shot deploy, no retained previous version | +| `two_phase: false` | Legacy one-shot deploy, no retained previous version | +| `activate: false` | Stage only; returns a `staged` `deployment_id` | +| `deployment_id` | Activate that staged deployment cluster-wide | +| `activate: false` or `deployment_id`, with `two_phase: false`, `replicated: false`, or `system` not replicated | **Rejected** | +| `two_phase: true` with `replicated: false` or `system` not replicated | **Rejected** | +| `revert_on_failure` (any value) | **Rejected** — see [activation failures](#activation-failures) | + +`revert_on_failure` was part of an earlier draft of this operation and is now refused outright rather than accepted and ignored, so a caller that was relying on it finds out. + +#### Activation failures + +If the activate phase fails on some nodes after others have already gone live, the deploy reports the split nodes and the deployment stays in an `activating` state rather than being rolled back automatically. Recover by rolling forward (stage and activate a known-good version) or by rolling back explicitly with [`revert_component`](#revert_component). + +There is deliberately no automatic rollback. Once a node is past the activation barrier, a peer reporting failure does not prove that peer did not activate — it can complete its swap and then fail, or die before replying — so automatically reverting "the failed nodes" risks rolling an untouched node an extra version back and leaving the cluster split three ways instead of converging it. A human deciding to roll forward or back is the only step that reliably converges the cluster. + #### Deploy credentials (`credentials`) When a component is installed from a private source, `credentials` supplies the authentication. It is an array of entries; each entry is one of two kinds, identified by its key: @@ -720,10 +748,16 @@ Stage now, activate later: -Puts a component's **retained previous version** back in service across the cluster, then restarts. Every `deploy_component` activation retains the version it replaced, so this is a fast rollback that resolves no package, decrypts no secret, downloads no artifact and runs no install — every node already has the bytes, and the swap is a single atomic directory rename per node. +Puts a component's **retained previous version** back in service across the cluster. This is a fast rollback that resolves no package, decrypts no secret, downloads no artifact and runs no install — every node already has the bytes, and the swap is a single atomic directory rename per node. This is the rollback for the bad release you just shipped: deploy a new version, run your own health checks against it, and put the old one back if you are not happy — even when the cluster otherwise looks healthy. +:::caution +**Only a two-phase activation retains a previous version.** The retained copy is created by the activate phase, so `revert_component` can only roll back to a version that went live that way. + +A version deployed with `two_phase: false`, or deployed on a cluster where the `system` database is excluded from replication, leaves nothing to revert to — and the call fails, no matter how many times that component has been deployed. If rollback matters, confirm the deploy took the two-phase path rather than assuming repeated deploys have built up a rollback target. +::: + `to_deployment_id` is **required**, and names the deployment you expect to be live once the call returns: ```json @@ -741,13 +775,16 @@ Naming the target is what makes the operation safe to retry. If that version is | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | `project` | **Required.** The component to revert. | | `to_deployment_id` | **Required.** The deployment you expect to be live afterwards. `list_deployments` reports it, and `deploy_component` returns it. | -| `restart` | `true` to restart immediately, or `"rolling"` for a rolling restart. | +| `restart` | `true` to restart immediately, or `"rolling"` for a rolling restart. **Optional** — omitted, the files are swapped but Harper is not restarted. | | `ignore_replication_errors` | Treat peer failures as non-fatal. | | `deployment_timeout` | Per-operation budget (ms) for peers. | +| `force` | Bypass the safety checks on the revert target. Reserved for recovering a component whose retained state is inconsistent. | + +`restart` being optional matters more here than on a deploy: a reverted component whose code is already loaded keeps serving the version you just rolled away from until something restarts it. Pass `restart: "rolling"` unless you are deliberately batching the restart yourself. The response reports `reverted` (`false` when the target was already live), `to_deployment_id`, and `from_deployment_id` — the version taken out of service, which is also recorded as `rollback_of` on the new `hdb_deployment` row for the audit trail. -**Only the immediately previous version is retained**, so `revert_component` reaches back exactly one activation. It fails for a component that has only ever been deployed once ("no previous version is retained"), and refuses a `to_deployment_id` that is neither live nor the retained previous, naming what the component can actually be reverted to. To return to an older version, redeploy it with `deploy_component` — that is a deploy, not a revert. +**Only the immediately previous version is retained**, so `revert_component` reaches back exactly one activation. It fails with "no previous version is retained" whenever there is no retained copy — a component deployed only once, or one whose deploys took the one-shot path described above — and refuses a `to_deployment_id` that is neither live nor the retained previous, naming what the component can actually be reverted to. To return to an older version, redeploy it with `deploy_component` — that is a deploy, not a revert. Reverting also rewrites the component's stored `package:` reference in `harperdb-config.yaml` and its entry in the boot-time application lock, as part of the same operation. So a revert is a config-level rollback too: a node provisioned _after_ the revert — a newly joined peer, or an existing node whose components directory is rebuilt — installs the version the cluster is actually running, not the one you reverted away from. Reverting away from a `package` deploy to a payload-deployed version removes the package reference entirely, for the same reason. @@ -870,7 +907,15 @@ The deployment must be in a terminal status (`success`, `failed`, or `rolled_bac Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. Supply the private key with `key`, or omit it and pass `generate: true` to have Harper mint the keypair itself. -The stored private key is encrypted at rest and only ever crosses the cluster as ciphertext; `list_ssh_keys` and the logs never return key material. +`list_ssh_keys` and the logs never return key material. + +The stored private key is encrypted at rest and crosses the cluster as ciphertext **when secret custody is configured**. Custody is present by default — the file tier generates a cluster keypair on first boot — so this is the normal case. + +:::warning +On a node with **no** secret custody registered, `add_ssh_key` stores and replicates the private key in **plaintext**. It logs a WARN saying so and the operation still succeeds, because SSH keys predate custody and must keep working on a node that has none. + +That means encryption at rest is a property of your configuration, not a guarantee of the operation. If you are relying on it — and `generate: true` in particular reads as though the key can never be exposed — verify `secretCustody` is configured on every node in the cluster, and check the logs for that warning after adding a key. See [Secrets](../security/secrets.md). +::: Adding an existing key: @@ -914,7 +959,7 @@ Register that `public_key` with your git host (e.g. as a GitHub deploy key) to a `key` and `generate` are mutually exclusive — sending both is rejected. Generation happens in-process, so it requires no `ssh-keygen` binary on the host and the minted private key is never written to a temporary file on its way into storage. :::note -`public_key` is returned **only** on the generating call — that response is the one time the public half is handed back. Harper stores the sealed private key and the host config; it does not retain the public key for later retrieval, and `update_ssh_key` requires a key you supply (it can't mint one). So capture `public_key` from this response — if you lose it, `delete_ssh_key` then `add_ssh_key` with `generate: true` again to mint a fresh pair, and re-register the new public key with your git host. +`public_key` is returned **only** on the generating call — that response is the one time the public half is handed back. Harper stores the private key (sealed, subject to the custody caveat above) and the host config; it does not retain the public key for later retrieval, and `update_ssh_key` requires a key you supply (it can't mint one). So capture `public_key` from this response — if you lose it, `delete_ssh_key` then `add_ssh_key` with `generate: true` again to mint a fresh pair, and re-register the new public key with your git host. ::: --- From 95db83f47f0535dc7a96ac0bb13950edd30d2d3e Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Mon, 17 Aug 2026 16:12:40 -0400 Subject: [PATCH 11/17] =?UTF-8?q?docs(security):=20OIDC=20trusted=20publis?= =?UTF-8?q?hing=20=E2=80=94=20deploy=20from=20CI=20with=20no=20stored=20cr?= =?UTF-8?q?edential?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Companion to HarperFast/harper#2173, which had no docs PR. Four operations, a CLI credential source, and a workflow shape. `reference/operations-api/operations.md` — a new **OIDC Trusted Publishing** subsection under Token Authentication, since a trust policy is an authentication mechanism rather than a component or security-config concern: - `add_oidc_trust` / `list_oidc_trust` / `drop_oidc_trust` — super_user only, with the full parameter table, and the replace-not-merge semantics (a partial update to a claim set is how an over-broad policy gets created by accident). - `exchange_oidc_token` — unauthenticated by design, one-hour token, no refresh token issued. The parts a reader has to know and cannot guess: - **Policy specificity for GitHub Actions**, as a table of the three structural requirements — pin the repository, pin the workflow, gate the ref — with what each leaves open when unsatisfied. The ref gate is stricter than npm's model and is the rule most likely to block a real workflow, so it gets the reason: without it, anyone who can push a branch can add the trusted workflow to that branch and mint a token. - The consequences that follow: a tag-triggered release can't pin `workflow_ref` and must pin `workflow_path` plus an environment; `ref_type: tag` is not a gate; `sub` is not a valid pin for GitHub and *is* the required pin for everyone else; `pull_request_target` is denied unless the policy constrains `event_name`. - **Audience must be instance-specific**, and why `https://github.com/` is refused. - **A constrained claim absent from the token fails**, so a policy can't be weakened by an issuer that stops emitting a claim. - **Single-use tokens**, recorded as a hash before minting, and the honest account of the cross-node replay race and why it is tolerable. - **One rejection message for every failure**, with the reason in the `oidc-trust` log — otherwise the docs would send readers hunting for a specific error the endpoint will never give them. `reference/cli/authentication.md` — the exchange as precedence entry 8, plus a **Workload identity (OIDC)** section under Method 2. The emphasis is that it ranks *last*: adding `id-token: write` to a workflow that still sets `HARPER_CLI_REFRESH_TOKEN` does not change which identity deploys, which is a deliberate property someone migrating needs to know. Also covers what the CLI prints, and that an unrecognized runtime falls through silently rather than failing. Verified against the branch as it stands, not the PR description, which has drifted in three places: the code is now issuer-agnostic with GitHub as a profile (so a generic-issuer path is documented), replay keys on a token hash rather than `issuer|jti`, and the auth-audit-log gap listed as not-done is implemented. --- reference/cli/authentication.md | 35 ++++++ reference/operations-api/operations.md | 156 ++++++++++++++++++++++++- 2 files changed, 187 insertions(+), 4 deletions(-) diff --git a/reference/cli/authentication.md b/reference/cli/authentication.md index d211f20dc..0d566b95f 100644 --- a/reference/cli/authentication.md +++ b/reference/cli/authentication.md @@ -45,6 +45,7 @@ For remote Operations API commands, the CLI uses the first complete authenticati 5. `HARPER_CLI_OPERATION_TOKEN` and `HARPER_CLI_REFRESH_TOKEN` environment variables, or their legacy `CLI_TARGET_` equivalents — see [Token credentials for CI/CD](#token-credentials-for-cicd) 6. A token saved by `harper login` 7. `username=` and `password=` operation parameters (legacy fallback) +8. A [workload identity token](#workload-identity-oidc) exchanged with the runtime's OIDC provider Credentials are resolved as a pair and are never combined across sources. An incomplete pair supplied with dedicated authentication parameters or in the target URL causes the command to fail. An incomplete environment-variable pair is skipped with a warning so that a saved login token can still be used. @@ -218,6 +219,40 @@ Expose the two values to the deploy step and no other credentials are needed: Create a **dedicated CI user** and run `harper login --for-ci` as that user. That scopes the pipeline's permissions to what it actually needs, and lets you revoke its access without disturbing anyone else. ::: +##### Workload identity (OIDC) + + + +On a runner that can prove its own identity, the CLI needs **no stored credential at all**. It asks the runtime for an identity token addressed to your instance and trades it for a one-hour operation token. Nothing durable is stored in your CI provider, and there is no 30-day token to rotate. + +Configure the instance to trust the workflow once with [`add_oidc_trust`](../operations-api/operations.md#add_oidc_trust), then grant the token permission in the workflow: + +```yaml +permissions: + id-token: write + contents: read +environment: production +steps: + - run: harper deploy by_ref=true restart=true replicated=true + env: + HARPER_CLI_TARGET: ${{ vars.HARPER_CLI_TARGET }} # a var, not a secret +``` + +`HARPER_CLI_TARGET` is the only variable the step needs, and it is not sensitive — hence `vars` rather than `secrets`. + +**This ranks last, below every configured credential.** Adding `id-token: write` to a workflow that still sets `HARPER_CLI_REFRESH_TOKEN` does not change which identity deploys; the stored token keeps winning. That is deliberate — enabling a new capability should not silently re-point an existing pipeline at a different user. Remove the secret when you want the exchange to take over. + +The exchange is attempted only when the runtime actually offers an identity. GitHub Actions sets `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN` together on a job that declares `id-token: write`; without both, the CLI falls through to its other credential sources rather than reporting a failure. An unrecognized runtime falls through the same way. + +On success the CLI prints, to stderr, which policy authenticated it and as whom: + +``` +Requesting a GitHub Actions identity token for https://my-instance.harperdb.io:9925/... +Authenticated as 'ci-deploy' via OIDC trust policy 'my-app-prod'. +``` + +If Harper rejects the token it says so and continues unauthenticated, which surfaces as a 401 on the operation itself. The server deliberately does not report which check failed — see [`exchange_oidc_token`](../operations-api/operations.md#exchange_oidc_token) — so diagnose with `list_oidc_trust` and the instance's `oidc-trust` log. + #### Method 3: Dedicated Authentication Parameters diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index c47abab50..784fc8562 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -532,10 +532,14 @@ Operations for JWT token creation and refresh. Detailed documentation: [JWT Authentication](../security/jwt-authentication.md) -| Operation | Description | Role Required | -| ------------------------------ | ------------------------------------------------------- | ---------------------- | -| `create_authentication_tokens` | Creates an operation token and refresh token for a user | none (unauthenticated) | -| `refresh_operation_token` | Creates a new operation token from a refresh token | any | +| Operation | Description | Role Required | +| ------------------------------ | ------------------------------------------------------------- | ---------------------- | +| `create_authentication_tokens` | Creates an operation token and refresh token for a user | none (unauthenticated) | +| `refresh_operation_token` | Creates a new operation token from a refresh token | any | +| `exchange_oidc_token` | Trades a CI workload identity token for an operation token | none (unauthenticated) | +| `add_oidc_trust` | Creates or replaces an OIDC trust policy | super_user | +| `list_oidc_trust` | Lists all OIDC trust policies, including disabled ones | super_user | +| `drop_oidc_trust` | Deletes an OIDC trust policy | super_user | ### `create_authentication_tokens` @@ -560,6 +564,150 @@ Creates a new operation token from an existing refresh token. } ``` +### OIDC Trusted Publishing + + + +A CI runner can authenticate to Harper with **no stored credential**. It presents an identity token minted by its own provider; if that token verifies against a stored **trust policy**, Harper returns a one-hour operation token for the user the policy names. This is the same exchange npm, PyPI, and AWS STS `AssumeRoleWithWebIdentity` use. + +The alternative is a `HARPER_CLI_REFRESH_TOKEN` secret: a 30-day credential, one per user, that expires on a schedule nobody tracks. A trust policy replaces it with a rule you configure once, and revoke with `drop_oidc_trust`. + +```yaml +permissions: + id-token: write + contents: read +environment: production +steps: + - run: harper deploy by_ref=true + env: + HARPER_CLI_TARGET: ${{ vars.HARPER_CLI_TARGET }} # a var, not a secret +``` + +No secret at all — `HARPER_CLI_TARGET` is not sensitive. See [CLI Authentication](../cli/authentication.md#workload-identity-oidc) for the client half and where the exchange sits in credential precedence. + +Policies live in the replicated `system.hdb_oidc_trust` table, so configuring one on any node applies cluster-wide. + +#### `add_oidc_trust` + +Creates or replaces a trust policy. **super_user only** — a policy lets an external system authenticate as a Harper user, so granting one is equivalent to handing out a credential. + +```json +{ + "operation": "add_oidc_trust", + "id": "my-app-prod", + "issuer": "https://token.actions.githubusercontent.com", + "audience": "https://my-instance.harperdb.io:9925/", + "user": "ci-deploy", + "claims": { + "repository_id": "67890", + "workflow_ref": "HarperFast/my-app/.github/workflows/deploy.yml@refs/heads/main", + "environment": "production" + } +} +``` + +| Parameter | Description | +| ------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `id` | **Required.** Policy identifier, 1–128 characters of letters, numbers, `_`, `-`, and `.`. | +| `issuer` | **Required.** The token issuer (`iss`) this policy trusts. | +| `audience` | **Required.** The audience the token must be addressed to. Must identify **this instance** — see below. | +| `claims` | **Required.** The claim constraints a token must satisfy. At least one, and specific enough for the issuer's profile. | +| `user` | **Required.** The Harper user a matching run authenticates as. Must already exist and be active. | +| `enabled` | Defaults to `true`. A disabled policy is kept but never matched. | +| `description` | Optional free text, up to 1024 characters. | + +This **replaces** the policy rather than merging into it. A partial update is how an over-broad policy gets created by accident, and the point of `claims` is that every constraint in it was written deliberately. + +`user` is resolved at write time. A policy naming a user that does not exist, or one that is inactive, is rejected — otherwise it would fail only at exchange time, inside CI, with nothing to point at. If the named user is a **super_user**, the policy is still created but the response carries a `warning`: any run matching it gains full administrative access. + +**Claim constraints** are exact string matches. A value may be a string, or an array of strings meaning any-of: + +```json +{ "claims": { "repository": "HarperFast/my-app", "ref": ["refs/heads/main", "refs/heads/release"] } } +``` + +A constrained claim that is **absent** from the token fails rather than passes, so a policy cannot be weakened by an issuer that stops emitting a claim. + +**The audience must be instance-specific.** For GitHub Actions, `https://github.com/` is rejected: that is the provider's default, shared by every repository under the owner, so accepting it would make a token minted by any of them valid here — the exact thing the audience field exists to prevent. + +##### Policy specificity for GitHub Actions + +For `https://token.actions.githubusercontent.com`, a policy must satisfy all three of these, each closing a distinct way a policy can be accidentally broad: + +| Requirement | Satisfied by one of | Left open otherwise | +| ---------------------- | ----------------------------------------------------------------------- | -------------------------------------------- | +| **Pin the repository** | `repository_id`, `repository` | Any repository | +| **Pin the workflow** | `workflow_ref`, `workflow_path`, `job_workflow_ref`, `job_workflow_path` | Any workflow in that repository | +| **Gate the ref** | `workflow_ref`, `job_workflow_ref`, `ref`, `environment` | Any branch that can be pushed to the repository | + +The ref gate is the one worth understanding, and it is stricter than npm's model. Pinning repository and workflow without also pinning a ref is not safe: anyone who can push a branch can add the trusted workflow to that branch and mint a token. npm accepts that shape and relies on environment protection instead. + +Consequences worth planning around: + +- **`repository_id` is preferred over `repository`** because it is immutable — it survives a repository rename, and is immune to org-name recycling. +- **A tag-triggered release cannot pin `workflow_ref`**, since the tag is unknown when the policy is written. Pin `workflow_path` instead — Harper derives it from `workflow_ref` by removing the ref — and gate on `environment`. +- **`ref_type: tag` is deliberately not accepted as a ref gate.** Anyone with push access can create a tag. +- **`sub` is not accepted as a pin.** It varies by trigger, and its format changed for repositories created after 2026-07-15 (immutable subjects embed owner and repository ids), so a policy pinning it would have to handle two shapes indefinitely. +- **`pull_request_target` runs are denied** unless the policy explicitly constrains `event_name`. Such a run executes the base repository's workflow, with its secrets, while a fork controls the checked-out code. A plain `pull_request` run from a fork cannot mint at all, since it gets no `id-token: write`. + +##### Other issuers + +An issuer with no registered profile gets a strict generic profile: the policy must pin **`sub`**. That is the one claim every OIDC issuer defines as identifying a single principal, and it makes workload identity work with no provider-specific code — a Kubernetes service-account token (`system:serviceaccount::`), a GCP service account, and a SPIFFE SVID all carry a stable canonical subject. + +GitHub Actions needs its own profile precisely because its `sub` is the one claim you should *not* pin. + +#### `exchange_oidc_token` + +Trades an identity token for a Harper operation token. **Unauthenticated by design** — this operation *is* the authentication, the way `create_authentication_tokens` is against a password. The CLI calls it for you; you would call it directly only from a client that mints its own requests. + +```json +{ + "operation": "exchange_oidc_token", + "token": "eyJhbGciOi..." +} +``` + +Response: + +```json +{ + "operation_token": "eyJhbGciOi...", + "expires_in": 3600, + "username": "ci-deploy", + "policy": "my-app-prod" +} +``` + +The operation token is valid for **one hour** — long enough to cover a slow deploy, short enough to be worthless by the time it reaches a log. No refresh token is issued; a subsequent run performs a new exchange. + +:::note +**Every rejection returns the same message.** The endpoint is unauthenticated, so a caller told which check failed could enumerate a policy one claim at a time. The specific reason is written to the `oidc-trust` logger, which is where to look when a workflow that should match does not. +::: + +**An identity token can be exchanged once.** Harper records a SHA-256 fingerprint of each spent token in `system.hdb_oidc_token_use`, expiring with the token itself, so the table stays proportional to in-flight tokens and never holds a credential. The record is written *before* the operation token is minted: if minting then fails the identity token is burned, costing a CI re-run, where the reverse order would leave a spendable token behind. + +That replay check is replicated, but replication is asynchronous, so two simultaneous replays against **different nodes** can both succeed. This is not a privilege escalation — whoever holds the token could obtain one operation token regardless — and what it does stop is the realistic case: a token that leaks after a legitimate run and is reused inside its window. + +Exchanges are recorded in the authentication audit stream alongside Basic, Bearer, and mTLS events, for failures as well as successes — a run repeatedly failing to authenticate is what an audit trail is for. Enable it with `logging.auditAuthEvents.logSuccessful` and `logging.auditAuthEvents.logFailed`. + +#### `list_oidc_trust` + +Lists every policy, **including disabled ones**, sorted by `id`. **super_user only** — the policy set names exactly which repository and workflow are worth compromising. + +```json +{ "operation": "list_oidc_trust" } +``` + +Returns `{ "policies": [ ... ] }`. Each entry carries `id`, `issuer`, `audience`, `claims`, `user`, `enabled`, `description`, `updated_by`, and timestamps. + +#### `drop_oidc_trust` + +Deletes a policy, revoking every workflow that matched it. **super_user only.** Fails with `404` if no policy has that `id`. + +```json +{ "operation": "drop_oidc_trust", "id": "my-app-prod" } +``` + --- ## Components From ffc0361a522b7c86a58800d34f824fa06ed7d7d0 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Mon, 17 Aug 2026 16:13:45 -0400 Subject: [PATCH 12/17] docs(release-notes): add the 5.3 page covering the deploy and OIDC surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes @kriszyp's discoverability findings. The reference pages carry version badges now, but a user who only reads release notes had no way to find any of this — and there was no 5.3 page at all. The directory is autogenerated into the sidebar, so the new page needs no registration. Covers, in the house style of 5.2.md: two-phase cluster deploys, stage-now/activate-later, component rollback, deploying by git reference, sealed deploy credentials, OIDC trusted publishing, and server-side SSH key generation. Each entry states the limits alongside the capability rather than leaving them to the reference page, because these are the ones that change whether a reader should adopt the feature: the two-phase barrier is not an all-or-nothing go-live, it silently degrades on a cluster whose `system` database isn't replicated, rollback reaches back exactly one two-phase activation, and the GitHub Actions ref-gate rule will refuse policy shapes that npm accepts. `login --for-ci` and the token env vars are not here — they shipped in v5.2.0 and belong in 5.2.md, which is #630's. --- release-notes/v5-lincoln/5.3.md | 53 +++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 release-notes/v5-lincoln/5.3.md diff --git a/release-notes/v5-lincoln/5.3.md b/release-notes/v5-lincoln/5.3.md new file mode 100644 index 000000000..805292d19 --- /dev/null +++ b/release-notes/v5-lincoln/5.3.md @@ -0,0 +1,53 @@ +--- +title: '5.3' +--- + +# 5.3 Release Notes + +### Patch Releases + +All patch release notes for 5.3.x are available on the [releases page](https://github.com/HarperFast/harper/releases?q=v5.3&expanded=true). + +## Deployment + +### Two-Phase Cluster Deploys + +`deploy_component` now runs across a cluster in two phases separated by an all-nodes staging barrier. The incoming version is downloaded, extracted, and `npm install`ed into a hidden staging directory on **every** node before any node touches its live component; only once every node reports a successful stage does the swap happen. A node that can't fetch the package or fails its install now fails during staging, leaving the live component untouched everywhere, instead of leaving part of the cluster half-updated. The request and response shape are unchanged. + +Activation is still per node, so this is a staging barrier rather than an all-or-nothing go-live: a swap that fails on one node after others have gone live leaves the cluster on mixed versions, reported as an `activating` deployment for a human to roll forward or back. There is deliberately no automatic rollback — past the barrier, a peer reporting failure does not prove it did not activate, so reverting "the failed nodes" can split the cluster three ways instead of converging it. + +Two-phase deploy requires operation replication and replication of the `system` database. A cluster with `system` excluded from replication silently takes the legacy one-shot path, with no staging barrier and no rollback target; the staged-phase parameters are rejected outright rather than downgraded. See [`deploy_component`](/reference/v5/operations-api/operations#deploy_component). + +### Stage Now, Activate Later + +`deploy_component` accepts `activate: false` to prepare and verify a release on every node without going live, returning a `staged` deployment ID. A later call passing that `deployment_id` swaps it in cluster-wide with no re-fetch and no re-install, turning go-live into a fast, separate step. Harper retains the most recent staged builds per component (default 5, `deployment_stagingRetention_maxCount`); a `deployment_id` that has aged out fails rather than silently rebuilding. `harper stage` and `harper activate` are the CLI verbs. See [Deploy modes](/reference/v5/operations-api/operations#deploy-modes). + +### Component Rollback + +The new `revert_component` operation puts a component's retained previous version back in service across the cluster. Every two-phase activation retains the version it replaced, so a rollback resolves no package, decrypts no secret, downloads no artifact and runs no install — every node already has the bytes, and the swap is one atomic directory rename per node. It rewrites the stored `package:` reference and the boot-time application lock in the same operation, so a newly joined peer or a rebuilt components directory installs the version the cluster is actually running. + +`to_deployment_id` is required and names the deployment you expect to be live afterwards, which makes the operation idempotent: a client that loses the response and retries cannot flip a rejected release back in. Only the immediately previous version is retained, and only a two-phase activation creates that retained copy. `harper revert` is the CLI verb. See [`revert_component`](/reference/v5/operations-api/operations#revert_component). + +### Deploying by Git Reference + +`harper deploy by_ref=true` builds a package reference from the local git repository, so a deploy records which commit it came from instead of shipping an anonymous snapshot of the working directory. Redeploying the same reference is an exact redeploy, and rolling back is deploying an older one. + +Tags and branches are resolved to a full commit SHA before the deploy is sent — from the local checkout when it has the ref, and from the remote when it doesn't, as a shallow CI clone usually doesn't. A ref that can't be resolved either way stops the deploy rather than being sent by name, because peers resolve the package independently and a name that moves mid-deploy would leave nodes on different code. The CLI warns when the working tree is dirty and when the commit isn't on any remote branch. Under GitHub Actions on a `pull_request` run it deploys the pull request's head commit, not the merge commit the runner checks out. See [Deploying by Reference](/reference/v5/components/applications#deploying-by-reference). + +### Sealed Deploy Credentials + +`harper deploy setup=true` provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an `enc:v1:` envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. Later deploys pass `credential=true` and the host is derived from the package being deployed, so the credential always matches the clone it authenticates. See [Provisioning a Deploy Credential](/reference/v5/components/applications#provisioning-a-deploy-credential). + +## Security + +### OIDC Trusted Publishing + +A CI runner can now authenticate to Harper with no stored credential. It presents an identity token minted by its own provider, and if that token verifies against a trust policy configured on the instance, Harper returns a one-hour operation token for the user the policy names — the same exchange npm, PyPI, and AWS STS `AssumeRoleWithWebIdentity` use. This replaces a 30-day refresh-token secret with a rule you configure once and revoke with `drop_oidc_trust`. + +Policies are managed with `add_oidc_trust`, `list_oidc_trust`, and `drop_oidc_trust` (all super_user), and live in the replicated `system.hdb_oidc_trust` table. A policy pins the issuer, an instance-specific audience, and a set of claim constraints. For GitHub Actions those constraints must pin the repository, pin the workflow, and gate the ref — a policy naming only a repository and workflow is refused, because anyone who can push a branch could otherwise add the trusted workflow to it and mint a token. Issuers with no registered profile must pin `sub`, which makes Kubernetes service accounts, GCP service accounts, and SPIFFE identities work with no provider-specific configuration. + +Identity tokens are single-use, exchanges are recorded in the authentication audit stream, and every rejection returns the same message with the specific reason written to the `oidc-trust` log. In the CLI the exchange ranks below every configured credential, so enabling it does not silently re-point a pipeline that still has a token secret set. See [OIDC Trusted Publishing](/reference/v5/operations-api/operations#oidc-trusted-publishing) and [Workload identity](/reference/v5/cli/authentication#workload-identity-oidc). + +### Server-Side SSH Key Generation + +`add_ssh_key` accepts `generate: true` to have Harper mint an ed25519 keypair itself and return only the public half, so a deploy key's private half is never carried in a request body, shell history, or CI log. Generation happens in process, requiring no `ssh-keygen` binary on the host. The public key is returned only on the generating call — Harper does not retain it — and is commented `harper:` so it is identifiable in the git host's key list. See [Server-side key generation](/reference/v5/operations-api/operations#server-side-key-generation-generate). From 627d8c8cd25027b913dadadc88131fba60b8780a Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Mon, 17 Aug 2026 16:15:14 -0400 Subject: [PATCH 13/17] docs(deploy): resync deploy setup=true with harper#1851 after review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three things this section described had changed upstream since it was written. **The recommended token is now a fine-grained PAT, not the `gh` session token.** Review on #1851 pointed out that offering the `gh` token first made it the default selection — putting least privilege on the path of most resistance, next to guidance recommending Contents: Read-only on one repo. The prompt now leads with the PAT and warns when `gh` is chosen. The docs listed both neutrally, `gh` first, so they now lead with the PAT and say why it matters here specifically: what this flow seals is durable and replayed on every cold deploy and rollback. **It is two calls, not one.** `set_secret` stores the ciphertext and `grant_secret` grants the component in a separate locked step; the client-side grant merge was removed because it opened a TOCTOU with `revoke_secret`. The docs said "granted to the component" as though `set_secret` did both. **The tier is now explicit.** `set_secret` sends `processEnv: false`, so a pre-existing global row at the derived name is converted to the component-scoped tier rather than the pasted token being written into the tier every component and child process reads. Worth stating, since the difference is invisible from the CLI output and is what "Granted to component X" actually promises. --- reference/components/applications.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/reference/components/applications.md b/reference/components/applications.md index da362964f..e24716643 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -194,15 +194,20 @@ Deploying by reference means the **cluster** installs and builds the component f harper deploy setup=true ``` -It asks which private source needs a credential (a GitHub repository or an npm registry) and sources a token — from your `gh` CLI session, or one you paste — then: +It asks which private source needs a credential (a GitHub repository or an npm registry), sources a token, and then: 1. Fetches the cluster's public key with `get_secrets_public_key`. 2. **Encrypts the token locally** into an `enc:v1:` envelope. -3. Stores only the ciphertext via `set_secret`, granted to the component. -4. Prints the `credentials` reference for the deploy to use. +3. Stores only the ciphertext with `set_secret`, in the component-scoped tier. +4. Grants this component permission to resolve it with `grant_secret`. +5. Prints the `credentials` reference for the deploy to use. The plaintext never leaves your machine: the operations API, its logs, and replication only ever carry the envelope, and the cluster decrypts it in memory at deploy time. This requires a cluster with secrets custody (Harper Pro / Fabric) — see [Client-side encryption](../security/secrets.md#client-side-encryption-encrypt-before-it-leaves-the-client). +**Prefer a fine-grained PAT.** For a GitHub repository the prompt offers, and defaults to, pasting a fine-grained personal access token with **Contents: Read-only on that one repository**. If you have the `gh` CLI authenticated it also offers its session token, which is one keypress cheaper but typically carries `repo`, `read:org`, `gist`, and `workflow` scopes across your whole account; choosing it prints a warning. What this flow seals is durable and replayed on every cold deploy and rollback, so it is worth being the narrowest credential that does the job. + +The secret is stored **scoped to the component**, never in the global `processEnv` tier that every component and child process can read. If a global secret already exists at the derived name, it is converted to the scoped tier — the name is derived from the component, so a global secret there was never serving anything the scoped one doesn't. Existing grants on the row are preserved. + Because the stored token is durable, later deploys — including re-fetching an older reference — reuse it without re-entering anything. :::note From 4d1084d19fdfed3c607ae901e8ece3f5b5f35014 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 18 Aug 2026 12:06:56 -0400 Subject: [PATCH 14/17] docs: correct credential precedence, revert force, and barrier opt-outs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five defects found by a claim-versus-source pass over this branch's own diff. The first is the one that mattered. **The OIDC exchange was documented one position too low.** The precedence list put it at 8, below the legacy `username=`/`password=` fallback, and the prose said it "ranks last". In `bin/cliOperations.ts` the exchange is an `else if` on the token branch, so it runs before the legacy fallback, which is gated on no Authorization header already being set. Real order is 7 = exchange, 8 = legacy. That inversion matters in the same direction as the v5.2.0 precedence change: on a runner with `id-token: write` and no configured credential, `harper add_user username=svc password=... target=...` authenticates as the trust policy's user, not as `svc`. The docs promised the opposite. Now stated, including the migration hazard for a script that was passing those fields as credentials. **`force` on `revert_component` was described with the wrong semantics** — "bypass the safety checks on the revert target". It is consumed only by `assertNotProtectedCoreComponent`, so it permits writing over a protected core component name and has no effect on `to_deployment_id` validation. As written it invited reaching for `force` when a revert is refused for naming an un-revertable target, where it cannot help, while hiding what it does do. **The deploy-modes table omitted the two knobs that defeat the barrier.** The section calls `replicated` one of the interacting parameters and then never gives it a row, and `ignore_replication_errors: true` makes stage failures non-fatal (`operations.js:875`) so activation proceeds despite the barrier having failed. The barrier is the feature's headline guarantee; its opt-out belongs in the table a reader consults to decide which mode is safe, not only in a parameter bullet. **The blank-token paragraph contradicted itself** — "an error, not a fallback", then described the fallback, then denied it. The CLI warns and continues down the precedence list, which does mean running as whoever last logged in; the warning is the only thing that makes it non-silent. Rewritten to say that, and to treat the warning as a CI failure signal. **Audience instance-specificity was stated as universal validation.** Only the GitHub profile enforces it; the generic profile's `assertAudienceIsSpecific` is empty. Scoped, with a pointer to the `sub` pin that binds a generic policy — the same guidance-versus-enforcement distinction this branch just applied to the SSH-key encryption claim. Also: `reference/operations-api/operations.md` was unformatted and would have failed CI. `npm run format` is `prettier .` with no `--write`, so it prints and changes nothing; the gate is `format:write` / `format:check`, per AGENTS.md. --- reference/cli/authentication.md | 12 ++-- reference/operations-api/operations.md | 88 +++++++++++++------------- 2 files changed, 52 insertions(+), 48 deletions(-) diff --git a/reference/cli/authentication.md b/reference/cli/authentication.md index 0d566b95f..ba790cd21 100644 --- a/reference/cli/authentication.md +++ b/reference/cli/authentication.md @@ -44,12 +44,12 @@ For remote Operations API commands, the CLI uses the first complete authenticati 4. Legacy `CLI_TARGET_USERNAME` and `CLI_TARGET_PASSWORD` environment variables 5. `HARPER_CLI_OPERATION_TOKEN` and `HARPER_CLI_REFRESH_TOKEN` environment variables, or their legacy `CLI_TARGET_` equivalents — see [Token credentials for CI/CD](#token-credentials-for-cicd) 6. A token saved by `harper login` -7. `username=` and `password=` operation parameters (legacy fallback) -8. A [workload identity token](#workload-identity-oidc) exchanged with the runtime's OIDC provider +7. A [workload identity token](#workload-identity-oidc) exchanged with the runtime's OIDC provider +8. `username=` and `password=` operation parameters (legacy fallback) Credentials are resolved as a pair and are never combined across sources. An incomplete pair supplied with dedicated authentication parameters or in the target URL causes the command to fail. An incomplete environment-variable pair is skipped with a warning so that a saved login token can still be used. -Entries 5 and 6 authenticate with a bearer token and apply to **remote targets only**. A local operation goes over the domain socket, which the server already trusts, so token environment variables are deliberately ignored there — a token minted for one instance would otherwise be attached to every local `harper` command in that shell and rejected. +Entries 5 through 7 authenticate with a bearer token and apply to **remote targets only**. A local operation goes over the domain socket, which the server already trusts, so token environment variables are deliberately ignored there — a token minted for one instance would otherwise be attached to every local `harper` command in that shell and rejected. Before v5.2.0, `username=` and `password=` operation parameters took precedence over environment variables and saved login tokens. This could authenticate an operation as the wrong user when those fields were part of the operation payload, such as the user being created by `add_user`. @@ -209,7 +209,7 @@ Expose the two values to the deploy step and no other credentials are needed: **Refresh behavior.** The CLI mints an operation token from the refresh token when none is supplied, and again whenever the supplied one has expired. A token refreshed from an environment variable is held in memory for that invocation only — nothing is written to `~/.harperdb/credentials.json`, because there is no file entry for an environment-supplied credential. If the refresh token itself is rejected, the command reports that and exits non-zero rather than falling back to another identity. -**A blank token variable is an error, not a fallback.** If a namespace is set but empty — the usual shape of a misconfigured CI secret — the CLI says so and falls back to saved login credentials. It does not silently run as whoever last logged in on that machine. +**A blank token variable is reported, then skipped.** If a namespace is set but empty — the usual shape of a misconfigured CI secret — the CLI warns and continues down the precedence list, so the run proceeds under the saved `harper login` token if that machine has one. The warning is the only thing separating this from silently deploying as whoever last logged in, so treat it as a failure signal in CI rather than assuming a blank secret stops the run. **Lifetimes.** Operation tokens expire after `authentication.operationTokenTimeout` (default `1d`) and refresh tokens after `authentication.refreshTokenTimeout` (default `30d`). The pipeline needs a new refresh token when that window closes. @@ -240,7 +240,9 @@ steps: `HARPER_CLI_TARGET` is the only variable the step needs, and it is not sensitive — hence `vars` rather than `secrets`. -**This ranks last, below every configured credential.** Adding `id-token: write` to a workflow that still sets `HARPER_CLI_REFRESH_TOKEN` does not change which identity deploys; the stored token keeps winning. That is deliberate — enabling a new capability should not silently re-point an existing pipeline at a different user. Remove the secret when you want the exchange to take over. +**This ranks below every configured credential**, and above only the legacy `username=`/`password=` payload fallback. Adding `id-token: write` to a workflow that still sets `HARPER_CLI_REFRESH_TOKEN` does not change which identity deploys; the stored token keeps winning. That is deliberate — enabling a new capability should not silently re-point an existing pipeline at a different user. Remove the secret when you want the exchange to take over. + +Note the one case where the exchange _does_ take over: because it outranks the legacy fallback, a command that passes `username=` and `password=` as operation parameters on a runner with no configured credential authenticates as the trust policy's user, not as the pair in the payload. That is the intended reading of those fields — for `add_user` they describe the user being created — but it means a script relying on them as credentials changes identity the moment a policy matches. The exchange is attempted only when the runtime actually offers an identity. GitHub Actions sets `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN` together on a job that declares `id-token: write`; without both, the CLI falls through to its other credential sources rather than reporting a failure. An unrecognized runtime falls through the same way. diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 784fc8562..a915b0a2f 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -532,14 +532,14 @@ Operations for JWT token creation and refresh. Detailed documentation: [JWT Authentication](../security/jwt-authentication.md) -| Operation | Description | Role Required | -| ------------------------------ | ------------------------------------------------------------- | ---------------------- | -| `create_authentication_tokens` | Creates an operation token and refresh token for a user | none (unauthenticated) | -| `refresh_operation_token` | Creates a new operation token from a refresh token | any | -| `exchange_oidc_token` | Trades a CI workload identity token for an operation token | none (unauthenticated) | -| `add_oidc_trust` | Creates or replaces an OIDC trust policy | super_user | -| `list_oidc_trust` | Lists all OIDC trust policies, including disabled ones | super_user | -| `drop_oidc_trust` | Deletes an OIDC trust policy | super_user | +| Operation | Description | Role Required | +| ------------------------------ | ---------------------------------------------------------- | ---------------------- | +| `create_authentication_tokens` | Creates an operation token and refresh token for a user | none (unauthenticated) | +| `refresh_operation_token` | Creates a new operation token from a refresh token | any | +| `exchange_oidc_token` | Trades a CI workload identity token for an operation token | none (unauthenticated) | +| `add_oidc_trust` | Creates or replaces an OIDC trust policy | super_user | +| `list_oidc_trust` | Lists all OIDC trust policies, including disabled ones | super_user | +| `drop_oidc_trust` | Deletes an OIDC trust policy | super_user | ### `create_authentication_tokens` @@ -606,15 +606,15 @@ Creates or replaces a trust policy. **super_user only** — a policy lets an ext } ``` -| Parameter | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------ | -| `id` | **Required.** Policy identifier, 1–128 characters of letters, numbers, `_`, `-`, and `.`. | -| `issuer` | **Required.** The token issuer (`iss`) this policy trusts. | -| `audience` | **Required.** The audience the token must be addressed to. Must identify **this instance** — see below. | -| `claims` | **Required.** The claim constraints a token must satisfy. At least one, and specific enough for the issuer's profile. | -| `user` | **Required.** The Harper user a matching run authenticates as. Must already exist and be active. | -| `enabled` | Defaults to `true`. A disabled policy is kept but never matched. | -| `description` | Optional free text, up to 1024 characters. | +| Parameter | Description | +| ------------- | -------------------------------------------------------------------------------------------------------------------------- | +| `id` | **Required.** Policy identifier, 1–128 characters of letters, numbers, `_`, `-`, and `.`. | +| `issuer` | **Required.** The token issuer (`iss`) this policy trusts. | +| `audience` | **Required.** The audience the token must be addressed to. Should identify **this instance**; enforced for GitHub Actions. | +| `claims` | **Required.** The claim constraints a token must satisfy. At least one, and specific enough for the issuer's profile. | +| `user` | **Required.** The Harper user a matching run authenticates as. Must already exist and be active. | +| `enabled` | Defaults to `true`. A disabled policy is kept but never matched. | +| `description` | Optional free text, up to 1024 characters. | This **replaces** the policy rather than merging into it. A partial update is how an over-broad policy gets created by accident, and the point of `claims` is that every constraint in it was written deliberately. @@ -628,17 +628,17 @@ This **replaces** the policy rather than merging into it. A partial update is ho A constrained claim that is **absent** from the token fails rather than passes, so a policy cannot be weakened by an issuer that stops emitting a claim. -**The audience must be instance-specific.** For GitHub Actions, `https://github.com/` is rejected: that is the provider's default, shared by every repository under the owner, so accepting it would make a token minted by any of them valid here — the exact thing the audience field exists to prevent. +**The audience should be instance-specific.** For GitHub Actions this is enforced: `https://github.com/` is rejected, because that is the provider's default, shared by every repository under the owner, so accepting it would make a token minted by any of them valid here — the exact thing the audience field exists to prevent. For an issuer with no registered profile the audience is not checked for specificity, and the required `sub` pin is what binds the policy to one principal instead. ##### Policy specificity for GitHub Actions For `https://token.actions.githubusercontent.com`, a policy must satisfy all three of these, each closing a distinct way a policy can be accidentally broad: -| Requirement | Satisfied by one of | Left open otherwise | -| ---------------------- | ----------------------------------------------------------------------- | -------------------------------------------- | -| **Pin the repository** | `repository_id`, `repository` | Any repository | -| **Pin the workflow** | `workflow_ref`, `workflow_path`, `job_workflow_ref`, `job_workflow_path` | Any workflow in that repository | -| **Gate the ref** | `workflow_ref`, `job_workflow_ref`, `ref`, `environment` | Any branch that can be pushed to the repository | +| Requirement | Satisfied by one of | Left open otherwise | +| ---------------------- | ------------------------------------------------------------------------ | ----------------------------------------------- | +| **Pin the repository** | `repository_id`, `repository` | Any repository | +| **Pin the workflow** | `workflow_ref`, `workflow_path`, `job_workflow_ref`, `job_workflow_path` | Any workflow in that repository | +| **Gate the ref** | `workflow_ref`, `job_workflow_ref`, `ref`, `environment` | Any branch that can be pushed to the repository | The ref gate is the one worth understanding, and it is stricter than npm's model. Pinning repository and workflow without also pinning a ref is not safe: anyone who can push a branch can add the trusted workflow to that branch and mint a token. npm accepts that shape and relies on environment protection instead. @@ -654,11 +654,11 @@ Consequences worth planning around: An issuer with no registered profile gets a strict generic profile: the policy must pin **`sub`**. That is the one claim every OIDC issuer defines as identifying a single principal, and it makes workload identity work with no provider-specific code — a Kubernetes service-account token (`system:serviceaccount::`), a GCP service account, and a SPIFFE SVID all carry a stable canonical subject. -GitHub Actions needs its own profile precisely because its `sub` is the one claim you should *not* pin. +GitHub Actions needs its own profile precisely because its `sub` is the one claim you should _not_ pin. #### `exchange_oidc_token` -Trades an identity token for a Harper operation token. **Unauthenticated by design** — this operation *is* the authentication, the way `create_authentication_tokens` is against a password. The CLI calls it for you; you would call it directly only from a client that mints its own requests. +Trades an identity token for a Harper operation token. **Unauthenticated by design** — this operation _is_ the authentication, the way `create_authentication_tokens` is against a password. The CLI calls it for you; you would call it directly only from a client that mints its own requests. ```json { @@ -684,7 +684,7 @@ The operation token is valid for **one hour** — long enough to cover a slow de **Every rejection returns the same message.** The endpoint is unauthenticated, so a caller told which check failed could enumerate a policy one claim at a time. The specific reason is written to the `oidc-trust` logger, which is where to look when a workflow that should match does not. ::: -**An identity token can be exchanged once.** Harper records a SHA-256 fingerprint of each spent token in `system.hdb_oidc_token_use`, expiring with the token itself, so the table stays proportional to in-flight tokens and never holds a credential. The record is written *before* the operation token is minted: if minting then fails the identity token is burned, costing a CI re-run, where the reverse order would leave a spendable token behind. +**An identity token can be exchanged once.** Harper records a SHA-256 fingerprint of each spent token in `system.hdb_oidc_token_use`, expiring with the token itself, so the table stays proportional to in-flight tokens and never holds a credential. The record is written _before_ the operation token is minted: if minting then fails the identity token is burned, costing a CI re-run, where the reverse order would leave a spendable token behind. That replay check is replicated, but replication is asynchronous, so two simultaneous replays against **different nodes** can both succeed. This is not a privilege escalation — whoever holds the token could obtain one operation token regardless — and what it does stop is the realistic case: a token that leaks after a legitimate run and is reused inside its window. @@ -781,16 +781,18 @@ Additional parameters: `activate`, `deployment_id`, `two_phase`, and `replicated` are not independent knobs — a request that asks for a staged phase without the machinery to support it is rejected rather than quietly doing something else. The valid combinations: -| Request | Result | -| --- | --- | -| No mode parameters, `system` replicated | Two-phase stage → barrier → activate | -| No mode parameters, `system` **not** replicated | Legacy one-shot deploy, no retained previous version | -| `two_phase: false` | Legacy one-shot deploy, no retained previous version | -| `activate: false` | Stage only; returns a `staged` `deployment_id` | -| `deployment_id` | Activate that staged deployment cluster-wide | -| `activate: false` or `deployment_id`, with `two_phase: false`, `replicated: false`, or `system` not replicated | **Rejected** | -| `two_phase: true` with `replicated: false` or `system` not replicated | **Rejected** | -| `revert_on_failure` (any value) | **Rejected** — see [activation failures](#activation-failures) | +| Request | Result | +| -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | +| No mode parameters, `system` replicated | Two-phase stage → barrier → activate | +| No mode parameters, `system` **not** replicated | Legacy one-shot deploy, no retained previous version | +| `two_phase: false` | Legacy one-shot deploy, no retained previous version | +| `replicated: false` | Legacy one-shot deploy on this node only, no retained previous version | +| `ignore_replication_errors: true` | Stage barrier **not enforced** — a node that fails to stage no longer blocks activation | +| `activate: false` | Stage only; returns a `staged` `deployment_id` | +| `deployment_id` | Activate that staged deployment cluster-wide | +| `activate: false` or `deployment_id`, with `two_phase: false`, `replicated: false`, or `system` not replicated | **Rejected** | +| `two_phase: true` with `replicated: false` or `system` not replicated | **Rejected** | +| `revert_on_failure` (any value) | **Rejected** — see [activation failures](#activation-failures) | `revert_on_failure` was part of an earlier draft of this operation and is now refused outright rather than accepted and ignored, so a caller that was relying on it finds out. @@ -919,14 +921,14 @@ A version deployed with `two_phase: false`, or deployed on a cluster where the ` Naming the target is what makes the operation safe to retry. If that version is **already live**, the call succeeds without changing anything — so a client that loses the response and retries cannot flip the rejected release back in. If it matches the retained previous version, the swap happens and the version it displaced becomes the new retained previous, so an explicitly targeted revert of a revert rolls forward again. -| Parameter | Description | -| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -| `project` | **Required.** The component to revert. | -| `to_deployment_id` | **Required.** The deployment you expect to be live afterwards. `list_deployments` reports it, and `deploy_component` returns it. | +| Parameter | Description | +| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| `project` | **Required.** The component to revert. | +| `to_deployment_id` | **Required.** The deployment you expect to be live afterwards. `list_deployments` reports it, and `deploy_component` returns it. | | `restart` | `true` to restart immediately, or `"rolling"` for a rolling restart. **Optional** — omitted, the files are swapped but Harper is not restarted. | -| `ignore_replication_errors` | Treat peer failures as non-fatal. | -| `deployment_timeout` | Per-operation budget (ms) for peers. | -| `force` | Bypass the safety checks on the revert target. Reserved for recovering a component whose retained state is inconsistent. | +| `ignore_replication_errors` | Treat peer failures as non-fatal. | +| `deployment_timeout` | Per-operation budget (ms) for peers. | +| `force` | Permit the operation on a protected core component name. Does **not** relax the `to_deployment_id` checks. | `restart` being optional matters more here than on a deploy: a reverted component whose code is already loaded keeps serving the version you just rolled away from until something restarts it. Pass `restart: "rolling"` unless you are deliberately batching the restart yourself. From e1da667787bf35ed031fd17ceb7e052976dfb685 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 18 Aug 2026 12:10:47 -0400 Subject: [PATCH 15/17] docs(agents): derive version badges from core tags, not the branch package.json The existing guidance says to derive the badge from "the core release the change ships in" and points at prerelease suffixes, which reads as an instruction to look at the feature branch's package.json. That is what went wrong on this PR: the branch said `5.2.0-beta.3`, so a `v5.2.0` badge looked confirmed, and it stayed wrong for about a month while 5.2.0, 5.2.1 and 5.2.2 were all cut without the feature. The whole surface had become v5.3.0. Records the two commands that actually answer the question and the re-check obligation for a long-lived docs PR, since a release cut between refresh passes invalidates a badge that was right when it was written. --- AGENTS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/AGENTS.md b/AGENTS.md index 3ecf2812d..f82b28f7a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,6 +54,7 @@ Prefer plain ASCII characters in Markdown unless a typographic character is genu - Tag minor-version availability inline: `` for new surface, `` for behavior changes to existing surface. - Derive the version from the core release the change ships in, stripping prerelease suffixes (`5.1.0-beta.1` → `v5.1.0`). +- **Determine that release from the core repo's git tags, not from the feature branch's `package.json`.** A branch reading `5.2.0-beta.3` says which release was open when the branch started, not which one the change lands in — if a release is cut before the feature merges, the badge is silently wrong. Check `git tag --sort=-creatordate` for the newest release, and `git tag --contains ` for whether the change is in one; a merged-but-untagged feature ships in the _next_ version, which may be a minor bump. Re-check on every refresh pass of a long-lived docs PR, because a release cut between passes invalidates a badge that was correct when written. - Each minor release gets a file under `release-notes//` (e.g. `release-notes/v5-lincoln/5.1.md`); the sidebar picks it up automatically. - Absolute links from `release-notes/` (or `learn/`) into current reference docs use the versioned path `/reference/v5/...` — the reference plugin maps the current version to the `v5` URL path. - When documenting a change from a core/pro PR, cross-link the feature PR and the docs PR in both descriptions. From 5d4ce0cae618bc69a3ad25d77c7f2b120df4bcf3 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 18 Aug 2026 12:25:21 -0400 Subject: [PATCH 16/17] docs: resync OIDC with harper#2173's token scoping, and qualify eight claims MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (codex leg) against 4d1084d1 raised 13 findings. Three were false positives from reading the wrong tree — it checked `revert_component` against harper `main`, where harper#1849 is unmerged, so the operation looks deleted; it read `add_ssh_key generate` before ab11290 moved keygen in-process; and its claim that activation rebuilds a missing staged tree is the opposite of `activateStagedApplication`, which throws. The other ten were real. The largest is that harper#2173 was rebased and gained eleven commits adding an operation-scoping feature this branch never documented. **`operations` on a trust policy.** 1-100 unique names, validated at write time against the registry `add_role` uses, carried into the minted token as a narrowing-only intersection with the user's role. Documented with the two things that decide whether it is the control an operator thinks it is: - It covers the Operations API and SQL (`verifyPerms`/`verifyPermsAST`) and **not** an application's REST/GraphQL resource path, which authorizes through `checkPermission` — so a scoped token still carries the role's full table CRUD there. It bounds what CI can administer, not what data it can reach, and the policy's user still has to be a least-privilege role. - A runtime-registered operation (`server.registerOperation`) is not recognized, so a policy naming one is rejected. Fails closed, and shared with add_role. Also that a scoped token cannot trade itself for a login session. **`job_workflow_ref` no longer gates the ref**, and the removal is deliberate: for a reusable workflow it names the workflow that ran, not the caller, so its `@ref` is constant however it is invoked and would admit any caller branch. It still pins the workflow. The table said otherwise, which would have produced policies an admin believed were ref-gated. Six claims narrowed to what the code actually guarantees — the same guidance-versus-enforcement distinction applied earlier to the SSH-key text: - Audience: Harper rejects GitHub's shared default shape, but does not compare the value against its own identity, so a typo is accepted at write time and fails at exchange time instead. Not checked at all for a generic issuer. - The one-hour token is a reduced exposure window, not "worthless by the time it reaches a log" — inside the hour it is a live credential. - `drop_oidc_trust` stops future exchanges; it cannot revoke an already-issued stateless JWT. Names the user/role action that actually contains a leak. - A rejected OIDC exchange does not reliably 401: the CLI continues down precedence and a payload `username=`/`password=` pair will authenticate, so a policy mismatch can look like a working deploy under the wrong identity. - A pinned SHA fixes the source revision, not the built artifact — the cluster builds from source per node. - `credential=true` is the git-by-reference path; an npm-registry credential is attached with the printed entry naming `registry`. Release notes gain the replay race (records replicate asynchronously, so simultaneous replays on different nodes can both land) and that auth-event auditing is off by default. Ref resolution in `applications.md` also corrected here rather than only in #617, since #599 would otherwise publish the local-only description on its own. --- reference/cli/authentication.md | 4 ++- reference/components/applications.md | 6 ++-- reference/operations-api/operations.md | 48 +++++++++++++++++++++++--- release-notes/v5-lincoln/5.3.md | 6 ++-- 4 files changed, 54 insertions(+), 10 deletions(-) diff --git a/reference/cli/authentication.md b/reference/cli/authentication.md index ba790cd21..5741e18bb 100644 --- a/reference/cli/authentication.md +++ b/reference/cli/authentication.md @@ -253,7 +253,9 @@ Requesting a GitHub Actions identity token for https://my-instance.harperdb.io:9 Authenticated as 'ci-deploy' via OIDC trust policy 'my-app-prod'. ``` -If Harper rejects the token it says so and continues unauthenticated, which surfaces as a 401 on the operation itself. The server deliberately does not report which check failed — see [`exchange_oidc_token`](../operations-api/operations.md#exchange_oidc_token) — so diagnose with `list_oidc_trust` and the instance's `oidc-trust` log. +If Harper rejects the token, the CLI reports that and carries on down the precedence list rather than aborting. Usually nothing is left, so the operation returns 401 — but if the command also passes `username=` and `password=`, the legacy fallback applies them and the operation **succeeds as that user instead**. A policy mismatch can therefore look like a working deploy under the wrong identity, so don't leave a payload credential pair on a command you expect the exchange to authenticate. + +The server deliberately does not report which check failed — see [`exchange_oidc_token`](../operations-api/operations.md#exchange_oidc_token) — so diagnose with `list_oidc_trust` and the instance's `oidc-trust` log. #### Method 3: Dedicated Authentication Parameters diff --git a/reference/components/applications.md b/reference/components/applications.md index e24716643..b044b8d07 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -142,7 +142,9 @@ For SSH-based private repos, use the [Add SSH Key](#add_ssh_key) operation to re Omitting `package` uploads a snapshot of your working directory. The result is an anonymous artifact: nothing records _which_ commit it came from, so reproducing it later — or stepping back to a previous release — means finding those exact files again. -Deploying by **reference** sends a pinned git reference instead, and the cluster fetches that exact commit. Redeploying the same reference is an exact redeploy, and rolling back is deploying an older one. +Deploying by **reference** sends a pinned git reference instead, and the cluster fetches that exact commit. Redeploying the same reference deploys the same source revision, and rolling back is deploying an older one. + +A pinned SHA fixes the _source_, not the built artifact. The cluster installs and builds from that source on each node, so unpinned dependency ranges, a mutable registry artifact, install scripts, or a different toolchain can still produce different bytes — or a failure — from the same commit. Commit your lockfile if you need the build itself to be reproducible. `harper deploy by_ref=true` builds that reference from the local git repository, so you don't assemble the URL yourself: @@ -166,7 +168,7 @@ harper deploy ref=v1.2.0 restart=true replicated=true harper deploy ref=9f8c2a1 restart=true replicated=true ``` -**A reference is pinned to a SHA, not to the name you typed.** Tags and branches are resolved locally and the full commit SHA is what ships. This matters on a cluster: peers resolve the package independently, so a tag that moves mid-deploy — or a branch that advances — could otherwise leave nodes running different code. +**A reference is pinned to a SHA, not to the name you typed.** Tags and branches are resolved to a full commit SHA before the deploy is sent — from your local checkout when it has the ref, and from the remote when it doesn't, which is the usual case in a shallow CI clone. If neither can name a commit the deploy stops rather than sending the name for peers to resolve. This matters on a cluster: peers resolve the package independently, so a tag that moves mid-deploy — or a branch that advances — could otherwise leave nodes running different code. **Commit and push first.** The cluster clones from the remote, so it only sees commits that have been pushed. `by_ref` warns when the working tree is dirty, since uncommitted changes won't be part of the deploy. diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index a915b0a2f..42137c2a2 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -613,11 +613,40 @@ Creates or replaces a trust policy. **super_user only** — a policy lets an ext | `audience` | **Required.** The audience the token must be addressed to. Should identify **this instance**; enforced for GitHub Actions. | | `claims` | **Required.** The claim constraints a token must satisfy. At least one, and specific enough for the issuer's profile. | | `user` | **Required.** The Harper user a matching run authenticates as. Must already exist and be active. | +| `operations` | Narrow the minted token to these operations, 1–100 unique names. Omit for the user's full role — see below. | | `enabled` | Defaults to `true`. A disabled policy is kept but never matched. | | `description` | Optional free text, up to 1024 characters. | This **replaces** the policy rather than merging into it. A partial update is how an over-broad policy gets created by accident, and the point of `claims` is that every constraint in it was written deliberately. +##### Narrowing what the token may do (`operations`) + +The user the policy names is the privilege boundary: a matching run gets that user's role. `operations` narrows it further, so one CI user can back several policies that each do less than the role allows: + +```json +{ + "operation": "add_oidc_trust", + "id": "my-app-prod", + "issuer": "https://token.actions.githubusercontent.com", + "audience": "https://my-instance.harperdb.io:9925/", + "user": "ci-deploy", + "operations": ["deploy_component", "get_deployment", "restart_service"], + "claims": { "repository_id": "67890", "environment": "production" } +} +``` + +It is **narrowing only** — never widening. An operation the role forbids stays forbidden, so the scope cannot be used to grant something the user does not already have. Omit `operations` and the token carries the user's full role. + +Names are validated when the policy is written, against the same registry `add_role` and `alter_role` use, so a typo is rejected there rather than failing later inside CI with nothing to point at. One consequence: an operation a component registered at runtime with `server.registerOperation` is **not** recognized, because that registry is process-local, so a policy naming one is rejected. It fails closed — a rejected policy, never a widened one — and the same gap applies to `add_role` and `alter_role`. + +:::warning +**The scope covers the Operations API and SQL, not the application data path.** It is enforced at the `verifyPerms` / `verifyPermsAST` gate, so a scoped token still carries the role's full table-level CRUD when it reaches an application's REST or GraphQL resources, which authorize through `checkPermission` instead. + +So `operations` bounds what a CI credential can _administer_, not what data it can read or write. If that matters, point the policy's `user` at a role that is itself least-privilege for the data the token can reach, rather than relying on the scope alone. +::: + +A scoped token also cannot trade itself for a browser session: `create_authentication_tokens` with `purpose: "login"` is refused, because a session carries no operation scope and would silently restore the user's full role. + `user` is resolved at write time. A policy naming a user that does not exist, or one that is inactive, is rejected — otherwise it would fail only at exchange time, inside CI, with nothing to point at. If the named user is a **super_user**, the policy is still created but the response carries a `warning`: any run matching it gains full administrative access. **Claim constraints** are exact string matches. A value may be a string, or an array of strings meaning any-of: @@ -628,7 +657,9 @@ This **replaces** the policy rather than merging into it. A partial update is ho A constrained claim that is **absent** from the token fails rather than passes, so a policy cannot be weakened by an issuer that stops emitting a claim. -**The audience should be instance-specific.** For GitHub Actions this is enforced: `https://github.com/` is rejected, because that is the provider's default, shared by every repository under the owner, so accepting it would make a token minted by any of them valid here — the exact thing the audience field exists to prevent. For an issuer with no registered profile the audience is not checked for specificity, and the required `sub` pin is what binds the policy to one principal instead. +**The audience should identify this instance.** For GitHub Actions, Harper rejects the provider's shared default — anything shaped like `https://github.com/` — because that value is shared by every repository under the owner, so accepting it would make a token minted by any of them valid here. + +That check is a guard against the one known-dangerous value, not a proof of correctness: Harper does not compare the audience against its own identity, so an arbitrary or mistyped value is accepted at write time and instead fails to match at exchange time, when the CLI derives the audience from its target URL. Use the instance URL your CI targets. For an issuer with no registered profile the audience is not checked for specificity at all, and the required `sub` pin is what binds the policy to one principal. ##### Policy specificity for GitHub Actions @@ -638,7 +669,7 @@ For `https://token.actions.githubusercontent.com`, a policy must satisfy all thr | ---------------------- | ------------------------------------------------------------------------ | ----------------------------------------------- | | **Pin the repository** | `repository_id`, `repository` | Any repository | | **Pin the workflow** | `workflow_ref`, `workflow_path`, `job_workflow_ref`, `job_workflow_path` | Any workflow in that repository | -| **Gate the ref** | `workflow_ref`, `job_workflow_ref`, `ref`, `environment` | Any branch that can be pushed to the repository | +| **Gate the ref** | `workflow_ref`, `ref`, `environment` | Any branch that can be pushed to the repository | The ref gate is the one worth understanding, and it is stricter than npm's model. Pinning repository and workflow without also pinning a ref is not safe: anyone who can push a branch can add the trusted workflow to that branch and mint a token. npm accepts that shape and relies on environment protection instead. @@ -648,6 +679,7 @@ Consequences worth planning around: - **A tag-triggered release cannot pin `workflow_ref`**, since the tag is unknown when the policy is written. Pin `workflow_path` instead — Harper derives it from `workflow_ref` by removing the ref — and gate on `environment`. - **`ref_type: tag` is deliberately not accepted as a ref gate.** Anyone with push access can create a tag. - **`sub` is not accepted as a pin.** It varies by trigger, and its format changed for repositories created after 2026-07-15 (immutable subjects embed owner and repository ids), so a policy pinning it would have to handle two shapes indefinitely. +- **`job_workflow_ref` pins the workflow but does not gate the ref.** In a reusable workflow it names the reusable workflow that ran, not the caller that invoked it, and its `@ref` suffix is that workflow's own branch — constant however it is called. Accepting it as a ref gate would admit any branch of any repository that references the reusable workflow. Pin the workflow with it if you like, then gate the ref with `workflow_ref`, `ref`, or `environment`. - **`pull_request_target` runs are denied** unless the policy explicitly constrains `event_name`. Such a run executes the base repository's workflow, with its secrets, while a fork controls the checked-out code. A plain `pull_request` run from a fork cannot mint at all, since it gets no `id-token: write`. ##### Other issuers @@ -678,7 +710,7 @@ Response: } ``` -The operation token is valid for **one hour** — long enough to cover a slow deploy, short enough to be worthless by the time it reaches a log. No refresh token is issued; a subsequent run performs a new exchange. +The operation token is valid for **one hour** — long enough to cover a slow deploy, short enough to bound the exposure if it leaks. That is a reduced window, not safety: within the hour it is a live credential carrying the policy's identity, so treat it like any other secret and keep it out of logs and step outputs. No refresh token is issued; a subsequent run performs a new exchange. :::note **Every rejection returns the same message.** The endpoint is unauthenticated, so a caller told which check failed could enumerate a policy one claim at a time. The specific reason is written to the `oidc-trust` logger, which is where to look when a workflow that should match does not. @@ -698,16 +730,22 @@ Lists every policy, **including disabled ones**, sorted by `id`. **super_user on { "operation": "list_oidc_trust" } ``` -Returns `{ "policies": [ ... ] }`. Each entry carries `id`, `issuer`, `audience`, `claims`, `user`, `enabled`, `description`, `updated_by`, and timestamps. +Returns `{ "policies": [ ... ] }`. Each entry carries `id`, `issuer`, `audience`, `claims`, `user`, `operations` (`null` when unscoped), `enabled`, `description`, `updated_by`, and timestamps. #### `drop_oidc_trust` -Deletes a policy, revoking every workflow that matched it. **super_user only.** Fails with `404` if no policy has that `id`. +Stops every workflow that matched the policy from exchanging again. **super_user only.** Fails with `404` if no policy has that `id`. ```json { "operation": "drop_oidc_trust", "id": "my-app-prod" } ``` +:::caution +**This does not revoke operation tokens already issued.** The minted token is a stateless JWT valid until its one-hour expiry, so a token obtained moments before the policy was dropped keeps authorizing for the rest of that hour. + +Dropping the policy is therefore containment against _future_ runs. If you are responding to a suspected compromise rather than doing routine cleanup, also deactivate or re-role the user the policy named (`alter_user`), which is what stops a token that is already in someone's hands. +::: + --- ## Components diff --git a/release-notes/v5-lincoln/5.3.md b/release-notes/v5-lincoln/5.3.md index 805292d19..16309c865 100644 --- a/release-notes/v5-lincoln/5.3.md +++ b/release-notes/v5-lincoln/5.3.md @@ -36,7 +36,7 @@ Tags and branches are resolved to a full commit SHA before the deploy is sent ### Sealed Deploy Credentials -`harper deploy setup=true` provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an `enc:v1:` envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. Later deploys pass `credential=true` and the host is derived from the package being deployed, so the credential always matches the clone it authenticates. See [Provisioning a Deploy Credential](/reference/v5/components/applications#provisioning-a-deploy-credential). +`harper deploy setup=true` provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an `enc:v1:` envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. It handles both private-source kinds — a git host and a private npm registry — and prints the `credentials` entry the deploy should use. For a git-by-reference deploy that entry is attached by `harper deploy credential=true`, which derives the host from the package so the credential always matches the clone it authenticates; an npm-registry credential is attached by passing the printed entry, which names the `registry` rather than a host. See [Provisioning a Deploy Credential](/reference/v5/components/applications#provisioning-a-deploy-credential). ## Security @@ -46,7 +46,9 @@ A CI runner can now authenticate to Harper with no stored credential. It present Policies are managed with `add_oidc_trust`, `list_oidc_trust`, and `drop_oidc_trust` (all super_user), and live in the replicated `system.hdb_oidc_trust` table. A policy pins the issuer, an instance-specific audience, and a set of claim constraints. For GitHub Actions those constraints must pin the repository, pin the workflow, and gate the ref — a policy naming only a repository and workflow is refused, because anyone who can push a branch could otherwise add the trusted workflow to it and mint a token. Issuers with no registered profile must pin `sub`, which makes Kubernetes service accounts, GCP service accounts, and SPIFFE identities work with no provider-specific configuration. -Identity tokens are single-use, exchanges are recorded in the authentication audit stream, and every rejection returns the same message with the specific reason written to the `oidc-trust` log. In the CLI the exchange ranks below every configured credential, so enabling it does not silently re-point a pipeline that still has a token secret set. See [OIDC Trusted Publishing](/reference/v5/operations-api/operations#oidc-trusted-publishing) and [Workload identity](/reference/v5/cli/authentication#workload-identity-oidc). +A policy can also carry an `operations` list, narrowing the minted token to a subset of its user's role on the Operations API and SQL paths — narrowing only, and not covering an application's REST/GraphQL data path, so the policy's user should still be a least-privilege role for the data it can reach. + +Identity tokens are single-use, though the replay record replicates asynchronously, so two simultaneous replays against different nodes can both succeed. Exchanges are recorded in the authentication audit stream when `logging.auditAuthEvents.logSuccessful` / `logFailed` is enabled — they are off by default, so turn them on before you need the trail. Every rejection returns the same message, with the specific reason written to the `oidc-trust` log. In the CLI the exchange ranks below every configured credential, so enabling it does not silently re-point a pipeline that still has a token secret set. See [OIDC Trusted Publishing](/reference/v5/operations-api/operations#oidc-trusted-publishing) and [Workload identity](/reference/v5/cli/authentication#workload-identity-oidc). ### Server-Side SSH Key Generation From e44cf2845a395db253af205c05287f6956d7f824 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 18 Aug 2026 12:47:59 -0400 Subject: [PATCH 17/17] docs: correct the activate-existing rebuild path and four narrower claims MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 3 of the cross-model review (codex at 5d4ce0ca) re-raised as major something I had rejected as a false positive twice. It was right and I was wrong. **A `deployment_id` activate can re-fetch and re-install.** I checked `activateStagedApplication`, saw it throw on a missing staged tree, and concluded no rebuild path existed. The recovery is at the call site, not in the callee: `deployComponentActivateExisting` tests `hasCompleteStagedApplication` and, when it fails, calls `sourceStagedPayload` and `stageApplication` before activating. So "no new payload is fetched or installed" is only true while every staged tree is intact — a restart or disk repair between staging and activation makes that node rebuild, which is slower and can fail outright if the source or credential is no longer reachable. Documented in both operations.md and the release note. The lesson is narrow and worth stating: verifying a negative claim by reading the function that would perform the action is not enough when a caller can do it first. **Rollback retention is conditional.** `activateStagedApplication` only records a revertable copy when the caller passes `activationSpec` — its own docblock says omitting it leaves the component "not revertable afterwards". The release note claimed every two-phase activation retains its predecessor. Three narrower corrections: - `deploy setup=true` supports exactly two providers, `github` and `npm`, and rejects anything else. The release note implied any git host. - That flow calls `get_secrets_public_key` and `set_secret`, both super_user, so it needs an administrative credential rather than the CI identity it is provisioning. - The SHA-pins-source-not-artifact correction reached applications.md last round but not the release note, which still said "an exact redeploy". Also drops the inline `` from the precedence list item added last round. CONTRIBUTING.md reserves the component for standalone placement after a heading, and I had cited that same rule while declining a bot's inline-badge suggestion on #617 — then did it myself. Fourteen further findings from this round are unaddressed and unverified; they are in the PR description rather than silently dropped. --- reference/cli/authentication.md | 2 +- reference/components/applications.md | 2 +- reference/operations-api/operations.md | 2 +- release-notes/v5-lincoln/5.3.md | 10 +++++----- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/reference/cli/authentication.md b/reference/cli/authentication.md index 5741e18bb..951bc032e 100644 --- a/reference/cli/authentication.md +++ b/reference/cli/authentication.md @@ -44,7 +44,7 @@ For remote Operations API commands, the CLI uses the first complete authenticati 4. Legacy `CLI_TARGET_USERNAME` and `CLI_TARGET_PASSWORD` environment variables 5. `HARPER_CLI_OPERATION_TOKEN` and `HARPER_CLI_REFRESH_TOKEN` environment variables, or their legacy `CLI_TARGET_` equivalents — see [Token credentials for CI/CD](#token-credentials-for-cicd) 6. A token saved by `harper login` -7. A [workload identity token](#workload-identity-oidc) exchanged with the runtime's OIDC provider +7. A [workload identity token](#workload-identity-oidc) exchanged with the runtime's OIDC provider (v5.3.0) 8. `username=` and `password=` operation parameters (legacy fallback) Credentials are resolved as a pair and are never combined across sources. An incomplete pair supplied with dedicated authentication parameters or in the target URL causes the command to fail. An incomplete environment-variable pair is skipped with a warning so that a saved login token can still be used. diff --git a/reference/components/applications.md b/reference/components/applications.md index b044b8d07..59da64dcc 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -190,7 +190,7 @@ Deploying by reference means the **cluster** installs and builds the component f -`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source: +`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source. It calls `get_secrets_public_key` and `set_secret`, both of which require **super_user**, so run it with an administrative credential rather than the CI identity it provisions for: ```sh harper deploy setup=true diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 42137c2a2..c35fc2e54 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -807,7 +807,7 @@ Additional parameters: - `host` — the virtual hostname the component is served on (e.g. `"api.example.com"`). Must be a bare hostname or IPv6 literal — no scheme, port, path, or brackets. Persisted alongside `urlPath`. - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) - `activate` — set to `false` to **stage only** and stop before go-live. The build is prepared and verified on every node and the response returns a `deployment_id` in a `staged` state; nothing goes live. Activate it later by calling `deploy_component` again with that `deployment_id` (see below). Useful for pre-staging a release and flipping it live in a separate, fast step. -- `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). No new payload is fetched or installed; the already-staged build is swapped live cluster-wide. `project` is still required. For a `package` deploy you do not need to repeat `package` here: the identifier and credential references recorded when it was staged are recovered from the deployment and persisted to root config at activation, on every node — so a later restart or a newly joined peer reinstalls the version you activated. +- `deployment_id` — activate a previously-staged deployment (from an `activate: false` call). Normally nothing is re-fetched or re-installed: the build staged earlier is swapped live cluster-wide. If a node's staged tree is missing or incomplete by then — a restart or disk repair between staging and activation — that node re-sources the payload and rebuilds before swapping, so activation can take noticeably longer there and can fail on a package whose source or credential is no longer reachable. `project` is still required. For a `package` deploy you do not need to repeat `package` here: the identifier and credential references recorded when it was staged are recovered from the deployment and persisted to root config at activation, on every node — so a later restart or a newly joined peer reinstalls the version you activated. - `ignore_replication_errors` — treat replication/peer failures as non-fatal (best-effort deploy to a partially-available cluster). This also opts out of the stage barrier. Applies to both a full deploy and a `deployment_id` activate. - `deployment_timeout` — per-deploy budget (ms) for peers to receive the replicated deployment row; defaults to 120000. - `two_phase` — set to `false` to force the legacy single-phase (in-place) deploy instead of stage-then-activate. diff --git a/release-notes/v5-lincoln/5.3.md b/release-notes/v5-lincoln/5.3.md index 16309c865..ed76597e6 100644 --- a/release-notes/v5-lincoln/5.3.md +++ b/release-notes/v5-lincoln/5.3.md @@ -20,23 +20,23 @@ Two-phase deploy requires operation replication and replication of the `system` ### Stage Now, Activate Later -`deploy_component` accepts `activate: false` to prepare and verify a release on every node without going live, returning a `staged` deployment ID. A later call passing that `deployment_id` swaps it in cluster-wide with no re-fetch and no re-install, turning go-live into a fast, separate step. Harper retains the most recent staged builds per component (default 5, `deployment_stagingRetention_maxCount`); a `deployment_id` that has aged out fails rather than silently rebuilding. `harper stage` and `harper activate` are the CLI verbs. See [Deploy modes](/reference/v5/operations-api/operations#deploy-modes). +`deploy_component` accepts `activate: false` to prepare and verify a release on every node without going live, returning a `staged` deployment ID. A later call passing that `deployment_id` swaps it in cluster-wide, turning go-live into a fast, separate step — normally with no re-fetch and no re-install, though a node whose staged tree went missing in between rebuilds it first. Harper retains the most recent staged builds per component (default 5, `deployment_stagingRetention_maxCount`); a `deployment_id` that has aged out fails rather than silently rebuilding. `harper stage` and `harper activate` are the CLI verbs. See [Deploy modes](/reference/v5/operations-api/operations#deploy-modes). ### Component Rollback -The new `revert_component` operation puts a component's retained previous version back in service across the cluster. Every two-phase activation retains the version it replaced, so a rollback resolves no package, decrypts no secret, downloads no artifact and runs no install — every node already has the bytes, and the swap is one atomic directory rename per node. It rewrites the stored `package:` reference and the boot-time application lock in the same operation, so a newly joined peer or a rebuilt components directory installs the version the cluster is actually running. +The new `revert_component` operation puts a component's retained previous version back in service across the cluster. A two-phase activation retains the version it displaced, so a rollback normally resolves no package, decrypts no secret, downloads no artifact and runs no install — every node already has the bytes. Retention accompanies an activation that records its activation spec, which is the normal deploy path; a one-shot deploy retains nothing, so confirm the deploy took the two-phase path rather than assuming a rollback target exists. It rewrites the stored `package:` reference and the boot-time application lock in the same operation, so a newly joined peer or a rebuilt components directory installs the version the cluster is actually running. `to_deployment_id` is required and names the deployment you expect to be live afterwards, which makes the operation idempotent: a client that loses the response and retries cannot flip a rejected release back in. Only the immediately previous version is retained, and only a two-phase activation creates that retained copy. `harper revert` is the CLI verb. See [`revert_component`](/reference/v5/operations-api/operations#revert_component). ### Deploying by Git Reference -`harper deploy by_ref=true` builds a package reference from the local git repository, so a deploy records which commit it came from instead of shipping an anonymous snapshot of the working directory. Redeploying the same reference is an exact redeploy, and rolling back is deploying an older one. +`harper deploy by_ref=true` builds a package reference from the local git repository, so a deploy records which commit it came from instead of shipping an anonymous snapshot of the working directory. Redeploying the same reference deploys the same source revision, and rolling back is deploying an older one. -Tags and branches are resolved to a full commit SHA before the deploy is sent — from the local checkout when it has the ref, and from the remote when it doesn't, as a shallow CI clone usually doesn't. A ref that can't be resolved either way stops the deploy rather than being sent by name, because peers resolve the package independently and a name that moves mid-deploy would leave nodes on different code. The CLI warns when the working tree is dirty and when the commit isn't on any remote branch. Under GitHub Actions on a `pull_request` run it deploys the pull request's head commit, not the merge commit the runner checks out. See [Deploying by Reference](/reference/v5/components/applications#deploying-by-reference). +A pinned SHA fixes the source revision rather than the built artifact, since the cluster still builds from source on each node. Tags and branches are resolved to a full commit SHA before the deploy is sent — from the local checkout when it has the ref, and from the remote when it doesn't, as a shallow CI clone usually doesn't. A ref that can't be resolved either way stops the deploy rather than being sent by name, because peers resolve the package independently and a name that moves mid-deploy would leave nodes on different code. The CLI warns when the working tree is dirty and when the commit isn't on any remote branch. Under GitHub Actions on a `pull_request` run it deploys the pull request's head commit, not the merge commit the runner checks out. See [Deploying by Reference](/reference/v5/components/applications#deploying-by-reference). ### Sealed Deploy Credentials -`harper deploy setup=true` provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an `enc:v1:` envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. It handles both private-source kinds — a git host and a private npm registry — and prints the `credentials` entry the deploy should use. For a git-by-reference deploy that entry is attached by `harper deploy credential=true`, which derives the host from the package so the credential always matches the clone it authenticates; an npm-registry credential is attached by passing the printed entry, which names the `registry` rather than a host. See [Provisioning a Deploy Credential](/reference/v5/components/applications#provisioning-a-deploy-credential). +`harper deploy setup=true` provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an `enc:v1:` envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. It handles the two private-source kinds it supports — a private GitHub repository and a private npm registry — and prints the `credentials` entry the deploy should use. For a git-by-reference deploy that entry is attached by `harper deploy credential=true`, which derives the host from the package so the credential always matches the clone it authenticates; an npm-registry credential is attached by passing the printed entry, which names the `registry` rather than a host. See [Provisioning a Deploy Credential](/reference/v5/components/applications#provisioning-a-deploy-credential). ## Security