From cdb4805107751455dc186fd47338a9bad5a9ec1e Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 14:35:24 -0600 Subject: [PATCH 01/30] chore(porch): 13 init pir --- .../status.yaml | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml new file mode 100644 index 000000000..6f50facb9 --- /dev/null +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -0,0 +1,18 @@ +id: '13' +title: add-ci-concepts-to-the-forge-l +protocol: pir +phase: plan +plan_phases: [] +current_plan_phase: null +gates: + plan-approval: + status: pending + dev-approval: + status: pending + pr: + status: pending +iteration: 1 +build_complete: false +history: [] +started_at: '2026-08-21T20:35:23.798Z' +updated_at: '2026-08-21T20:35:23.799Z' From ca1a7f6a3aafc0a417b341c57670881c7c7937dd Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 14:58:00 -0600 Subject: [PATCH 02/30] [PIR #13] Plan draft: CI concepts for the forge layer Co-Authored-By: Claude Opus 5 --- codev/plans/13-ci-forge-concepts.md | 294 ++++++++++++++++++++++++++++ codev/state/pir-13_thread.md | 35 ++++ 2 files changed, 329 insertions(+) create mode 100644 codev/plans/13-ci-forge-concepts.md create mode 100644 codev/state/pir-13_thread.md diff --git a/codev/plans/13-ci-forge-concepts.md b/codev/plans/13-ci-forge-concepts.md new file mode 100644 index 000000000..1f56d88e8 --- /dev/null +++ b/codev/plans/13-ci-forge-concepts.md @@ -0,0 +1,294 @@ +# PIR Plan: CI concepts for the forge layer — `ci-runs`, `ci-run-view`, `ci-failures`, `ci-run-log` + +Issue: #13 · Branch: `builder/pir-13` · Verification targets: `pseudoseed/codev` on GitHub (`gh` 2.87.0), `~/dev/entriq` (live Forgejo **15.0.2**, `tea` 0.14.2), and `codeberg.org/forgejo/forgejo` (live Forgejo **16.0.0-dev**, public, unauthenticated). + +## Understanding + +A builder asking "why did CI fail" currently shells out and pastes log text into its context. `KNOWN_CONCEPTS` (`packages/codev/src/lib/forge.ts:65-70`) has 18 concepts and none of them touch workflow runs. The issue and its two comments ask for four concepts, tiered so that only one of them ever fetches a log: + +| Question | Concept | Fetches a log? | +|---|---|---| +| Did my push pass? | `ci-runs` | No | +| Is it still running / which job is pending? | `ci-runs`, `ci-run-view` | No | +| It failed — why? | `ci-failures` | Yes, one job | +| Is this mine or pre-existing? | `ci-runs` (same workflow, other commits) | No | +| Extraction gave up — show me the log | `ci-run-log` (deliberate, separate) | Yes, windowed | + +I spent the investigation measuring the two providers rather than reasoning about them, because the issue's premises about both turned out to be **wrong in ways that change the design**. + +--- + +## What is actually true, measured 2026-08-21 + +### 1. `gh run view --log-failed` does NOT narrow to the failing step + +The issue says GitHub "already does the extraction for you." It does not, at least not here. Against a real failing run in this repository: + +``` +gh run view 32515040122 --log-failed → 2528 lines, 293 KB, 2.0 s +``` + +Every one of those 2528 lines is labelled `UNKNOWN STEP`: + +``` +Unit Tests UNKNOWN STEP 2026-08-21T18:47:09.5820646Z Current runner version: '2.336.0' +``` + +`--log-failed` selected the failing **job** and returned all of it. `gh` maps log files to steps by name and falls back to `UNKNOWN STEP` when that mapping fails, which is what happened. **293 KB is precisely the context bomb this issue exists to prevent**, so codev must do its own extraction on GitHub too — the same extractor both providers use. What `--log-failed` still buys is job selection: it is one call, and it never returns a passing job's log. + +### 2. The naive extraction heuristics both pick the wrong line on that same log + +The real failure is at line 2471 of 2528: + +``` +2471: FAIL src/commands/consult/__tests__/agy-auth-cache.test.ts > gemini lane burst behaviour (#1077 regression) +2472: AssertionError: expected null to be 'unauth' // Object.is equality +2491: Test Files 1 failed | 276 passed +2497: ##[error]AssertionError: expected null to be 'unauth' // Object.is equality +``` + +- **"First line matching an error pattern"** returns line 1257: `[artifact-canvas] Error: host blew up` — a fixture string printed by a **passing** test. A builder handed that would go debugging a host crash that never happened. +- **"Last N lines"** happens to work on this log and would not on the next one; the run's tail here is git-submodule cleanup and a Node deprecation warning. +- **`Test Files …`** appears **five** times, the first four all `passed` (lines 296, 317, 1450, 1477). Matching the first is wrong; the summary that matters is the one containing `failed`. +- Every payload line is wrapped in ANSI SGR codes — the raw bytes are `\e[41m\e[1m FAIL \e[22m\e[49m src/…`. **A matcher that does not strip ANSI first matches nothing at all.** This is the single most likely way for an extractor to silently return `extracted: false` on a log that plainly contains the answer. +- GitHub emits `##[error]` at the true failure (2 occurrences, both real). On GitHub that marker is the highest-precision signal available and costs nothing. + +### 3. Forgejo 15.0.2 has **no Actions log API at all**, and `tea actions` is broken against it + +`tea actions runs list --status failure --output json` works but is **lossy** — `workflow`, `branch`, `started` and `duration` all come back empty strings: + +```json +{ "id": "11130", "status": "failure", "workflow": "", "branch": "", "event": "pull_request", "started": "", "duration": "" } +``` + +The other two subcommands from the issue do not work at all against this server: + +``` +tea actions runs view 11130 → Error: failed to get jobs: unknown API error: 404 + GET /api/v1/repos/pseudoseed/entriq/actions/runs/11130/jobs +tea actions runs logs 6881 --job 40084 + → Error: failed to get logs for job 40084: unknown API error: 404 + GET /api/v1/repos/pseudoseed/entriq/actions/jobs/40084/logs +``` + +Both routes 404 because **they were added in Forgejo v16.0** (released 2026-07-16); `git.pseudoseed.com` reports `15.0.2+gitea-1.22.0`. I probed every plausible alternative on 15.0.2 — `actions/runs/{id}/jobs`, `actions/runs/{n}/jobs/{j}/logs`, `actions/tasks/{id}`, `actions/tasks/{id}/logs`, `actions/jobs/{id}`, `actions/artifacts`, `actions/workflows` — all 404. The web UI's own log route (`/{owner}/{repo}/actions/runs/{n}/jobs/{j}/logs`) exists but is **session-only**: it rejects both `Authorization: token` and HTTP basic auth with the API token (the API accepts the same token fine — verified as a control). There is no token-reachable path to a log on 15.0.2. + +So on this fork's own reference Forgejo, `ci-failures` and `ci-run-log` **cannot work**. That is a fact about the server, not a gap in the implementation, and the only correct response is to say so by name — see "the version gate" below. + +### 4. What Forgejo 15.0.2 *does* have, and what it costs + +| Endpoint | What it returns | Measured | +|---|---|---| +| `actions/runs?page=1&limit=N` | runs, each embedding a **full repository object** | 0.30 s / 3.8 KB at N=1; 0.33 s / **892 KB** at N=50 | +| `actions/runs/{id}` | one run | 0.3 s | +| `actions/tasks?page=1&limit=N` | **jobs**, GitHub-shaped (`id`, `name`, `head_branch`, `head_sha`, `run_number`, `status`, `workflow_id`, `url`) | 0.22 s / 507 B at N=1; 0.30 s / 24 KB at N=50 | + +Unlike `/pulls` in #12, these are priced **per request, not per item** — 50 runs cost the same 0.3 s as one. The cost is bytes, not seconds: `actions/runs` is 17.8 KB per run because of the embedded repo object, so it must be reduced with `jq` immediately. `actions/tasks` is 482 B per job and is the better list to build on. + +Three query-parameter facts, all measured, all footguns: + +- **`limit` is ignored unless `page` is also present.** `actions/runs?limit=3` returned **all 6922 runs**; `actions/runs?page=1&limit=3` returned 3. That is a #12-class hazard sitting in a default. +- **`status=` is honoured server-side** (`status=failure` → `total_count` 1541 of 6922). +- **`branch=` and `event=` are silently ignored** on both endpoints. Filtering by branch must happen client-side. +- `actions/tasks` caps at 50 per page regardless of `limit`. + +And the branch itself is not what you would expect: for `pull_request` runs, Forgejo reports `head_branch` / `prettyref` as **`#3847`** — the PR number, not the branch name. Only `push`/`schedule` runs carry a real branch. In the first 100 tasks on entriq: `#3869`×32, `#3865`×10, `main`×7, `v1.0.230`×1. So `CODEV_BRANCH_NAME=builder/pir-13` matches **nothing** on a repo that runs CI on pull requests unless the branch is first resolved to its PR number. + +Finally, Forgejo has **two id spaces** and both are valid inputs to the same route: run `id` 11130 has `index_in_repo` 6881, and `actions/runs/6881` resolves to a *different* run (a real one, whose own index is 4258). The web URL shows the index. A concept that guessed would confidently answer about the wrong run. + +### 5. Forgejo 16 does have it — verified live, unauthenticated + +`codeberg.org` runs `16.0.0-dev-694` and `forgejo/forgejo` is public with real Forgejo Actions history, so the v16 code path can be verified for free: + +| Call | Result | +|---|---| +| `GET repos/forgejo/forgejo/actions/runs/6554924/jobs` | JSON array of jobs: `id`, `task_id`, `name`, `status`, `needs`, `runs_on` | +| `GET repos/forgejo/forgejo/actions/jobs/11952749/logs` | **200**, `text/plain`, 142 KB, 1599 lines, 1.02 s, `accept-ranges: bytes` | +| the same with `?step=3` | **byte-identical** — `?step=` is not honoured on this build | +| `GET repos/forgejo/forgejo/actions/runs/6554924/logs` | 200, `application/zip`, 522 KB, every job's log | + +Two things to carry into the design: the log endpoint takes the **job `id`** (11952749), *not* the `task_id` (8848703) that `actions/tasks` lists — passing the task id returns `{"message":"resource does not exist"}`, which is exactly the sort of 404 that reads as "no logs" if it is not distinguished. And `accept-ranges: bytes` means a tail can be fetched as a byte range instead of downloading 142 KB to print 50 lines. + +--- + +## Proposed Change + +Four concepts, one shared extraction contract, and a hard rule that **every response is a JSON envelope that says how much of the truth it contains**. + +### The envelope + +Every ci-* concept prints one JSON object on stdout, on success *and* on failure, so a caller always gets something structured and never has to interpret an empty string: + +```jsonc +// ci-failures, extraction succeeded +{ + "ok": true, "provider": "github", "runId": 32515040122, + "failures": [{ + "jobId": 96874679182, "jobName": "Unit Tests", + "stepName": "Run unit tests with coverage", "stepNumber": 15, + "matchedBy": "vitest", // which rung of the ladder fired + "text": "FAIL src/…/agy-auth-cache.test.ts > gemini lane burst…\nAssertionError: expected null to be 'unauth'…", + "logLines": 2528, "returnedLines": 24, "truncated": false + }], + "jobsFailed": 1, "truncated": false +} + +// ci-failures, extraction gave up — a HANDOFF, not a dead end (issue comment 2) +{ + "ok": true, "provider": "gitea", "runId": 11130, + "extracted": false, "reason": "no recognized failure pattern", + "failures": [{ "jobId": 11952749, "jobName": "test-unit", "logLines": 1599 }], + "next": "ci-run-log CODEV_CI_RUN_ID=11130 CODEV_CI_JOB_ID=11952749 CODEV_CI_LOG_TAIL=80" +} + +// any concept, transport failure — a timeout is NEVER an empty result +{ "ok": false, "error": "timeout", "seconds": 60, + "detail": "GET repos/pseudoseed/entriq/actions/tasks did not return within 60s", + "remedy": "raise CODEV_FORGE_TIMEOUT" } + +// gitea, server too old +{ "ok": false, "error": "unsupported-server", "serverVersion": "15.0.2+gitea-1.22.0", + "needs": ">=16.0", + "detail": "this Forgejo has no Actions job-log API (added in Forgejo 16.0); ci-runs and ci-run-view still work" } +``` + +`logLines` / `returnedLines` / `truncated` are on **every** response that returns log text, per the issue. Never a bare array, never a bare string. + +**Exit statuses** keep the #12 contract — `0` answered, `1` could not answer, `2` bad input — with one addition specific to these concepts: **the JSON envelope is printed on stdout even when the exit status is non-zero**, because "the concept failed" and "the concept failed *because the API timed out at 60 s*" must not be the same observation. stderr still carries the one-line human message. + +### `ci-runs` — the cheap question + +Inputs: `CODEV_BRANCH_NAME` (opt), `CODEV_CI_STATUS` (opt: `success|failure|pending|queued|in_progress|skipped|canceled`), `CODEV_CI_LIMIT` (opt, default 20), `CODEV_CI_WORKFLOW` (opt — the "is it mine or flaky" filter). + +Output: `{ ok, provider, runs: [{ id, number, name, workflow, status, conclusion, branch, sha, event, url, createdAt }], truncated }`. + +- **github**: one `gh run list --json …` call (measured 0.7 s), `--branch`/`--status`/`--workflow`/`--limit` passed through. +- **gitea**: `actions/runs?page=1&limit=N&status=…`, always with `page=1` (see the `limit` footgun), reduced by `jq` on the way out. `status` server-side; `branch` **client-side**, and when `CODEV_BRANCH_NAME` is set and the repo runs CI on pull requests, the branch is first resolved to its PR number with the base/head lookup #12 already built (`gitea_default_branch` + `pulls/{base}/{head}`, ~1 s), then matched against `#N` as well as the literal branch. `conclusion` is emitted as `null` — Forgejo has no such field; `status` carries `failure`/`success` — and this asymmetry is documented rather than faked. +- The client-side branch filter walks at most `CODEV_CI_MAX_PAGES` (default 4 → 200 runs) and sets `truncated: true` with a stderr line if it stops early. It never returns a short list that reads as a complete one. + +### `ci-run-view` — status per job, still no log + +Input: `CODEV_CI_RUN_ID` (required). Output: the run plus `jobs: [{ id, name, status, conclusion, startedAt, completedAt, failedSteps: [{ name, number, conclusion }] }]`. + +- **github**: `gh run view --json jobs,status,conclusion,headBranch,headSha,workflowName,url,displayTitle` — 1.3 s, and it already carries per-step conclusions, so `failedSteps` is a `jq` filter over data we already paid for. +- **gitea, Forgejo ≥16**: `actions/runs/{id}` + `actions/runs/{id}/jobs`. Job `id` (not `task_id`) is what the output carries, because that is what the log endpoint takes. +- **gitea, Forgejo 15**: `actions/runs/{id}` gives `index_in_repo`; jobs are recovered by scanning `actions/tasks` for `run_number == index_in_repo`, bounded to `CODEV_CI_MAX_PAGES`. The response says which route answered (`"jobSource": "runs/jobs" | "tasks-scan"`) so a reader knows whether it is seeing the server's own grouping or ours. Forgejo has no per-step data on either version, so `failedSteps` is `[]` there — stated in the output, not silently omitted. + +`CODEV_CI_RUN_ID` is always the **API id** from `ci-runs`'s `id` field, never the URL number. `ci-runs` emits both `id` and `number` and the docs say which to pass; the two-id-space ambiguity above is why the concept refuses to guess. + +### `ci-failures` — the one that matters + +Input: `CODEV_CI_RUN_ID` (required), `CODEV_CI_JOB_ID` (opt, to pin one job). + +1. Find the failing jobs (from `ci-run-view`'s data, one call). +2. Fetch **only the first failing job's** log — `gh run view --log-failed` on GitHub (job-scoped, one call), `actions/jobs/{id}/logs` on Forgejo ≥16. "Prefer the FIRST failing step in the first failing job" per the issue; later failures are usually downstream. Other failing jobs are *listed* (name + id) but not fetched, and the response says so. +3. Strip ANSI, strip the `job\tstep\t` prefix and the leading RFC3339 timestamp, then run the ladder. +4. Cap: **2 KB per failing step, 8 KB per response**, `truncated: true` when either bites. + +**The extraction ladder** (first rung that matches wins, and the response names the rung in `matchedBy`): + +| Rung | Matches | Returns | +|---|---|---| +| 1. `runner-marker` | `##[error]` (GitHub) | the marker line plus 3 lines of leading context | +| 2. `vitest` / `jest` | `FAIL > ` and/or `AssertionError:` / `expected … to …`, anchored by the `Test Files … failed` summary — **the one containing `failed`**, not the first | the `Failed Tests` block from the first `FAIL` to the summary | +| 3. `tsc` | `error TS####:` | the first such line plus following lines of the same diagnostic | +| 4. `first-error` | first `^Error:` / `error:` / `Exception` **after the last passing-suite boundary**, never the first in the file | the line plus 3 either side | +| 5. give up | — | `extracted: false` + `jobId`, `jobName`, `logLines`, and a ready-to-run `next` | + +Rung 4 is deliberately weaker than "first match in the file": on the real log above, the unqualified version returns a passing test's fixture string from line 1257. The rule is only allowed to fire after the last recognisable "N passed" boundary, and if there is no such boundary it falls to rung 5 rather than guessing. **Rung 5 never returns arbitrary lines.** A builder that receives 50 unexplained lines treats them as the diagnosis; one told extraction failed goes and looks, which is correct and cheaper. + +### `ci-run-log` — the deliberate escape hatch + +Input: `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (opt — defaults to the first failing job, else the only job), and **exactly one** window: `CODEV_CI_LOG_TAIL=N`, `CODEV_CI_LOG_HEAD=N`, or `CODEV_CI_LOG_GREP=` with `CODEV_CI_LOG_CONTEXT` (default 3). Zero windows or more than one is exit 2 with a named message — not a silent default, because a defaulted window is how this becomes "tail by habit," which the issue's second comment exists to prevent. + +Output: `{ ok, runId, jobId, jobName, logLines, returnedLines, from, to, truncated, lines: [...] }`. Same 8 KB response cap. `from`/`to` are 1-based line numbers into the full log, so a builder always knows where it is standing. + +It is a separate concept, not a flag on `ci-failures`, for exactly the reason the issue gives. + +### Caching + +A completed run's log is immutable. Both log-fetching concepts cache the **raw** log to `${TMPDIR}/codev-ci-logs///.log`, and read from it when present, **only when the job's status is terminal** (`success|failure|skipped|canceled`). An in-progress job is never cached and never read from cache. `CODEV_CI_NO_CACHE=1` bypasses; entries over `CODEV_CI_CACHE_MAX_MB` (default 32) are not written. This makes the realistic sequence — `ci-failures`, then `ci-run-log … TAIL`, then `ci-run-log … GREP` — cost exactly one download. + +### Timeouts, and making a timeout say "timeout" + +Three layers, because the current stack loses the distinction at every one of them: + +1. **Scripts.** gitea routes through `gitea_api` (#12's watchdog, `CODEV_FORGE_TIMEOUT`, default 60 s). GitHub has no equivalent today: `github/*.sh` call `gh` bare. I will extract #12's `gitea_timeout` into `scripts/forge/_timeout.sh`, source it from both providers, and wrap every `gh` call in the new scripts. Existing github scripts are left alone — retrofitting them is a separate change. +2. **Envelope.** Timeout ⇒ `{"ok":false,"error":"timeout","seconds":N,…}` on stdout, message on stderr, exit 1. +3. **Dispatcher.** `executeForgeCommand` returns `null` for every failure mode (`forge.ts:390-403`), so a timeout, a crash and unparseable output are one value. I will add `executeForgeCommandDetailed()` returning `{ ok, data, stdout, stderr, exitCode, timedOut, durationMs }` — non-destructive, existing callers untouched — and have it set `timedOut` from `err.killed`/`err.signal`, which #12 verified fires reliably at the Node level. Without this the shell layer can be as honest as it likes and the TS layer still flattens it to `null`. + +### gitlab and linear must be *disabled*, not absent + +`buildPresetFromScripts` only sets keys for concepts that have a script, and `getForgeCommand` falls back to the **github default** for anything unset (`forge.ts:280-296`). So adding four concepts without touching those presets would make a GitLab repo silently run `gh run list` — the exact silent-fallthrough class #1455 closed. All four go in the disabled list for `gitlab` and `linear`, so they resolve as `disabled` and `describeUnavailableConcept` names the provider. + +### Commits + +1. `_timeout.sh` extraction + `executeForgeCommandDetailed` + `KNOWN_CONCEPTS` + preset disables (github/gitea unblocked, gitlab/linear explicitly null). +2. GitHub scripts: `ci-runs`, `ci-run-view`, `ci-failures`, `ci-run-log` + the shared extractor. +3. Gitea scripts: the same four, with the version gate and the 15-vs-16 job-source fallback. +4. Tests, both `SKILL.md` twins, `arch.md` / `lessons-learned.md` routing. + +--- + +## Files to Change + +- `packages/codev/src/lib/forge.ts:65-70` — add `ci-runs`, `ci-run-view`, `ci-failures`, `ci-run-log` to `KNOWN_CONCEPTS`. +- `packages/codev/src/lib/forge.ts:126-135` — add all four to the disabled lists for `gitlab` and `linear`; leave `gitea` enabled. +- `packages/codev/src/lib/forge.ts:390-403` — new `executeForgeCommandDetailed()` beside `executeForgeCommand`. +- `packages/codev/src/lib/forge-contracts.ts` — `CiRunItem`, `CiRunViewResult`, `CiFailuresResult`, `CiRunLogResult`, and the shared `CiError`. +- `packages/codev/scripts/forge/_timeout.sh` — **new**, `gitea_timeout` lifted verbatim (it is provider-neutral) with its comments; `gitea/_lib.sh` sources it instead of defining it. +- `packages/codev/scripts/forge/_ci-extract.sh` — **new**, the ANSI-stripping extraction ladder, shared by both providers so they cannot drift. +- `packages/codev/scripts/forge/github/{ci-runs,ci-run-view,ci-failures,ci-run-log}.sh` — **new** (mode 755; `bugfix-693-forge-exec-bit.test.ts` pins the bit). +- `packages/codev/scripts/forge/gitea/{ci-runs,ci-run-view,ci-failures,ci-run-log}.sh` — **new**. +- `packages/codev/src/__tests__/pir-13-ci-concepts.test.ts` — **new**. +- `.claude/skills/forge/SKILL.md` and `.codex/skills/forge/SKILL.md` — the concept table, the env-var table, the envelope, the Forgejo-16 requirement. Byte-identical; the twin test from #12 covers it. +- `codev/resources/arch.md` (§ Integration Points → Forge Concept Commands) — the measured Forgejo facts. `codev/resources/lessons-learned.md` — the extraction lesson. Hot-tier promotion only if it displaces something; likely not. +- `codev/reviews/13-ci-forge-concepts.md` — at review time. + +No `codev-skeleton/` mirror: forge scripts and `SKILL.md` are single-source (established in #12). + +--- + +## Risks & Alternatives Considered + +- **Risk: `ci-failures` cannot work on the fork's own reference Forgejo.** entriq is 15.0.2 and the log API is 16.0+. Mitigation: the version gate returns `error: "unsupported-server"` naming both versions, `ci-runs`/`ci-run-view` keep working there, and the v16 path is verified against Codeberg. This is loud degradation, which is what the issue asks for from `gitlab`; it would be dishonest to hide it for gitea. **If `git.pseudoseed.com` is upgraded to Forgejo 16, the same code starts working with no change.** +- **Risk: the extractor is tuned to the two logs I have.** Mitigation: both are captured as fixtures (a 2528-line GitHub vitest failure and a 1599-line Forgejo `test-unit` failure), the ladder names its rung in `matchedBy`, and rung 5 is a first-class outcome rather than a fallback into guessing. When it is wrong it says so. +- **Risk: `##[error]` is GitHub-only.** Forgejo runners do not emit it. Rungs 2-4 carry Forgejo, which is why the ladder is shared rather than per-provider. +- **Alternative rejected: parse `gh run view --log-failed`'s step column.** It is `UNKNOWN STEP` on the very first run I tested. Step attribution comes from `--json jobs` instead, which is structured and reliable. +- **Alternative rejected: a `tail` flag on `ci-failures`.** The issue's second comment gives the reason and I agree with it: a flag gets passed by habit and every status question starts dragging a log. +- **Alternative rejected: implementing via `tea actions …` subcommands.** They are broken against 15.0.2 and lossy where they work. `tea api` is the load-bearing surface, as #12 established. +- **Alternative rejected: caching under `.codev/`.** Logs are large and disposable; `$TMPDIR` avoids polluting a repo and inherits OS cleanup. +- **Risk: response caps hide the answer.** Mitigation: `truncated` is always present, and `ci-run-log` exists precisely so the next call is targeted rather than blind. + +--- + +## Test Plan + +**Unit** (`pir-13-ci-concepts.test.ts`, stubbing `gh`/`tea` on `PATH`, the #12 pattern): + +1. Extraction against the **real captured** 2528-line GitHub log: returns the `agy-auth-cache` assertion, and specifically **not** `[artifact-canvas] Error: host blew up` (line 1257) and **not** the four passing `Test Files` summaries. +2. ANSI-wrapped `FAIL`/`AssertionError` still match (the raw fixture bytes carry the escapes). +3. A log with no recognisable failure ⇒ `extracted: false` with `jobId`/`jobName`/`logLines`/`next`, and **zero** log lines in the payload. +4. Caps: a >2 KB step extract sets `truncated: true`; `returnedLines < logLines`. +5. `ci-run-log`: zero windows ⇒ exit 2; two windows ⇒ exit 2; tail/head/grep return correct `from`/`to`; grep honours `CODEV_CI_LOG_CONTEXT`. +6. A stub that sleeps past `CODEV_FORGE_TIMEOUT` ⇒ `{"ok":false,"error":"timeout"}` on stdout, non-zero exit, and `executeForgeCommandDetailed().timedOut === true`. +7. gitea `ci-runs` sends `page=1` (the `limit`-ignored footgun) and never calls a bare `actions/runs?limit=`. +8. gitea `ci-runs` with `CODEV_BRANCH_NAME` matches a `#` `head_branch`, not just a literal branch. +9. `ci-failures` on a Forgejo-15 stub ⇒ `error: "unsupported-server"` with both versions named; `ci-run-view` on the same stub still answers via the tasks scan and reports `jobSource: "tasks-scan"`. +10. `resolveAllConcepts` reports all four as `disabled` for `gitlab` and `linear`, and as `preset` for `gitea`. +11. Exec bit on all eight new scripts; `SKILL.md` twins byte-identical. + +**Live, through the real dispatcher** (config load → preset → env → script → JSON parse), with timings reported in the review the way #12 did: + +- **GitHub / `pseudoseed/codev`**: `ci-runs` (all, by branch, `--status failure`), `ci-run-view` on run 32515040122, `ci-failures` on the same run (must return the assertion, not 293 KB), `ci-run-log` tail/head/grep, and a second `ci-failures` proving the cache makes it free. +- **Forgejo 15.0.2 / `~/dev/entriq`**, bare `gitea` preset, no overrides: `ci-runs` unfiltered, `--status failure`, and by branch via the PR-number mapping; `ci-run-view` on run 11130 via the tasks scan; `ci-failures` and `ci-run-log` returning the named `unsupported-server` envelope. entriq's config is read-only for this — nothing is written to it. +- **Forgejo 16 / `codeberg.org/forgejo/forgejo`**: the v16 log path end-to-end on run 6554924 / job 11952749. See the open question below about how deep this can go. + +**Manual, at the `dev-approval` gate**: `codev doctor` in this worktree and in entriq shows the four concepts with the right source per provider; a deliberately broken `CODEV_CI_RUN_ID` produces a named error rather than an empty object. + +--- + +## Open Questions for the Architect + +1. **Codeberg token.** The gitea v16 log path is verified at HTTP level today (unauthenticated curl against `codeberg.org`, timings above). Driving it through the **real dispatcher** needs `tea` to have a login for codeberg, and `tea` requires a token to add one. If you have a Codeberg read-only token, that lane becomes a full end-to-end verification instead of an HTTP-level one. If not, I ship with the HTTP-level verification plus fixture tests and say exactly that in the review. +2. **Is a Forgejo 16 upgrade for `git.pseudoseed.com` on the cards?** It does not change the code — the same scripts light up — but it changes whether `ci-failures` is usable on entriq the day this merges, and I would rather the review state the real situation than a hopeful one. diff --git a/codev/state/pir-13_thread.md b/codev/state/pir-13_thread.md new file mode 100644 index 000000000..818c34d54 --- /dev/null +++ b/codev/state/pir-13_thread.md @@ -0,0 +1,35 @@ +# pir-13 thread — CI concepts for the forge layer (#13) + +## PLAN phase + +Investigation was mostly measurement, and it overturned three premises the issue rests on. + +1. **`gh run view --log-failed` does not narrow to the failing step.** On run 32518... (32515040122, + this repo, `Tests`, branch `builder/air-14`) it returned 2528 lines / 293 KB with every line + labelled `UNKNOWN STEP`. Job-scoped, not step-scoped. Codev has to extract on GitHub too. +2. **Forgejo 15.0.2 has no Actions log API at all.** `tea actions runs view` and `tea actions runs + logs` both 404 against git.pseudoseed.com — they call `/actions/runs/{id}/jobs` and + `/actions/jobs/{id}/logs`, which landed in **Forgejo 16.0** (2026-07-16). Probed every + alternative route; the web UI log route exists but is session-only (rejects token and basic auth, + while the API accepts the same token). So `ci-failures`/`ci-run-log` cannot work on entriq today. +3. **Forgejo 16 verified live and free** on codeberg.org (16.0.0-dev, `forgejo/forgejo` public): + `runs/{id}/jobs` ✓, `actions/jobs/{jobId}/logs` → 200 text/plain 142 KB / 1599 lines / 1.0 s, + `accept-ranges: bytes`, `?step=` ignored. Note the log route takes the **job id**, not the + `task_id` that `actions/tasks` lists. + +Other measured facts that shaped the design: Forgejo `actions/runs?limit=N` **ignores limit unless +`page` is present** (a bare `limit=3` returned all 6922 runs); `status=` filters server-side but +`branch=`/`event=` are ignored; PR-triggered runs report `head_branch` as `#`, not the branch, +so branch filtering needs the #12 base/head PR lookup first; run `id` and `index_in_repo` are two +valid id spaces for the same route, so the concept refuses to guess which one it was handed. + +And the extraction evidence: in that GitHub log the first `Error:` line is `[artifact-canvas] Error: +host blew up` — a **passing** test's fixture string at line 1257, while the real failure is at 2471. +`Test Files … passed` appears four times before the failing summary. Every payload line is wrapped in +ANSI SGR codes, so a matcher that does not strip ANSI matches nothing. All three are pinned as tests. + +Plan at `codev/plans/13-ci-forge-concepts.md`. Two open questions for the architect: a Codeberg token +(would upgrade the v16 lane from HTTP-level to full-dispatcher verification), and whether +git.pseudoseed.com is due a Forgejo 16 upgrade. + +Awaiting `plan-approval`. From a35eaa2a4d44a75ee4e4f60decbf4bc80f76884b Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 14:58:03 -0600 Subject: [PATCH 03/30] chore(porch): 13 plan-approval gate-requested --- codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 6f50facb9..3941e53f7 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -7,6 +7,7 @@ current_plan_phase: null gates: plan-approval: status: pending + requested_at: '2026-08-21T20:58:03.660Z' dev-approval: status: pending pr: @@ -15,4 +16,4 @@ iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T20:35:23.799Z' +updated_at: '2026-08-21T20:58:03.661Z' From 660b8c86eaf0b04c99e02f59c749989dfdb8333c Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 14:58:47 -0600 Subject: [PATCH 04/30] chore(porch): 13 plan-approval gate-approved --- codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 3941e53f7..67749e906 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -6,8 +6,9 @@ plan_phases: [] current_plan_phase: null gates: plan-approval: - status: pending + status: approved requested_at: '2026-08-21T20:58:03.660Z' + approved_at: '2026-08-21T20:58:47.286Z' dev-approval: status: pending pr: @@ -16,4 +17,4 @@ iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T20:58:03.661Z' +updated_at: '2026-08-21T20:58:47.287Z' From 45e3429f8017401aa5cb8d6883f7c7f94ce90904 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 14:59:08 -0600 Subject: [PATCH 05/30] chore(porch): 13 implement phase-transition --- codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 67749e906..38ca610b9 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -1,7 +1,7 @@ id: '13' title: add-ci-concepts-to-the-forge-l protocol: pir -phase: plan +phase: implement plan_phases: [] current_plan_phase: null gates: @@ -17,4 +17,4 @@ iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T20:58:47.287Z' +updated_at: '2026-08-21T20:59:08.728Z' From 700aefc63a24fe3e2430dc006b4307e02a2d1c83 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:25:58 -0600 Subject: [PATCH 06/30] =?UTF-8?q?[PIR=20#13]=20feat(forge):=20CI=20concept?= =?UTF-8?q?=20plumbing=20=E2=80=94=20shared=20timeout,=20extraction=20ladd?= =?UTF-8?q?er,=20envelope?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Registers ci-runs, ci-run-view, ci-failures and ci-run-log in KNOWN_CONCEPTS and DISABLES all four for gitlab and linear — a concept with no script falls through to the github default, so leaving them unset would make a GitLab repo silently run `gh run list` (#1455's class). Adds executeForgeCommandDetailed: executeForgeCommand collapses a timeout, a non-zero exit, unparseable output and a disabled concept to one `null`, which is how #12 shipped a pr-exists whose null read as 'no PR exists'. The detailed variant keeps stdout even on failure, because the CI concepts print their error envelope there. _timeout.sh lifts #12's watchdog out of gitea/_lib.sh so gh calls get the same guarantee, and reaps the watchdog so it stops printing 'Terminated: 15' onto every caller's stderr. _ci-extract.sh is the extraction ladder. It strips ANSI first — without that, the payload line `ESC[41mESC[1m FAIL ESC[22m…` matches nothing and a log that plainly contains a failure reports none. Ladder: vitest/jest, go test, tsc, the runner's own ##[error] marker, then a line-ANCHORED first error, then an honest refusal. Anchoring is load-bearing: the first line containing 'Error:' in the reference log is '[artifact-canvas] Error: host blew up', printed by a passing test 1214 lines above the real failure. Co-Authored-By: Claude Opus 5 --- packages/codev/scripts/forge/_ci-extract.sh | 202 +++++++++++ packages/codev/scripts/forge/_ci-lib.sh | 355 ++++++++++++++++++++ packages/codev/scripts/forge/_timeout.sh | 100 ++++++ packages/codev/scripts/forge/gitea/_lib.sh | 87 +---- packages/codev/src/lib/forge-contracts.ts | 181 ++++++++++ packages/codev/src/lib/forge.ts | 130 ++++++- 6 files changed, 974 insertions(+), 81 deletions(-) create mode 100644 packages/codev/scripts/forge/_ci-extract.sh create mode 100644 packages/codev/scripts/forge/_ci-lib.sh create mode 100644 packages/codev/scripts/forge/_timeout.sh diff --git a/packages/codev/scripts/forge/_ci-extract.sh b/packages/codev/scripts/forge/_ci-extract.sh new file mode 100644 index 000000000..e7a403658 --- /dev/null +++ b/packages/codev/scripts/forge/_ci-extract.sh @@ -0,0 +1,202 @@ +# Extraction ladder for the CI concepts (#13). SOURCED, not executed. +# +# Turns a raw CI job log into the few lines that say why the job failed, or into +# an honest refusal. Shared by github/ and gitea/ so the two providers cannot +# drift: the runners differ, the failure text does not. +# +# WHY THIS EXISTS AT ALL, ON BOTH PROVIDERS +# +# Issue #13 says `gh run view --log-failed` "already does the extraction for +# you" and tells the implementer not to duplicate it. Measured on run +# 32515040122 of this repository, it does not: it returned 2528 lines / 293 KB +# with every single line tagged "UNKNOWN STEP", i.e. it selected the failing +# JOB and returned all of it. (The architect independently reproduced this on +# run 32448538074: 919 lines, all 919 tagged UNKNOWN STEP.) `gh` attributes log +# lines to steps by matching filenames, and falls back to UNKNOWN STEP when that +# mapping fails. So codev extracts on GitHub too, and the concepts fetch +# `repos/{owner}/{repo}/actions/jobs/{id}/logs` instead — one job, no invented +# step column, and the same shape Forgejo 16 serves. +# +# THE THREE THINGS THAT MAKE NAIVE EXTRACTION LIE +# +# All three are from the same real log, and all three are pinned by tests: +# +# 1. ANSI. The payload line is not `FAIL src/…`, it is +# `\033[41m\033[1m FAIL \033[22m\033[49m src/…`. A matcher that does not +# strip ANSI first matches NOTHING and reports "no recognized failure" on a +# log that plainly contains one. Cleaning is not cosmetic; it is the +# difference between working and silently giving up. +# 2. "First line matching an error pattern" returns line 1257 of 2528: +# `[artifact-canvas] Error: host blew up` — a fixture string printed by a +# PASSING test. The real failure is at 2471. Hence rung 4 anchors its +# patterns at the START of the line: that fixture's "Error:" is mid-line, so +# anchoring alone kills it, and anchoring holds even in logs with no test +# summary to measure against. +# 3. `Test Files … passed` appears FOUR times before the failing summary. Any +# rule that takes the first match reports a passing suite as the failure. +# +# And the rule that governs the whole ladder: when nothing matches, return +# nothing. A builder handed 50 arbitrary lines treats them as the diagnosis and +# reasons from noise; a builder told extraction failed goes and reads the log, +# which is correct and cheaper. See lessons-critical.md. + +# Clean a raw log on stdin into extractable text on stdout. +# +# Strips, in order: a UTF-8 BOM (GitHub's logs open with one), ANSI CSI/OSC +# escape sequences, carriage returns, and the leading RFC3339 timestamp that +# BOTH providers prefix to every line +# (`2026-08-21T18:47:09.5820646Z Current runner version: …`). +# +# Deliberately does NOT strip gh's `jobstep` prefix: the concepts do +# not use `--log-failed`, so no such prefix exists, and a rule that ate tab- +# separated leading fields would eat real log content on a runner that prints +# tables. +ci_clean_log() { + _esc=$(printf '\033') + LC_ALL=C sed \ + -e '1s/^\xef\xbb\xbf//' \ + -e "s/${_esc}\][^${_esc}]*${_esc}\\\\//g" \ + -e "s/${_esc}\[[0-9;?]*[ -\/]*[@-~]//g" \ + -e "s/${_esc}[()][A-Za-z0-9]//g" \ + -e 's/\r$//' \ + -e 's/^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9][.0-9]*Z //' +} + +# Run the ladder over a CLEANED log file. +# +# ci_extract +# +# On a match, prints a header line `\t\t` followed by +# the extracted lines. On no match, prints NOTHING and returns 1 — which is the +# `extracted: false` path, not an error. +# +# Line numbers are 1-based into the cleaned log, and the cleaner is line-count +# preserving, so they are also line numbers into the raw log — which is what +# makes the `ci-run-log` handoff land where `ci-failures` was looking. +ci_extract() { + awk ' + function is_noise_marker(s) { + # `##[error]Process completed with exit code 1.` is the runner reporting + # that the step failed. True, and not a diagnosis of anything — returning + # it alone would be an arbitrary line dressed as an answer. + return (s ~ /^##\[error\](Process completed with exit code|The (job|operation) was canceled)/) + } + { line[NR] = $0 } + END { + n = NR + if (n == 0) exit 1 + + # ---- pass 1: index the landmarks ------------------------------------- + first_marker = 0 + last_fail_summary = 0 # vitest "Test Files … failed" / jest "Tests: … failed" + last_pass_boundary = 0 # the last "N passed" summary of ANY suite + last_failed_banner = 0 # vitest "⎯⎯ Failed Tests N ⎯⎯" + first_ts_error = 0 + first_go_fail = 0 + for (i = 1; i <= n; i++) { + s = line[i] + if (!first_marker && s ~ /^##\[error\]/ && !is_noise_marker(s)) first_marker = i + if (s ~ /Failed Tests/) last_failed_banner = i + if (s ~ /(Test Files|Tests:|Test Suites:)/) { + if (s ~ /fail/) last_fail_summary = i + else if (s ~ /pass/) last_pass_boundary = i + } + if (s ~ /^[0-9]+ (passing|tests? passed)/) last_pass_boundary = i + if (!first_ts_error && s ~ /error TS[0-9]+:/) first_ts_error = i + if (!first_go_fail && s ~ /^ *--- FAIL: /) first_go_fail = i + } + + # ---- rung 1: a recognised test runner -------------------------------- + # Anchored on the summary that says "failed" — NOT the first summary, of + # which there were four saying "passed" in the log this was built from. + if (last_fail_summary) { + start = 0 + if (last_failed_banner && last_failed_banner < last_fail_summary) start = last_failed_banner + if (!start) { + # earliest FAIL after the last passing summary that precedes the + # failing one, so a re-run of the same suite does not drag in the + # passing run above it + floor = 0 + for (i = 1; i < last_fail_summary; i++) + if (line[i] ~ /(Test Files|Tests:|Test Suites:)/ && line[i] ~ /pass/) floor = i + for (i = floor + 1; i < last_fail_summary; i++) + if (line[i] ~ /(^| )FAIL( |$)/ || line[i] ~ /^ *(✕|×) /) { start = i; break } + } + if (start) { emit("vitest", start, last_fail_summary); exit 0 } + } + + # ---- rung 2: go test ------------------------------------------------- + # The Forgejo fixture this was verified against is a Go suite, and it is + # the case that best justifies the whole ladder: the failure sits at line + # 1292 of 1599 and the last 25 lines of that log are git credential + # cleanup, so a tail returns nothing at all. The block runs from the + # --- FAIL: line to the package summary (FAILpkg0.302s). + if (first_go_fail) { + to = first_go_fail + for (i = first_go_fail + 1; i <= n && i <= first_go_fail + 40; i++) { + to = i + if (line[i] ~ /^FAIL\t/ || line[i] ~ /^ok \t/) break + } + emit("go-test", first_go_fail, to) + exit 0 + } + + # ---- rung 3: a compiler ---------------------------------------------- + if (first_ts_error) { + to = first_ts_error + for (i = first_ts_error + 1; i <= n && i <= first_ts_error + 20; i++) { + if (line[i] ~ /error TS[0-9]+:/ || line[i] ~ /^[ \t]/) to = i; else break + } + emit("tsc", first_ts_error, to) + exit 0 + } + + # ---- rung 4: the runner error marker --------------------------------- + # GitHub puts the message INSIDE the marker (##[error]AssertionError: + # expected null to be unauth), so the marker line is itself the answer and + # what FOLLOWS it is the useful context. What precedes it, on the log this + # was built from, is a vitest duration line and two blank lines. This sits + # below runner recognition, not above it, because rung 1 returns the whole + # Failed Tests block (test name, assertion, expected/received, file:line) + # where this returns one sentence. Issue #13 priority order agrees: + # recognise the runner first. + if (first_marker) { + to = first_marker + 3; if (to > n) to = n + emit("runner-marker", first_marker, to) + exit 0 + } + + # ---- rung 5: the first ANCHORED error, preferring after the last pass - + # Anchoring at the start of the line is what makes this safe: the false + # positive this ladder was built against ("[artifact-canvas] Error: host + # blew up", printed by a passing test) has its "Error:" mid-line and + # cannot match. Searching after the last passing summary is an additional + # preference, not the safety property — logs from a failed install or a + # crashed runner have no summary at all, and refusing to answer for that + # whole class would buy no safety. + for (pass = 1; pass <= 2; pass++) { + start = (pass == 1) ? last_pass_boundary + 1 : 1 + if (pass == 2 && last_pass_boundary == 0) break + for (i = start; i <= n; i++) { + s = line[i] + if (s ~ /^(Error|error|ERROR|Exception|FATAL|fatal( error)?|panic|Traceback \(most recent call last\)|npm ERR!|##\[error\])[: ]/ || + s ~ /^[A-Za-z_][A-Za-z0-9_.]*(Error|Exception): / || + s ~ /^error(\[[A-Z0-9]+\])?: /) { + if (is_noise_marker(s)) continue + from = i - 3; if (from < 1) from = 1 + to = i + 3; if (to > n) to = n + emit("first-error", from, to) + exit 0 + } + } + } + + # ---- rung 6: give up honestly ---------------------------------------- + exit 1 + } + function emit(rung, from, to, i) { + printf "%s\t%d\t%d\n", rung, from, to + for (i = from; i <= to; i++) print line[i] + } + ' "$1" +} diff --git a/packages/codev/scripts/forge/_ci-lib.sh b/packages/codev/scripts/forge/_ci-lib.sh new file mode 100644 index 000000000..0f0c44e6d --- /dev/null +++ b/packages/codev/scripts/forge/_ci-lib.sh @@ -0,0 +1,355 @@ +# Shared plumbing for the CI concepts (#13). SOURCED, not executed. +# +# Holds the response envelope, the caps, the log cache, and the window parsing — +# everything the four concepts share that is not the extraction ladder itself +# (that is _ci-extract.sh). POSIX sh; jq is required, as it already is for +# pr-list and every gitea concept. +# +# THE ENVELOPE, AND WHY ERRORS ARE PRINTED ON STDOUT +# +# Every ci-* concept prints ONE JSON object on stdout, on success and on +# failure. A caller therefore always has something structured to read, and never +# has to interpret an empty string. +# +# That includes the failure paths, which is a deliberate departure from the +# other concepts. `executeForgeCommand` collapses every failure mode to `null` +# (forge.ts), so "the API timed out at 60s", "the run does not exist" and "the +# server has no log API" arrive identically — and #12 spent a phase on exactly +# that ambiguity with pr-exists returning null-that-read-as-false. Printing the +# envelope regardless of exit status means the class of failure survives the +# trip. Exit statuses still follow the #12 contract: 0 answered, 1 could not +# answer, 2 missing or unusable input. +# +# The one rule that outranks the rest: a response that carries log text ALWAYS +# carries logLines, returnedLines and truncated, so a trimmed answer can never +# read as a whole one. + +CI_LIMIT_DEFAULT=20 + +# Per-extract and per-response byte caps (issue #13: "a few KB per failing +# step"). A cap that bites is always reported as truncated:true — never +# silently. +CI_MAX_STEP_BYTES=${CODEV_CI_MAX_STEP_BYTES:-2048} +CI_MAX_RESPONSE_BYTES=${CODEV_CI_MAX_BYTES:-8192} +case "$CI_MAX_STEP_BYTES" in ''|*[!0-9]*) CI_MAX_STEP_BYTES=2048 ;; esac +case "$CI_MAX_RESPONSE_BYTES" in ''|*[!0-9]*) CI_MAX_RESPONSE_BYTES=8192 ;; esac + +# How many list pages a client-side filter may walk before it stops and says so. +CI_MAX_PAGES=${CODEV_CI_MAX_PAGES:-4} +case "$CI_MAX_PAGES" in ''|*[!0-9]*|0) CI_MAX_PAGES=4 ;; esac + +# A separate, higher ceiling for the Forgejo-15 task scan, which is a TARGETED +# lookup for one known run rather than an open-ended filter: it stops the moment +# it has walked past that run, so a recent run costs one page and the ceiling +# only bites on old ones. Measured on the reference Forgejo, a page of 50 tasks +# spans about 6 runs (~8 jobs per run), so 4 pages reached only 24 runs back and +# reported truncation for anything older — technically honest and practically +# useless. 20 pages is ~120 runs of history at ~0.25s per page. +CI_TASKS_MAX_PAGES=${CODEV_CI_TASKS_MAX_PAGES:-20} +case "$CI_TASKS_MAX_PAGES" in ''|*[!0-9]*|0) CI_TASKS_MAX_PAGES=20 ;; esac + +# The status vocabulary a caller may use, provider-independent. Taken from what +# Forgejo accepts; GitHub spells one of them differently and is translated in +# ci_status_for. +CI_STATUS_VOCABULARY="success failure pending queued in_progress skipped canceled" + +# Print the error envelope on stdout, a human line on stderr, and return 1. +# +# ci_fail [extra-json-object] +# +# `kind` is a stable machine token — timeout, not-found, unsupported-server, +# forge-error, bad-input — and `detail` is the sentence a person reads. +ci_fail() { + _concept="$1"; _kind="$2"; _detail="$3"; _extra="$4" + [ -n "$_extra" ] || _extra='{}' + jq -cn --arg kind "$_kind" --arg detail "$_detail" --argjson extra "$_extra" \ + '{ok: false, error: $kind, detail: $detail} + $extra' + echo "${_concept}: ${_detail}" >&2 + # Returns 0 DELIBERATELY. Every concept runs under `set -e`, so a non-zero + # return here would abort the script at this line — before the caller's own + # `exit 2` for a bad input could run, turning every input error into a + # generic exit 1. Reporting and deciding the exit status are separate jobs; + # this one only reports. + return 0 +} + +# The timeout envelope. Kept separate from ci_fail so that every concept spells +# a timeout the same way and none of them can quietly turn one into a generic +# failure or an empty result — the specific thing #17, #8 and #12 all ran into. +ci_fail_timeout() { + _concept="$1"; _what="$2"; _seconds="$3" + ci_fail "$_concept" timeout \ + "${_what} did not return within ${_seconds}s" \ + "$(jq -cn --argjson s "$_seconds" '{seconds: $s, remedy: "raise CODEV_FORGE_TIMEOUT"}')" +} + +# Validate a caller-supplied status against the shared vocabulary. +# Unknown values exit 2 naming the accepted set rather than being passed to the +# forge, where GitHub would reject them with its own wording and Forgejo would +# silently return everything. +ci_check_status() { + _concept="$1"; _status="$2" + [ -n "$_status" ] || return 0 + for _s in $CI_STATUS_VOCABULARY; do + [ "$_s" = "$_status" ] && return 0 + done + _msg="CODEV_CI_STATUS=${_status} is not one of: ${CI_STATUS_VOCABULARY}" + jq -cn --arg d "$_msg" '{ok: false, error: "bad-input", detail: $d}' + echo "${_concept}: ${_msg}" >&2 + exit 2 +} + +# Translate the shared vocabulary into what a provider CLI expects. +# GitHub spells it "cancelled"; Forgejo spells it "canceled". Everything else is +# shared, which is why the vocabulary is worth having. +ci_status_for() { + _provider="$1"; _status="$2" + if [ "$_provider" = "github" ] && [ "$_status" = "canceled" ]; then + printf 'cancelled' + else + printf '%s' "$_status" + fi +} + +# Is this job status terminal? Only a terminal job has an immutable log, and +# only an immutable log may be cached. +ci_status_is_terminal() { + case "$1" in + success|failure|skipped|canceled|cancelled|completed|timed_out|neutral|stale|startup_failure|failed) return 0 ;; + *) return 1 ;; + esac +} + +# --------------------------------------------------------------------------- +# Log cache +# --------------------------------------------------------------------------- +# +# A completed run log is immutable, so the realistic sequence — ci-failures, +# then ci-run-log with a tail, then ci-run-log with a grep — should cost exactly +# one download instead of three. Cached under TMPDIR rather than in the repo: +# these are large and disposable, and TMPDIR inherits the OS cleanup. +# +# In-progress jobs are NEVER cached and never read from cache. Caching a running +# job would hand back a log that stops mid-failure and looks complete, which is +# the failure mode this whole issue is about. + +CI_CACHE_MAX_MB=${CODEV_CI_CACHE_MAX_MB:-32} +case "$CI_CACHE_MAX_MB" in ''|*[!0-9]*) CI_CACHE_MAX_MB=32 ;; esac + +ci_cache_path() { + _provider="$1"; _slug="$2"; _job="$3" + _slug=$(printf '%s' "$_slug" | tr '/ ' '__') + printf '%s/codev-ci-logs/%s/%s/%s.log' "${TMPDIR:-/tmp}" "$_provider" "$_slug" "$_job" +} + +# Copy a cached log to stdout if it exists and caching is enabled. +ci_cache_read() { + [ "$CODEV_CI_NO_CACHE" = "1" ] && return 1 + [ -s "$1" ] || return 1 + cat "$1" +} + +# Store a log, but only for a terminal job and only under the size cap. +ci_cache_write() { + _path="$1"; _src="$2"; _status="$3" + [ "$CODEV_CI_NO_CACHE" = "1" ] && return 0 + ci_status_is_terminal "$_status" || return 0 + _bytes=$(wc -c < "$_src" | tr -d ' ') + [ "$_bytes" -le $((CI_CACHE_MAX_MB * 1024 * 1024)) ] || return 0 + mkdir -p "$(dirname "$_path")" 2>/dev/null || return 0 + cp "$_src" "${_path}.tmp$$" 2>/dev/null && mv "${_path}.tmp$$" "$_path" 2>/dev/null + return 0 +} + +# --------------------------------------------------------------------------- +# Windows for ci-run-log +# --------------------------------------------------------------------------- +# +# Exactly one of head/tail/grep, and no default. A defaulted window is how this +# concept turns into "tail by habit", which is the thing the issue asks for it +# to be separate in order to prevent. Zero windows and two windows are both +# exit 2 with a named message. +ci_window_parse() { + _concept="$1" + CI_WIN_KIND="" + CI_WIN_ARG="" + _count=0 + if [ -n "$CODEV_CI_LOG_TAIL" ]; then CI_WIN_KIND=tail; CI_WIN_ARG="$CODEV_CI_LOG_TAIL"; _count=$((_count + 1)); fi + if [ -n "$CODEV_CI_LOG_HEAD" ]; then CI_WIN_KIND=head; CI_WIN_ARG="$CODEV_CI_LOG_HEAD"; _count=$((_count + 1)); fi + if [ -n "$CODEV_CI_LOG_GREP" ]; then CI_WIN_KIND=grep; CI_WIN_ARG="$CODEV_CI_LOG_GREP"; _count=$((_count + 1)); fi + + if [ "$_count" -eq 0 ]; then + _msg="exactly one window is required: CODEV_CI_LOG_TAIL=N, CODEV_CI_LOG_HEAD=N, or CODEV_CI_LOG_GREP=" + elif [ "$_count" -gt 1 ]; then + _msg="exactly one window may be set; got ${_count} of CODEV_CI_LOG_TAIL / CODEV_CI_LOG_HEAD / CODEV_CI_LOG_GREP" + else + case "$CI_WIN_KIND" in + head|tail) + case "$CI_WIN_ARG" in + ''|*[!0-9]*|0) + _upper=$(printf '%s' "$CI_WIN_KIND" | tr 'a-z' 'A-Z') + _msg="CODEV_CI_LOG_${_upper} must be a positive number of lines, got '${CI_WIN_ARG}'" ;; + *) return 0 ;; + esac + ;; + *) return 0 ;; + esac + fi + + jq -cn --arg d "$_msg" '{ok: false, error: "bad-input", detail: $d}' + echo "${_concept}: ${_msg}" >&2 + exit 2 +} + +CI_GREP_CONTEXT=${CODEV_CI_LOG_CONTEXT:-3} +case "$CI_GREP_CONTEXT" in ''|*[!0-9]*) CI_GREP_CONTEXT=3 ;; esac + +# --------------------------------------------------------------------------- +# Turning text into a capped JSON payload +# --------------------------------------------------------------------------- + +# Copy a file to , capped at bytes, whole lines only, from the head +# — for an extract the assertion is at the top and half a line is worse than a +# missing one. Echoes " ". +# +# The count and the truncation flag are RETURNED, not assigned to globals: every +# caller runs this inside a command substitution, where a subshell assignment is +# discarded and the caller then hands jq an empty --argjson. That mistake was +# made here once and cost a debugging round; the shape now makes it impossible. +ci_cap_file() { + _src="$1"; _cap="$2"; _dest="$3" + _bytes=$(wc -c < "$_src" | tr -d ' ') + if [ "$_bytes" -le "$_cap" ]; then + cp "$_src" "$_dest" + _trunc=false + else + awk -v cap="$_cap" '{ t += length($0) + 1; if (t > cap) exit; print }' "$_src" > "$_dest" + _trunc=true + fi + printf '%s %s' "$(awk 'END {print NR}' "$_dest")" "$_trunc" +} + +# --------------------------------------------------------------------------- +# Tool invocation +# --------------------------------------------------------------------------- + +. "$(dirname "$0")/../_timeout.sh" + +# Wall-clock ceiling for one forge CLI call. Same knob as the gitea concepts +# use, deliberately: an operator raising CODEV_FORGE_TIMEOUT for a slow forge +# should not have to discover that CI has its own. +CI_TIMEOUT=${CODEV_FORGE_TIMEOUT:-60} +case "$CI_TIMEOUT" in ''|*[!0-9]*) CI_TIMEOUT=60 ;; esac + +# Run a forge CLI under the timeout. Returns the command status, or 124 for a +# timeout — which the caller MUST turn into ci_fail_timeout rather than folding +# into a generic failure. This function does not print the envelope itself +# because every caller captures its stdout. +ci_tool() { + _rc=0 + forge_timeout "$CI_TIMEOUT" "$@" || _rc=$? + return "$_rc" +} + +# owner/repo for cache keys and API paths. Honors CODEV_REPO, else derives it +# from the origin remote. Unlike gitea_repo this never fails the concept — the +# only caller that cannot proceed without it says so itself — so an +# underivable slug degrades to "unknown-repo" for the cache path alone. +ci_repo_slug() { + _repo="${CODEV_REPO:-$(git remote get-url origin 2>/dev/null | sed -E -e 's#\.git$##' -e 's#.*[/:]([^/]+/[^/]+)$#\1#')}" + case "$_repo" in + */*) printf '%s' "$_repo" ;; + *) printf 'unknown-repo' ;; + esac +} + + +# --------------------------------------------------------------------------- +# Windowing a cleaned log into the ci-run-log response +# --------------------------------------------------------------------------- +# +# ci_window_emit +# +# Reads /clean.log, applies the window ci_window_parse selected, and +# prints the response. Shared by both providers so that a tail means the same +# thing on GitHub and on Forgejo. +# +# `from` and `to` are line numbers into the FULL log, always, so a caller knows +# where in the log it is standing and whether more exists on either side. In +# grep mode the selected lines are not contiguous, so `contiguous: false` says +# so and `matchLines` lists exactly which lines matched — otherwise a reader +# would have to infer which of the returned lines were hits and which were +# context, and inference is what these concepts exist to remove. +ci_window_emit() { + _concept="$1"; _tmp="$2"; _provider="$3"; _run="$4"; _job="$5"; _jobname="$6"; _cached="$7" + _log="${_tmp}/clean.log" + _total=$(awk 'END {print NR}' "$_log") + _matchlines=null + _matches=null + _contiguous=true + + case "$CI_WIN_KIND" in + head) + _from=1 + _to=$CI_WIN_ARG + [ "$_to" -gt "$_total" ] && _to=$_total + sed -n "${_from},${_to}p" "$_log" > "${_tmp}/window.txt" + ;; + tail) + _from=$((_total - CI_WIN_ARG + 1)) + [ "$_from" -lt 1 ] && _from=1 + _to=$_total + sed -n "${_from},${_to}p" "$_log" > "${_tmp}/window.txt" + ;; + grep) + if ! awk -v pat="$CI_WIN_ARG" -v ctx="$CI_GREP_CONTEXT" \ + -v out="${_tmp}/window.txt" -v nums="${_tmp}/matches.txt" ' + { line[NR] = $0; if ($0 ~ pat) hit[NR] = 1 } + END { + for (i = 1; i <= NR; i++) + if (i in hit) { + lo = i - ctx; if (lo < 1) lo = 1 + hi = i + ctx; if (hi > NR) hi = NR + for (j = lo; j <= hi; j++) sel[j] = 1 + print i > nums + } + first = 0; last = 0 + for (i = 1; i <= NR; i++) + if (i in sel) { print line[i] > out; if (!first) first = i; last = i } + print first, last + }' "$_log" > "${_tmp}/bounds.txt" 2>"${_tmp}/awk.err"; then + ci_fail "$_concept" bad-input \ + "CODEV_CI_LOG_GREP is not a usable pattern: $(tr -d '\n' < "${_tmp}/awk.err")" + exit 2 + fi + _from=$(cut -d' ' -f1 "${_tmp}/bounds.txt") + _to=$(cut -d' ' -f2 "${_tmp}/bounds.txt") + [ -f "${_tmp}/window.txt" ] || : > "${_tmp}/window.txt" + [ -f "${_tmp}/matches.txt" ] || : > "${_tmp}/matches.txt" + _matches=$(wc -l < "${_tmp}/matches.txt" | tr -d ' ') + _matchlines=$(jq -R -s -c 'split("\n") | map(select(. != "") | tonumber)' < "${_tmp}/matches.txt") + _contiguous=false + # No match is an answer, not a failure: an empty window with matches: 0. + [ "$_from" -eq 0 ] && _from=0 && _to=0 + ;; + esac + + _meta=$(ci_cap_file "${_tmp}/window.txt" "$CI_MAX_RESPONSE_BYTES" "${_tmp}/capped.txt") + _returned=${_meta% *} + _trunc=${_meta#* } + _lines=$(jq -R -s -c 'split("\n") | if (.[-1] == "") then .[0:-1] else . end' < "${_tmp}/capped.txt") + + jq -cn \ + --arg provider "$_provider" --arg run "$_run" --argjson job "$_job" --arg jobName "$_jobname" \ + --arg kind "$CI_WIN_KIND" --arg arg "$CI_WIN_ARG" \ + --argjson total "$_total" --argjson returned "$_returned" --argjson truncated "$_trunc" \ + --argjson from "$_from" --argjson to "$_to" --argjson lines "$_lines" \ + --argjson matchLines "$_matchlines" --argjson matches "$_matches" \ + --argjson contiguous "$_contiguous" --argjson cached "$_cached" \ + '{ok: true, provider: $provider, runId: ($run | tonumber? // $run), jobId: $job, jobName: $jobName, + window: {kind: $kind, arg: $arg}, + logLines: $total, returnedLines: $returned, from: $from, to: $to, + contiguous: $contiguous, truncated: $truncated, + matches: $matches, matchLines: $matchLines, + cached: $cached, lines: $lines}' +} diff --git a/packages/codev/scripts/forge/_timeout.sh b/packages/codev/scripts/forge/_timeout.sh new file mode 100644 index 000000000..4ee5f1d82 --- /dev/null +++ b/packages/codev/scripts/forge/_timeout.sh @@ -0,0 +1,100 @@ +# Shared wall-clock timeout for forge concept scripts. +# +# SOURCED, not executed, so it has no shebang and defines only functions. POSIX +# sh only — the concept scripts are #!/bin/sh and forge runs them via `sh -c`. +# The leading underscore keeps it out of the concept namespace (forge.ts builds +# presets from an explicit KNOWN_CONCEPTS allowlist, so this file is never +# registered as a concept). +# +# This started life inside gitea/_lib.sh (#12), where it fixed a `tea api` call +# that stalled a whole porch phase. The CI concepts (#13) need the identical +# guarantee around `gh`, and a second copy of a hand-rolled process watchdog is +# the last thing this repo needs — so it lives here and both providers source +# it. `gitea_timeout` remains a one-line alias in gitea/_lib.sh so #12's scripts +# and the comments that reference it by name still read true. + +# Run a command under a wall-clock limit. Returns the command's own exit status, +# or 124 (the exit status `timeout(1)` uses) if the limit was reached. +# +# Deliberately does NOT use timeout(1)/gtimeout even when present. macOS ships +# neither by default, so the fallback would be the path that actually runs for +# most adopters while the tested-in-CI path would be the one that doesn't — +# exactly the arrangement where the untested path rots. One implementation, +# same behaviour everywhere. +# +# TWO THINGS HERE ARE LOAD-BEARING, and both exist because killing the command +# is not the same as unblocking the caller: +# +# 1. The command's stdout goes to a TEMP FILE, not to the caller's pipe. Every +# caller runs this inside `$(...)`. A killed command can leave a grandchild +# holding the write end of that pipe, and the command substitution then +# blocks forever on a process nobody is waiting for — the timeout fires, the +# message prints, and the script still hangs. Measured, not theorised: with +# the command writing straight to the pipe, a 3s timeout against a wrapper +# that spawns `sleep 300` printed its timeout message at 3s and was still +# blocked two minutes later. +# 2. The watchdog subshell's own stdout goes to /dev/null, for the same reason. +# +# Grandchildren are swept with `pkill -P` where it exists. That is best-effort +# and not the guarantee — (1) is the guarantee, and it holds even where `pkill` +# does not exist. +forge_timeout() { + _limit="$1" + shift + # Both files live in a private mktemp DIRECTORY. The marker's path used to be + # derived from the output file's ("$_tf.fired"), which mktemp does not reserve + # — a predictable name in a world-writable tmpdir that anyone could pre-create + # to make every call report a timeout. + _dir=$(mktemp -d) || return 1 + _tf="${_dir}/out" + _fired="${_dir}/fired" + "$@" >"$_tf" & + _cmd_pid=$! + ( sleep "$_limit" + # Claim the timeout only if there is still something to kill. Writing the + # marker unconditionally misreports a command that finished in the same + # instant the deadline passed — it succeeded, and saying otherwise discards + # a good answer. `kill -0` narrows that window to the gap between this test + # and the signal; it cannot be closed entirely without a lock, and a + # false timeout is a retryable error rather than a wrong answer. + if kill -0 "$_cmd_pid" 2>/dev/null; then + : > "$_fired" + pkill -TERM -P "$_cmd_pid" 2>/dev/null + kill -TERM "$_cmd_pid" 2>/dev/null + sleep 2 + pkill -KILL -P "$_cmd_pid" 2>/dev/null + kill -KILL "$_cmd_pid" 2>/dev/null + fi + ) >/dev/null 2>&1 & + _wd_pid=$! + # `|| _rc=$?` rather than `wait; _rc=$?`: the concept scripts run under + # `set -e`, which would abort here the moment the wrapped command exited + # non-zero — before the timeout could be classified and named. + _rc=0 + wait "$_cmd_pid" || _rc=$? + # Kill the watchdog and REAP it. Without the wait, the shell prints its own + # "Terminated: 15" notice about the killed background job onto the caller's + # stderr, on every single call — noise that a concept returning structured + # JSON should not be emitting, and that trains a reader to ignore the stream + # where the real diagnostics also arrive. + kill "$_wd_pid" 2>/dev/null + wait "$_wd_pid" 2>/dev/null || : + + # Whether the watchdog fired is recorded by the watchdog, not INFERRED from + # the exit status. Inferring it (status 143/137 = "we killed it") looked + # equivalent and is not: a killed process can still exit 0. A wrapper whose + # own `wait` takes no operand does exactly that — POSIX defines operand-less + # `wait` as always returning zero — so its death by SIGTERM was reported as a + # successful call returning an empty body, and the caller then diagnosed an + # unreadable repository instead of a timeout. Found by the test that pins this + # function; the marker file cannot be wrong the same way. + if [ -f "$_fired" ]; then + rm -rf "$_dir" + return 124 + fi + # A half-written response is worse than no response, so output is emitted only + # on the non-timeout path. + cat "$_tf" + rm -rf "$_dir" + return "$_rc" +} diff --git a/packages/codev/scripts/forge/gitea/_lib.sh b/packages/codev/scripts/forge/gitea/_lib.sh index 726c23e17..dad3af36f 100755 --- a/packages/codev/scripts/forge/gitea/_lib.sh +++ b/packages/codev/scripts/forge/gitea/_lib.sh @@ -135,84 +135,17 @@ case "$GITEA_PAGED_DEADLINE" in ''|*[!0-9]*) GITEA_PAGED_DEADLINE=120 ;; esac -# Run a command under a wall-clock limit. Returns the command's own exit status, -# or 124 (the exit status `timeout(1)` uses) if the limit was reached. -# -# Deliberately does NOT use timeout(1)/gtimeout even when present. macOS ships -# neither by default, so the fallback would be the path that actually runs for -# most adopters while the tested-in-CI path would be the one that doesn't — -# exactly the arrangement where the untested path rots. One implementation, -# same behaviour everywhere. -# -# TWO THINGS HERE ARE LOAD-BEARING, and both exist because killing the command -# is not the same as unblocking the caller: -# -# 1. The command's stdout goes to a TEMP FILE, not to the caller's pipe. Every -# caller runs this inside `$(...)`. A killed command can leave a grandchild -# holding the write end of that pipe, and the command substitution then -# blocks forever on a process nobody is waiting for — the timeout fires, the -# message prints, and the script still hangs. Measured, not theorised: with -# the command writing straight to the pipe, a 3s timeout against a wrapper -# that spawns `sleep 300` printed its timeout message at 3s and was still -# blocked two minutes later. -# 2. The watchdog subshell's own stdout goes to /dev/null, for the same reason. -# -# Grandchildren are swept with `pkill -P` where it exists. That is best-effort -# and not the guarantee — (1) is the guarantee, and it holds even where `pkill` -# does not exist. -gitea_timeout() { - _limit="$1" - shift - # Both files live in a private mktemp DIRECTORY. The marker's path used to be - # derived from the output file's ("$_tf.fired"), which mktemp does not reserve - # — a predictable name in a world-writable tmpdir that anyone could pre-create - # to make every call report a timeout. - _dir=$(mktemp -d) || return 1 - _tf="${_dir}/out" - _fired="${_dir}/fired" - "$@" >"$_tf" & - _cmd_pid=$! - ( sleep "$_limit" - # Claim the timeout only if there is still something to kill. Writing the - # marker unconditionally misreports a command that finished in the same - # instant the deadline passed — it succeeded, and saying otherwise discards - # a good answer. `kill -0` narrows that window to the gap between this test - # and the signal; it cannot be closed entirely without a lock, and a - # false timeout is a retryable error rather than a wrong answer. - if kill -0 "$_cmd_pid" 2>/dev/null; then - : > "$_fired" - pkill -TERM -P "$_cmd_pid" 2>/dev/null - kill -TERM "$_cmd_pid" 2>/dev/null - sleep 2 - pkill -KILL -P "$_cmd_pid" 2>/dev/null - kill -KILL "$_cmd_pid" 2>/dev/null - fi - ) >/dev/null 2>&1 & - _wd_pid=$! - # `|| _rc=$?` rather than `wait; _rc=$?`: the concept scripts run under - # `set -e`, which would abort here the moment the wrapped command exited - # non-zero — before the timeout could be classified and named. - _rc=0 - wait "$_cmd_pid" || _rc=$? - kill "$_wd_pid" 2>/dev/null +# The wall-clock watchdog now lives in scripts/forge/_timeout.sh, because the +# CI concepts (#13) need the same guarantee around `gh` and a second hand-rolled +# process watchdog is the last thing this repo needs. `gitea_timeout` stays as +# the name every gitea script and comment already uses; the behaviour, and the +# two load-bearing details documented in _timeout.sh (the command's stdout goes +# to a temp file, and the watchdog records the timeout in a marker file rather +# than having it inferred from an exit status), are unchanged. +. "$(dirname "$0")/../_timeout.sh" - # Whether the watchdog fired is recorded by the watchdog, not INFERRED from - # the exit status. Inferring it (status 143/137 = "we killed it") looked - # equivalent and is not: a killed process can still exit 0. A wrapper whose - # own `wait` takes no operand does exactly that — POSIX defines operand-less - # `wait` as always returning zero — so its death by SIGTERM was reported as a - # successful call returning an empty body, and the caller then diagnosed an - # unreadable repository instead of a timeout. Found by the test that pins this - # function; the marker file cannot be wrong the same way. - if [ -f "$_fired" ]; then - rm -rf "$_dir" - return 124 - fi - # A half-written response is worse than no response, so output is emitted only - # on the non-timeout path. - cat "$_tf" - rm -rf "$_dir" - return "$_rc" +gitea_timeout() { + forge_timeout "$@" } # `tea api` under GITEA_TIMEOUT, with a named error on the timeout path. diff --git a/packages/codev/src/lib/forge-contracts.ts b/packages/codev/src/lib/forge-contracts.ts index 8a8e3e444..6efc9140b 100644 --- a/packages/codev/src/lib/forge-contracts.ts +++ b/packages/codev/src/lib/forge-contracts.ts @@ -201,3 +201,184 @@ export interface PrCreateResult { /** Output of the `auth-status` concept command: exit code only. */ // Exit code 0 = authenticated, non-zero = not authenticated. + +// ============================================================================= +// CI concepts (#13) +// ============================================================================= +// +// Four concepts, tiered so that the cheap question stays cheap: `ci-runs` and +// `ci-run-view` never read a log, `ci-failures` reads exactly one job's and +// returns a bounded extract, and `ci-run-log` is the deliberate raw window. +// +// Two contract-wide rules, both of which exist because breaking them produced +// real wrong answers: +// +// 1. **Every response that carries log text also carries `logLines`, +// `returnedLines` and `truncated`.** A trimmed answer must never read as a +// whole one. +// 2. **Failure is a value, not an absence.** These concepts print `CiError` on +// stdout even when they exit non-zero, so a timeout, a missing run and a +// server too old to answer stay distinguishable after +// `executeForgeCommand` has flattened everything else to `null`. Read them +// with `executeForgeCommandDetailed` when the distinction matters. + +/** Error envelope printed on stdout by any ci-* concept that cannot answer. */ +export interface CiError { + ok: false; + /** + * Stable machine token: + * - `timeout` — the forge did not answer inside the watchdog; carries `seconds` + * - `not-found` — no such run or job + * - `unsupported-server` — the forge is too old to have the API; carries + * `serverVersion` and `needs`. Distinct from "nothing failed", deliberately: + * Forgejo gained the Actions job-log API only in 16.0, and an empty + * `failures` array on an older server would say "your CI is fine" when the + * truth is "I cannot see your CI at all". + * - `forge-error` — the forge answered with an error + * - `bad-input` — missing or unusable input (exit 2) + */ + error: 'timeout' | 'not-found' | 'unsupported-server' | 'forge-error' | 'bad-input'; + detail: string; + seconds?: number; + remedy?: string; + serverVersion?: string; + needs?: string; + runId?: number | string; + jobId?: number | string; + jobName?: string; + /** Failing jobs the concept could still name on an unsupported server. */ + failingJobs?: Array<{ jobId?: number; taskId?: number; jobName: string }>; +} + +/** One run in `ci-runs` output. */ +export interface CiRunItem { + /** The API id. This is what ci-run-view / ci-failures / ci-run-log take. */ + id: number; + /** + * The human run number (`index_in_repo` on Forgejo, `number` on GitHub). + * Never pass this as CODEV_CI_RUN_ID: on Forgejo both id spaces resolve on + * the same route, so a number is silently a *different, real* run. + */ + number: number; + name: string; + workflow: string; + status: string; + /** GitHub only. Forgejo has no conclusion field — `status` carries it. */ + conclusion: string | null; + /** + * The branch as the forge recorded it. On Forgejo a `pull_request` run + * records `#` rather than a branch name; ci-runs resolves + * CODEV_BRANCH_NAME to that form before filtering. + */ + branch: string | null; + sha: string | null; + event: string; + url: string; + createdAt: string; +} + +/** Output of the `ci-runs` concept. */ +export interface CiRunsResult { + ok: true; + provider: string; + runs: CiRunItem[]; + /** True when more runs exist than were returned, for any reason. */ + truncated: boolean; + note?: string | null; +} + +/** One job in `ci-run-view` output. */ +export interface CiJobItem { + /** + * The job id the log concepts take. **Null on Forgejo < 16**, which exposes + * no jobs API: those jobs are recovered from `actions/tasks` and only a + * `taskId` exists. A task id looks like a job id and is not one, so it is + * never reported as `id`. + */ + id: number | null; + taskId?: number; + name: string; + status: string; + conclusion: string | null; + startedAt: string | null; + completedAt: string | null; + /** GitHub only; Forgejo exposes no per-step data, so this is `[]` there. */ + failedSteps: Array<{ name: string; number: number; conclusion: string }>; +} + +/** Output of the `ci-run-view` concept. */ +export interface CiRunViewResult { + ok: true; + provider: string; + /** Which route produced `jobs`: `run-view` | `runs-jobs` | `tasks-scan`. */ + jobSource: string; + run: Omit & { title?: string; name?: string }; + jobs: CiJobItem[]; + truncated?: boolean; +} + +/** One extracted failure in `ci-failures` output. */ +export interface CiFailureItem { + jobId: number; + jobName: string; + stepName?: string | null; + stepNumber?: number | null; + /** + * Which rung of the extraction ladder fired: `vitest`, `go-test`, `tsc`, + * `runner-marker`, `first-error`. Present so a reader can weigh the answer — + * a `first-error` match is a weaker claim than a `vitest` one. + */ + matchedBy?: string; + text?: string; + from?: number; + to?: number; + logLines: number; + returnedLines?: number; + truncated?: boolean; +} + +/** Output of the `ci-failures` concept. */ +export interface CiFailuresResult { + ok: true; + provider: string; + runId: number | string; + runStatus: string; + runConclusion: string | null; + jobsFailed: number; + /** + * False means extraction found nothing it recognised — NOT that the job + * passed. The response then carries `reason`, the job identity, `logLines`, + * and a ready-to-run `next` command, so the refusal is a handoff to + * ci-run-log rather than a dead end. It never falls back to returning + * arbitrary lines, which a reader would treat as a diagnosis. + */ + extracted: boolean; + reason?: string; + failures: CiFailureItem[]; + otherFailingJobs?: Array<{ id: number; name: string }>; + cached?: boolean; + next?: string; +} + +/** Output of the `ci-run-log` concept. */ +export interface CiRunLogResult { + ok: true; + provider: string; + runId: number | string; + jobId: number; + jobName: string; + window: { kind: 'head' | 'tail' | 'grep'; arg: string }; + logLines: number; + returnedLines: number; + /** First and last line numbers returned, into the FULL log. 0/0 for no match. */ + from: number; + to: number; + /** False in grep mode: the returned lines have gaps. See `matchLines`. */ + contiguous: boolean; + truncated: boolean; + /** grep mode only: how many lines matched, and which. */ + matches: number | null; + matchLines: number[] | null; + cached?: boolean; + lines: string[]; +} diff --git a/packages/codev/src/lib/forge.ts b/packages/codev/src/lib/forge.ts index 611df59fa..f8a34dce3 100644 --- a/packages/codev/src/lib/forge.ts +++ b/packages/codev/src/lib/forge.ts @@ -36,6 +36,16 @@ function resolveScriptPath(provider: string, concept: string): string { /** Default maxBuffer for forge commands (10MB). Prevents truncation for large diffs. */ const DEFAULT_MAX_BUFFER = 10 * 1024 * 1024; +/** + * Default wall-clock ceiling for a forge command (30s). + * + * The scripts carry their own, shorter watchdog (CODEV_FORGE_TIMEOUT, default + * 60s in scripts/forge/_timeout.sh) so that a stalled CLI surfaces as a NAMED + * timeout rather than as this outer kill, which can only report that something + * died. Both exist: the inner one explains, the outer one guarantees. + */ +const DEFAULT_TIMEOUT_MS = 30_000; + // ============================================================================= // Types // ============================================================================= @@ -55,6 +65,8 @@ export interface ForgeCommandOptions { raw?: boolean; /** Maximum stdout buffer size in bytes. Defaults to 10MB. */ maxBuffer?: number; + /** Wall-clock ceiling in ms. Defaults to 30s. */ + timeoutMs?: number; } // ============================================================================= @@ -66,6 +78,12 @@ const KNOWN_CONCEPTS = [ 'recently-closed', 'recently-merged', 'user-identity', 'team-activity', 'on-it-timestamps', 'pr-create', 'pr-merge', 'pr-search', 'pr-view', 'pr-diff', 'auth-status', 'repo-archive', + // CI concepts (#13), tiered so the cheap question stays cheap: ci-runs and + // ci-run-view answer "did it pass" and "which job" without touching a log, + // ci-failures fetches exactly one job's log and returns a bounded extract, + // and ci-run-log is the deliberate raw-window escape hatch. Only the last two + // ever read log bytes. + 'ci-runs', 'ci-run-view', 'ci-failures', 'ci-run-log', ] as const; // ============================================================================= @@ -125,7 +143,16 @@ function getProviderPresets(): Record> { if (_providerPresets) return _providerPresets; _providerPresets = { github: getDefaultCommands(), - gitlab: buildPresetFromScripts('gitlab', ['team-activity', 'on-it-timestamps']), + // The ci-* concepts are DISABLED for gitlab, not merely unimplemented. A + // concept with no script falls through to the github default, so leaving + // them out would make a GitLab repo silently run `gh run list` against + // whatever GitHub remote gh happens to resolve — the silent-fallthrough + // class #1455 closed. Issue #13 asks for gitlab to "degrade loudly, not + // silently"; this is what makes it loud. + gitlab: buildPresetFromScripts('gitlab', [ + 'team-activity', 'on-it-timestamps', + 'ci-runs', 'ci-run-view', 'ci-failures', 'ci-run-log', + ]), // pr-search and pr-diff were disabled here until #12 shipped gitea scripts // for them. team-activity and on-it-timestamps stay disabled and are not // coming: both are `gh api graphql` pass-throughs and Forgejo has no @@ -136,7 +163,11 @@ function getProviderPresets(): Record> { // concept of its own, and without this it silently falls through to the // github default (`gh pr create`) instead of failing loudly. That's the // exact silent-fallthrough bug class #1455 closes. - linear: buildPresetFromScripts('linear', ['team-activity', 'on-it-timestamps', 'pr-create']), + linear: buildPresetFromScripts('linear', [ + 'team-activity', 'on-it-timestamps', 'pr-create', + // Linear has no CI of its own, and the same silent fallthrough applies. + 'ci-runs', 'ci-run-view', 'ci-failures', 'ci-run-log', + ]), }; return _providerPresets; } @@ -392,7 +423,7 @@ export async function executeForgeCommand( const { stdout } = await execAsync(command, { cwd: options?.cwd, env: { ...process.env, ...forgeEnv, ...env }, - timeout: 30_000, + timeout: options?.timeoutMs ?? DEFAULT_TIMEOUT_MS, maxBuffer: options?.maxBuffer ?? DEFAULT_MAX_BUFFER, }); @@ -403,6 +434,97 @@ export async function executeForgeCommand( } } +/** Outcome of executeForgeCommandDetailed — every failure mode kept distinct. */ +export interface ForgeCommandResult { + /** True when the command exited 0. */ + ok: boolean; + /** Parsed stdout (JSON, or the raw string when `raw` is set). Null if unparseable or empty. */ + data: unknown | null; + /** Raw stdout, kept even on failure — the ci-* concepts print their error envelope there. */ + stdout: string; + stderr: string; + /** Process exit code, or null when the process was killed by a signal. */ + exitCode: number | null; + /** True when the command was killed for exceeding the timeout. */ + timedOut: boolean; + /** True when the concept has no command at all (disabled, or no provider script). */ + unavailable: boolean; + durationMs: number; +} + +/** + * Execute a forge concept command and report HOW it went, not merely whether. + * + * `executeForgeCommand` returns `null` for every failure mode: a timeout, a + * non-zero exit, unparseable output and a disabled concept are one value. That + * ambiguity has now cost this project several rounds — #12 shipped a fix for + * `pr-exists` returning a null that porch read as "no PR exists", and #17 and + * #8 both turned a stalled call into a generic failure with nothing naming the + * stall. A caller that needs to tell those apart uses this instead. + * + * Note in particular that `stdout` is returned even when `ok` is false. The CI + * concepts print a structured error envelope on stdout precisely so that the + * class of failure survives; discarding stdout on a non-zero exit would throw + * it away at the last step. + * + * Additive: `executeForgeCommand` is unchanged and no existing caller moves. + */ +export async function executeForgeCommandDetailed( + concept: string, + env?: Record, + options?: ForgeCommandOptions, +): Promise { + const started = Date.now(); + const forgeConfig = resolveForgeConfig(options); + const command = getForgeCommand(concept, forgeConfig); + + if (command === null) { + return { + ok: false, data: null, stdout: '', stderr: '', + exitCode: null, timedOut: false, unavailable: true, + durationMs: Date.now() - started, + }; + } + + const timeout = options?.timeoutMs ?? DEFAULT_TIMEOUT_MS; + const forgeEnv = buildForgeEnv(forgeConfig); + + try { + const { stdout, stderr } = await execAsync(command, { + cwd: options?.cwd, + env: { ...process.env, ...forgeEnv, ...env }, + timeout, + maxBuffer: options?.maxBuffer ?? DEFAULT_MAX_BUFFER, + }); + return { + ok: true, data: parseOutput(stdout, options?.raw), + stdout: String(stdout), stderr: String(stderr), + exitCode: 0, timedOut: false, unavailable: false, + durationMs: Date.now() - started, + }; + } catch (err: unknown) { + logDebug(concept, err); + const e = err as { code?: number | string; killed?: boolean; signal?: string; stdout?: string; stderr?: string }; + // `killed` plus a signal is how Node reports the timeout it enforced — + // verified during #12 against a command whose grandchild held the stdout + // pipe. An exit code alone cannot be read as a timeout: a killed process + // can still exit with a status, and a script that times out INTERNALLY (the + // shell watchdog in scripts/forge/_timeout.sh) exits non-zero with its own + // timeout envelope on stdout, which is why stdout is preserved below. + const timedOut = e.killed === true && typeof e.signal === 'string'; + return { + ok: false, + data: e.stdout ? parseOutput(e.stdout, options?.raw) : null, + stdout: e.stdout ?? '', + stderr: e.stderr ?? (err instanceof Error ? err.message : String(err)), + exitCode: typeof e.code === 'number' ? e.code : null, + timedOut, + unavailable: false, + durationMs: Date.now() - started, + }; + } +} + /** * Execute a forge concept command synchronously. * @@ -428,7 +550,7 @@ export function executeForgeCommandSync( cwd: options?.cwd, env: { ...process.env, ...forgeEnv, ...env }, encoding: 'utf-8', - timeout: 30_000, + timeout: options?.timeoutMs ?? DEFAULT_TIMEOUT_MS, maxBuffer: options?.maxBuffer ?? DEFAULT_MAX_BUFFER, stdio: ['pipe', 'pipe', 'pipe'], }); From d4605f6b2664640150f494f6351c618fdc1092e2 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:25:58 -0600 Subject: [PATCH 07/30] [PIR #13] feat(forge): the four CI concepts for GitHub MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Contrary to issue #13, `gh run view --log-failed` does NOT narrow to the failing step: on run 32515040122 it returned 2528 lines / 293 KB with every line tagged 'UNKNOWN STEP'. It selects the failing JOB. So these use `gh api repos/{owner}/{repo}/actions/jobs/{id}/logs` instead — one job, no invented step column, and the same shape Forgejo 16 serves, so both providers share one cache and one extractor. The failing step NAME comes from `gh run view --json jobs`, which is structured and reliable. Measured on that run: 293 KB of log becomes a 1.2 KB response carrying the assertion, the test name, the step name and the line range. Co-Authored-By: Claude Opus 5 --- packages/codev/scripts/forge/github/_lib.sh | 55 ++++++ .../codev/scripts/forge/github/ci-failures.sh | 158 ++++++++++++++++++ .../codev/scripts/forge/github/ci-run-log.sh | 95 +++++++++++ .../codev/scripts/forge/github/ci-run-view.sh | 67 ++++++++ .../codev/scripts/forge/github/ci-runs.sh | 73 ++++++++ 5 files changed, 448 insertions(+) create mode 100755 packages/codev/scripts/forge/github/_lib.sh create mode 100755 packages/codev/scripts/forge/github/ci-failures.sh create mode 100755 packages/codev/scripts/forge/github/ci-run-log.sh create mode 100755 packages/codev/scripts/forge/github/ci-run-view.sh create mode 100755 packages/codev/scripts/forge/github/ci-runs.sh diff --git a/packages/codev/scripts/forge/github/_lib.sh b/packages/codev/scripts/forge/github/_lib.sh new file mode 100755 index 000000000..679eab799 --- /dev/null +++ b/packages/codev/scripts/forge/github/_lib.sh @@ -0,0 +1,55 @@ +# Shared helpers for the GitHub CI concept scripts (#13). SOURCED, not executed. +# +# POSIX sh, no shebang, leading underscore so forge never registers it as a +# concept. The older github concepts are one-line `exec gh …` scripts and do not +# use this; the CI ones need a run lookup, a job-log fetch and a cache, and none +# of that should be written three times. + +. "$(dirname "$0")/../_ci-lib.sh" +. "$(dirname "$0")/../_ci-extract.sh" + +# The gh `run view` projection every CI concept needs. One call, ~1.3s measured, +# and it already carries per-step conclusions — so the failing STEP name comes +# from structured JSON rather than from parsing a log. +GH_RUN_FIELDS='databaseId,number,displayTitle,workflowName,name,status,conclusion,headBranch,headSha,event,url,createdAt,jobs' + +# Fetch a run as JSON. Echoes the JSON; returns 124 on timeout, 1 otherwise. +gh_run_json() { + ci_tool gh run view "$1" --json "$GH_RUN_FIELDS" +} + +# The jobs of a run that actually failed, as a JSON array, in run order. +# +# `cancelled` is NOT in this list. A cancelled job has no failure to diagnose, +# and treating it as one produces an extract of whatever the runner happened to +# be printing when the cancel landed — an arbitrary slice of log presented as a +# cause, which is the thing this concept exists to avoid. +GH_FAILED_JOBS_JQ='[.jobs[] | select(.conclusion == "failure" or .conclusion == "timed_out" or .conclusion == "startup_failure")]' + +# Fetch one job log, via the cache when the job is terminal. +# +# gh_job_log +# +# Uses `gh api repos/{owner}/{repo}/actions/jobs/{id}/logs` — NOT +# `gh run view --log-failed`. The reason is measured, and it contradicts the +# issue that asked for this work: --log-failed returned 2528 lines / 293 KB for +# one failing job on run 32515040122, every line tagged "UNKNOWN STEP", because +# it selects the failing JOB and cannot always attribute lines to steps. The +# per-job endpoint returns the same bytes without the invented step column, for +# exactly the job asked for, and mirrors the shape Forgejo 16 serves at +# `actions/jobs/{id}/logs` — so both providers share one cache and one +# extractor. +gh_job_log() { + _job="$1"; _status="$2"; _dest="$3" + _cache=$(ci_cache_path github "$(ci_repo_slug)" "$_job") + if ci_cache_read "$_cache" > "$_dest" 2>/dev/null && [ -s "$_dest" ]; then + CI_LOG_FROM_CACHE=true + return 0 + fi + CI_LOG_FROM_CACHE=false + _rc=0 + ci_tool gh api "repos/{owner}/{repo}/actions/jobs/${_job}/logs" > "$_dest" || _rc=$? + [ "$_rc" -eq 0 ] || return "$_rc" + ci_cache_write "$_cache" "$_dest" "$_status" + return 0 +} diff --git a/packages/codev/scripts/forge/github/ci-failures.sh b/packages/codev/scripts/forge/github/ci-failures.sh new file mode 100755 index 000000000..d203e2d7e --- /dev/null +++ b/packages/codev/scripts/forge/github/ci-failures.sh @@ -0,0 +1,158 @@ +#!/bin/sh +# Forge concept: ci-failures (GitHub via gh CLI) +# forge-executable: gh +# Input: CODEV_CI_RUN_ID (required — the `id` from ci-runs) +# CODEV_CI_JOB_ID (optional — pin one job instead of the first failing) +# Output: {ok, provider, runId, runStatus, runConclusion, jobsFailed, extracted, +# failures: [{jobId, jobName, stepName, stepNumber, matchedBy, text, +# from, to, logLines, returnedLines, truncated}], +# otherFailingJobs, cached} +# +# The concept that matters. Two calls: the run (structured, gives the failing +# job and the failing STEP name), then that ONE job's log, which is extracted +# down to the assertion and capped. The other failing jobs are listed by name +# and id and deliberately not fetched — later failures are usually downstream of +# the first, and fetching them is how a bounded answer becomes a log dump again. +# +# WHEN EXTRACTION FAILS IT SAYS SO AND HANDS OVER +# +# {"extracted": false, "reason": "no recognized failure pattern", +# "failures": [{"jobId": …, "jobName": …, "logLines": 1247}], +# "next": "ci-run-log CODEV_CI_RUN_ID=… CODEV_CI_JOB_ID=… CODEV_CI_LOG_TAIL=80"} +# +# It never falls back to "here are the last 50 lines". A builder handed 50 +# arbitrary lines treats them as the diagnosis and reasons from noise; a builder +# told extraction failed reads the log with a targeted ci-run-log call, which is +# both correct and cheaper. The response carries the job id it would need, so +# the refusal is a handoff rather than a dead end. +set -e +. "$(dirname "$0")/_lib.sh" + +CONCEPT=ci-failures + +if [ -z "$CODEV_CI_RUN_ID" ]; then + jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' + echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 + exit 2 +fi + +rc=0 +RUN=$(gh_run_json "$CODEV_CI_RUN_ID") || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "gh run view ${CODEV_CI_RUN_ID}" "$CI_TIMEOUT" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" not-found "run ${CODEV_CI_RUN_ID} could not be read (gh exit ${rc}); pass the \`id\` from ci-runs, not the run \`number\`" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" '{runId: $r}')" + exit 1 +fi + +FAILED=$(printf '%s' "$RUN" | jq -c "$GH_FAILED_JOBS_JQ") +RUN_STATUS=$(printf '%s' "$RUN" | jq -r '.status // "unknown"') +RUN_CONCLUSION=$(printf '%s' "$RUN" | jq -r '.conclusion // "null"') + +# Pin a job if asked. A job id that is not in this run is an error, not an empty +# answer — silently returning "nothing failed" for a mistyped id is the exact +# shape of wrong answer this concept exists to remove. +if [ -n "$CODEV_CI_JOB_ID" ]; then + TARGET=$(printf '%s' "$RUN" | jq -c --argjson j "$CODEV_CI_JOB_ID" 'first(.jobs[] | select(.databaseId == $j)) // empty') + if [ -z "$TARGET" ]; then + ci_fail "$CONCEPT" not-found "job ${CODEV_CI_JOB_ID} is not part of run ${CODEV_CI_RUN_ID}" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --arg j "$CODEV_CI_JOB_ID" '{runId: $r, jobId: $j}')" + exit 1 + fi +else + TARGET=$(printf '%s' "$FAILED" | jq -c '.[0] // empty') +fi + +JOBS_FAILED=$(printf '%s' "$FAILED" | jq 'length') + +# No failing job. Say which of the two reasons it is — a green run and a run +# still going are not the same answer, and neither is "the server could not tell +# me", which is an error envelope rather than this branch. +if [ -z "$TARGET" ]; then + printf '%s' "$RUN" | jq -c --arg concept "$CONCEPT" '{ + ok: true, provider: "github", runId: .databaseId, + runStatus: .status, runConclusion: .conclusion, + jobsFailed: 0, extracted: false, + reason: (if .status != "completed" then "run has not finished" else "no job in this run failed" end), + failures: [] + }' + exit 0 +fi + +JOB_ID=$(printf '%s' "$TARGET" | jq -r '.databaseId') +JOB_NAME=$(printf '%s' "$TARGET" | jq -r '.name') +JOB_STATE=$(printf '%s' "$TARGET" | jq -r '.conclusion // .status') +STEP=$(printf '%s' "$TARGET" | jq -c 'first(.steps[]? | select(.conclusion == "failure" or .conclusion == "timed_out")) // null') + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +rc=0 +gh_job_log "$JOB_ID" "$JOB_STATE" "$TMP/raw.log" || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "gh api repos/{owner}/{repo}/actions/jobs/${JOB_ID}/logs" "$CI_TIMEOUT" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "could not read the log for job ${JOB_ID} (${JOB_NAME}); gh exit ${rc}" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --argjson j "$JOB_ID" --arg n "$JOB_NAME" '{runId: $r, jobId: $j, jobName: $n}')" + exit 1 +fi + +ci_clean_log < "$TMP/raw.log" > "$TMP/clean.log" +# awk NR, not `wc -l`: a log with no trailing newline makes wc undercount by +# one, and logLines would then disagree with the from/to the extractor reports. +LOG_LINES=$(awk 'END {print NR}' "$TMP/clean.log") + +OTHERS=$(printf '%s' "$FAILED" | jq -c --argjson j "$JOB_ID" '[.[] | select(.databaseId != $j) | {id: .databaseId, name: .name}]') + +if ci_extract "$TMP/clean.log" > "$TMP/extract.txt" 2>/dev/null && [ -s "$TMP/extract.txt" ]; then + MATCHED=$(head -1 "$TMP/extract.txt" | cut -f1) + FROM=$(head -1 "$TMP/extract.txt" | cut -f2) + TO=$(head -1 "$TMP/extract.txt" | cut -f3) + tail -n +2 "$TMP/extract.txt" > "$TMP/text.txt" + CAP=$CI_MAX_STEP_BYTES + [ "$CAP" -gt "$CI_MAX_RESPONSE_BYTES" ] && CAP=$CI_MAX_RESPONSE_BYTES + META=$(ci_cap_file "$TMP/text.txt" "$CAP" "$TMP/capped.txt") + RETURNED_LINES=${META% *} + TRUNCATED=${META#* } + TEXT=$(jq -R -s -c . < "$TMP/capped.txt") + jq -cn \ + --arg run "$CODEV_CI_RUN_ID" --arg rs "$RUN_STATUS" --arg rc "$RUN_CONCLUSION" \ + --argjson job "$JOB_ID" --arg jobName "$JOB_NAME" --argjson step "$STEP" \ + --arg matched "$MATCHED" --argjson text "$TEXT" \ + --argjson from "$FROM" --argjson to "$TO" \ + --argjson logLines "$LOG_LINES" --argjson returned "$RETURNED_LINES" \ + --argjson truncated "$TRUNCATED" --argjson jobsFailed "$JOBS_FAILED" \ + --argjson others "$OTHERS" --argjson cached "$CI_LOG_FROM_CACHE" \ + '{ok: true, provider: "github", runId: ($run | tonumber? // $run), + runStatus: $rs, runConclusion: (if $rc == "null" then null else $rc end), + jobsFailed: $jobsFailed, extracted: true, + failures: [{ + jobId: $job, jobName: $jobName, + stepName: ($step.name // null), stepNumber: ($step.number // null), + matchedBy: $matched, text: $text, + from: $from, to: $to, + logLines: $logLines, returnedLines: $returned, truncated: $truncated + }], + otherFailingJobs: $others, cached: $cached}' + exit 0 +fi + +# Rung 6: nothing recognisable. Hand over the parameters for the follow-up +# rather than inventing a diagnosis. +jq -cn \ + --arg run "$CODEV_CI_RUN_ID" --arg rs "$RUN_STATUS" --arg rc "$RUN_CONCLUSION" \ + --argjson job "$JOB_ID" --arg jobName "$JOB_NAME" \ + --argjson logLines "$LOG_LINES" --argjson jobsFailed "$JOBS_FAILED" \ + --argjson others "$OTHERS" --argjson cached "$CI_LOG_FROM_CACHE" \ + '{ok: true, provider: "github", runId: ($run | tonumber? // $run), + runStatus: $rs, runConclusion: (if $rc == "null" then null else $rc end), + jobsFailed: $jobsFailed, extracted: false, + reason: "no recognized failure pattern", + failures: [{jobId: $job, jobName: $jobName, logLines: $logLines}], + otherFailingJobs: $others, cached: $cached, + next: ("ci-run-log CODEV_CI_RUN_ID=" + $run + " CODEV_CI_JOB_ID=" + ($job|tostring) + " CODEV_CI_LOG_TAIL=80")}' diff --git a/packages/codev/scripts/forge/github/ci-run-log.sh b/packages/codev/scripts/forge/github/ci-run-log.sh new file mode 100755 index 000000000..b037c3967 --- /dev/null +++ b/packages/codev/scripts/forge/github/ci-run-log.sh @@ -0,0 +1,95 @@ +#!/bin/sh +# Forge concept: ci-run-log (GitHub via gh CLI) +# forge-executable: gh +# Input: CODEV_CI_RUN_ID (required) +# CODEV_CI_JOB_ID (optional — defaults to the first failing job, or the +# only job if the run has exactly one) +# exactly ONE window: +# CODEV_CI_LOG_TAIL=N last N lines +# CODEV_CI_LOG_HEAD=N first N lines (setup, install, config) +# CODEV_CI_LOG_GREP= matching lines, with CODEV_CI_LOG_CONTEXT +# (default 3) lines either side +# Output: {ok, provider, runId, jobId, jobName, window, logLines, returnedLines, +# from, to, truncated, lines: [...], matchLines?: [...]} +# +# The escape hatch, and deliberately a SEPARATE concept rather than a flag on +# ci-failures: a window parameter on the main call gets passed by habit, and +# then every status question drags a log again — which is the thing this issue +# exists to prevent. Reaching for this should be a deliberate act. +# +# There is no default window. Zero windows and two windows are both exit 2 with +# a named message, because a default is how "deliberate" decays into "always". +# +# This is remote head/tail/grep -C over a log, and nothing more. The value codev +# adds is doing it against two forges with one contract and a bounded response, +# not inventing log analysis. +set -e +. "$(dirname "$0")/_lib.sh" + +CONCEPT=ci-run-log + +if [ -z "$CODEV_CI_RUN_ID" ]; then + jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' + echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 + exit 2 +fi + +# Window first: a malformed request should not cost an API call. +ci_window_parse "$CONCEPT" + +rc=0 +RUN=$(gh_run_json "$CODEV_CI_RUN_ID") || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "gh run view ${CODEV_CI_RUN_ID}" "$CI_TIMEOUT" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" not-found "run ${CODEV_CI_RUN_ID} could not be read (gh exit ${rc}); pass the \`id\` from ci-runs, not the run \`number\`" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" '{runId: $r}')" + exit 1 +fi + +if [ -n "$CODEV_CI_JOB_ID" ]; then + TARGET=$(printf '%s' "$RUN" | jq -c --argjson j "$CODEV_CI_JOB_ID" 'first(.jobs[] | select(.databaseId == $j)) // empty') + if [ -z "$TARGET" ]; then + ci_fail "$CONCEPT" not-found "job ${CODEV_CI_JOB_ID} is not part of run ${CODEV_CI_RUN_ID}" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --arg j "$CODEV_CI_JOB_ID" '{runId: $r, jobId: $j}')" + exit 1 + fi +else + TARGET=$(printf '%s' "$RUN" | jq -c "${GH_FAILED_JOBS_JQ} | .[0] // empty") + # No failing job: fall back to the only job, if there is exactly one. With + # several passing jobs there is no defensible default, and picking one would + # hand back a log the caller did not ask for. + if [ -z "$TARGET" ]; then + TARGET=$(printf '%s' "$RUN" | jq -c 'if (.jobs | length) == 1 then .jobs[0] else empty end') + fi + if [ -z "$TARGET" ]; then + NAMES=$(printf '%s' "$RUN" | jq -c '[.jobs[] | {id: .databaseId, name: .name}]') + ci_fail "$CONCEPT" bad-input "run ${CODEV_CI_RUN_ID} has no failing job and more than one job; set CODEV_CI_JOB_ID" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --argjson jobs "$NAMES" '{runId: $r, jobs: $jobs}')" + exit 2 + fi +fi + +JOB_ID=$(printf '%s' "$TARGET" | jq -r '.databaseId') +JOB_NAME=$(printf '%s' "$TARGET" | jq -r '.name') +JOB_STATE=$(printf '%s' "$TARGET" | jq -r '.conclusion // .status') + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +rc=0 +gh_job_log "$JOB_ID" "$JOB_STATE" "$TMP/raw.log" || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "gh api repos/{owner}/{repo}/actions/jobs/${JOB_ID}/logs" "$CI_TIMEOUT" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "could not read the log for job ${JOB_ID} (${JOB_NAME}); gh exit ${rc}" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --argjson j "$JOB_ID" --arg n "$JOB_NAME" '{runId: $r, jobId: $j, jobName: $n}')" + exit 1 +fi + +ci_clean_log < "$TMP/raw.log" > "$TMP/clean.log" +ci_window_emit "$CONCEPT" "$TMP" github "$CODEV_CI_RUN_ID" "$JOB_ID" "$JOB_NAME" "$CI_LOG_FROM_CACHE" diff --git a/packages/codev/scripts/forge/github/ci-run-view.sh b/packages/codev/scripts/forge/github/ci-run-view.sh new file mode 100755 index 000000000..73f9704d9 --- /dev/null +++ b/packages/codev/scripts/forge/github/ci-run-view.sh @@ -0,0 +1,67 @@ +#!/bin/sh +# Forge concept: ci-run-view (GitHub via gh CLI) +# forge-executable: gh +# Input: CODEV_CI_RUN_ID (required — the `id` field from ci-runs, not `number`) +# Output: {ok, provider, run: {...}, jobs: [{id, name, status, conclusion, +# startedAt, completedAt, failedSteps: [{name, number, conclusion}]}], +# jobSource} +# +# Still no log bytes. This answers "is it still running, and which job is +# pending" and "which step failed" — the second of which is normally enough to +# know whether a failure is yours, and costs one ~1.3s call. +# +# `failedSteps` comes from gh's structured per-step conclusions, NOT from +# parsing a log. That matters: `gh run view --log-failed` labels its lines +# "UNKNOWN STEP" whenever its filename-to-step mapping misses, which on the run +# this was built against was every line of all 2528. +set -e +. "$(dirname "$0")/_lib.sh" + +CONCEPT=ci-run-view + +if [ -z "$CODEV_CI_RUN_ID" ]; then + jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' + echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 + exit 2 +fi + +rc=0 +OUT=$(gh_run_json "$CODEV_CI_RUN_ID") || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "gh run view ${CODEV_CI_RUN_ID}" "$CI_TIMEOUT" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" not-found "run ${CODEV_CI_RUN_ID} could not be read (gh exit ${rc}); pass the \`id\` from ci-runs, not the run \`number\`" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" '{runId: $r}')" + exit 1 +fi + +printf '%s' "$OUT" | jq -c '{ + ok: true, + provider: "github", + jobSource: "run-view", + run: { + id: .databaseId, + number: .number, + title: .displayTitle, + workflow: .workflowName, + status: .status, + conclusion: .conclusion, + branch: .headBranch, + sha: .headSha, + event: .event, + url: .url, + createdAt: .createdAt + }, + jobs: [ .jobs[] | { + id: .databaseId, + name: .name, + status: .status, + conclusion: .conclusion, + startedAt: .startedAt, + completedAt: .completedAt, + failedSteps: [ .steps[]? | select(.conclusion == "failure" or .conclusion == "timed_out") + | {name: .name, number: .number, conclusion: .conclusion} ] + } ] +}' diff --git a/packages/codev/scripts/forge/github/ci-runs.sh b/packages/codev/scripts/forge/github/ci-runs.sh new file mode 100755 index 000000000..7a3c8bb89 --- /dev/null +++ b/packages/codev/scripts/forge/github/ci-runs.sh @@ -0,0 +1,73 @@ +#!/bin/sh +# Forge concept: ci-runs (GitHub via gh CLI) +# forge-executable: gh +# Input: CODEV_BRANCH_NAME (optional) — filter to one branch +# CODEV_CI_STATUS (optional) — success|failure|pending|queued|in_progress|skipped|canceled +# CODEV_CI_WORKFLOW (optional) — workflow file name or display name +# CODEV_CI_LIMIT (optional, default 20) +# Output: {ok, provider, runs: [{id, number, name, workflow, status, conclusion, +# branch, sha, event, url, createdAt}], truncated} +# +# The cheap question, and the whole point of tiering these concepts: "did my +# push pass" must never drag a log through a context window. One `gh run list` +# call, ~0.7s measured, no log bytes at any point. +# +# It also answers "is this mine or is it flaky", which is why CODEV_CI_WORKFLOW +# exists: the same workflow across other commits is a run-history question, and +# on 2026-08-21 two wrong conclusions were drawn about CI state that were both +# answerable from run history and neither checked, because asking was awkward. +# +# `id` is what ci-run-view / ci-failures / ci-run-log take. `number` is the +# human run number. On GitHub they differ; on Forgejo they differ AND both are +# valid inputs to the same route, so the concepts never guess which one they +# were handed. Pass `id`. +set -e +. "$(dirname "$0")/_lib.sh" + +CONCEPT=ci-runs + +ci_check_status "$CONCEPT" "$CODEV_CI_STATUS" + +LIMIT=${CODEV_CI_LIMIT:-$CI_LIMIT_DEFAULT} +case "$LIMIT" in ''|*[!0-9]*|0) LIMIT=$CI_LIMIT_DEFAULT ;; esac + +# One more than asked for, so that "there are more runs than this" can be +# reported instead of guessed. gh has no hasMore of its own, and a list that was +# cut by a limit is indistinguishable from a complete one once printed. +set -- run list --limit "$((LIMIT + 1))" --json databaseId,number,name,workflowName,status,conclusion,headBranch,headSha,event,url,createdAt +[ -n "$CODEV_BRANCH_NAME" ] && set -- "$@" --branch "$CODEV_BRANCH_NAME" +[ -n "$CODEV_CI_WORKFLOW" ] && set -- "$@" --workflow "$CODEV_CI_WORKFLOW" +if [ -n "$CODEV_CI_STATUS" ]; then + set -- "$@" --status "$(ci_status_for github "$CODEV_CI_STATUS")" +fi + +rc=0 +OUT=$(ci_tool gh "$@") || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "gh run list" "$CI_TIMEOUT" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "gh run list failed (exit ${rc}); see stderr above" + exit 1 +fi + +printf '%s' "$OUT" | jq -c --argjson limit "$LIMIT" '{ + ok: true, + provider: "github", + runs: [ .[] | { + id: .databaseId, + number: .number, + name: .name, + workflow: .workflowName, + status: .status, + conclusion: .conclusion, + branch: .headBranch, + sha: .headSha, + event: .event, + url: .url, + createdAt: .createdAt + } ] | .[0:$limit], + truncated: (length > $limit), + note: null +}' From 1fd18d491bbcf7db3422d0955c6eaa0204aa7270 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:26:12 -0600 Subject: [PATCH 08/30] [PIR #13] feat(forge): the four CI concepts for Gitea/Forgejo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured against Forgejo 15.0.2 (git.pseudoseed.com) and 16.0.0-dev (codeberg.org), because the issue's premises did not survive contact: - `tea actions runs view` and `tea actions runs logs` both 404 against 15.0.2 (they call /actions/runs/{id}/jobs and /actions/jobs/{id}/logs, added in Forgejo 16.0), and `tea actions runs list --output json` returns empty strings for workflow, branch, started and duration. So these go through `tea api`, as #12 established. - `limit` is IGNORED unless `page` is also sent: actions/runs?limit=3 returned all 6922 runs. Every list call here sends page=. - `branch=` and `event=` are silently ignored, and a pull_request run records head_branch as '#3847' — the PR number. So CODEV_BRANCH_NAME is resolved to its PR ref with #12's base/head lookup before filtering, client-side. - Run `id` and `index_in_repo` are two id spaces and BOTH resolve on /actions/runs/{x}, to different real runs. CODEV_CI_RUN_ID is always `id`. On a server without the log API, ci-failures and ci-run-log return the unsupported-server envelope naming the version found and the version needed, and carrying the failing job names they could still determine — never an empty failures array, which would say 'your CI is fine' when the truth is 'I cannot see your CI at all'. ci-run-view keeps working there by recovering jobs from actions/tasks (jobSource: tasks-scan), reporting taskId with a null id because a task id is not a job id and the log API does not accept one. Co-Authored-By: Claude Opus 5 --- packages/codev/scripts/forge/gitea/_ci.sh | 229 ++++++++++++++++++ .../codev/scripts/forge/gitea/ci-failures.sh | 167 +++++++++++++ .../codev/scripts/forge/gitea/ci-run-log.sh | 105 ++++++++ .../codev/scripts/forge/gitea/ci-run-view.sh | 76 ++++++ packages/codev/scripts/forge/gitea/ci-runs.sh | 105 ++++++++ 5 files changed, 682 insertions(+) create mode 100755 packages/codev/scripts/forge/gitea/_ci.sh create mode 100755 packages/codev/scripts/forge/gitea/ci-failures.sh create mode 100755 packages/codev/scripts/forge/gitea/ci-run-log.sh create mode 100755 packages/codev/scripts/forge/gitea/ci-run-view.sh create mode 100755 packages/codev/scripts/forge/gitea/ci-runs.sh diff --git a/packages/codev/scripts/forge/gitea/_ci.sh b/packages/codev/scripts/forge/gitea/_ci.sh new file mode 100755 index 000000000..1dee791ed --- /dev/null +++ b/packages/codev/scripts/forge/gitea/_ci.sh @@ -0,0 +1,229 @@ +# Shared helpers for the Gitea/Forgejo CI concept scripts (#13). SOURCED. +# +# Sources gitea/_lib.sh for gitea_repo / gitea_api / gitea_api_error (#12) and +# the shared _ci-lib.sh / _ci-extract.sh. POSIX sh, leading underscore, never a +# concept. +# +# WHAT FORGEJO ACTUALLY OFFERS, MEASURED 2026-08-21 +# +# The issue proposed driving this from `tea actions runs list|view|logs`. Those +# subcommands exist on tea 0.14.2, and two of the three do not work against +# Forgejo 15.0.2 (git.pseudoseed.com): +# +# tea actions runs view 11130 → 404 on /api/v1/repos/…/actions/runs/11130/jobs +# tea actions runs logs 6881 --job 40084 +# → 404 on /api/v1/repos/…/actions/jobs/40084/logs +# +# Both routes were added in **Forgejo 16.0** (released 2026-07-16). And +# `tea actions runs list --output json` is lossy even where it works: workflow, +# branch, started and duration all come back as empty strings. So these scripts +# go through `tea api`, as #12 established for the PR concepts. +# +# What 15.0.2 does have: +# +# GET actions/runs?page=1&limit=N[&status=…] runs; 0.3s; ~17.8 KB PER RUN +# (each embeds a full repo object) +# GET actions/runs/{id} one run +# GET actions/tasks?page=1&limit=N[&status=…] JOBS, GitHub-shaped, 482 B each +# +# Three query-parameter facts, all measured, all footguns: +# +# * `limit` is IGNORED unless `page` is also present. `actions/runs?limit=3` +# returned all 6922 runs. Every call here passes page=1. +# * `status=` filters server-side and works. +# * `branch=` and `event=` are silently IGNORED. Branch filtering is +# client-side, and it is not a string compare — see below. +# +# THE BRANCH IS NOT THE BRANCH +# +# For `pull_request` runs Forgejo reports head_branch / prettyref as `#3847` — +# the PR number. Only push and schedule runs carry a real branch name. On the +# reference repo the first 100 tasks were: #3869 ×32, #3865 ×10, main ×7, +# v1.0.230 ×1. So filtering by `builder/pir-13` matches NOTHING on a repo that +# runs CI on pull requests, unless the branch is first resolved to its PR +# number — which is one base/head lookup, the same primitive #12 built. +# +# TWO ID SPACES +# +# Run `id` 11130 has `index_in_repo` 6881, and `actions/runs/6881` resolves to a +# DIFFERENT, real run. The web URL shows the index. So CODEV_CI_RUN_ID is always +# the `id` field from ci-runs, never the number, and these scripts never guess +# which one they were handed. + +. "$(dirname "$0")/_lib.sh" +. "$(dirname "$0")/../_ci-lib.sh" +. "$(dirname "$0")/../_ci-extract.sh" + +# The Forgejo release that first exposed Actions jobs and logs over the API. +GITEA_CI_LOG_MIN_VERSION="16.0" + +# The server version string, for error messages. Best effort: a server that will +# not answer `version` still gets a usable message, just a vaguer one. +gitea_server_version() { + _v=$(gitea_api "version" 2>/dev/null) || _v= + printf '%s' "$_v" | jq -r '.version // "unknown"' 2>/dev/null || printf 'unknown' +} + +# The unsupported-server envelope. +# +# This exists so that an old Forgejo can never be mistaken for a run with no +# failures. Those two are the same observation to a caller that only sees an +# empty array, and they are opposite facts: one means "your CI is fine", the +# other means "I cannot see your CI at all". +gitea_ci_unsupported() { + _concept="$1"; _route="$2"; _extra="$3" + [ -n "$_extra" ] || _extra='{}' + _ver=$(gitea_server_version) + ci_fail "$_concept" unsupported-server \ + "this Forgejo has no Actions ${_route} API (added in Forgejo ${GITEA_CI_LOG_MIN_VERSION}); server reports ${_ver}. ci-runs and ci-run-view still work here." \ + "$(printf '%s' "$_extra" | jq -c --arg v "$_ver" --arg n "$GITEA_CI_LOG_MIN_VERSION" '. + {serverVersion: $v, needs: (">=" + $n)}')" + return 0 +} + +# Fetch a JSON response INTO A FILE and classify it. +# +# gitea_ci_fetch +# 0 = a JSON object or array is in +# 44 = the endpoint said 404 (on Forgejo 15 that is how a missing Actions +# API announces itself) +# 45 = some other error body, left in for the caller to quote +# 124 = timed out (gitea_api already said so on stderr) +# +# A file rather than a shell variable because a single page of `actions/runs` is +# ~892 KB — Forgejo embeds the entire repository object in every run — and the +# jq reduction should read it from disk rather than after it has been copied +# through two shell strings. +gitea_ci_fetch() { + _url="$1"; _dest="$2" + _rc=0 + gitea_api "$_url" > "$_dest" || _rc=$? + [ "$_rc" -eq 0 ] || return "$_rc" + # Classify from the FILE, not from a prefix of it. Reading `head -c 400` and + # handing that to gitea_api_error looked equivalent and is not: a truncated + # JSON prefix does not parse, so every large healthy response classified as an + # error. Found on the first live call against Forgejo. + if jq -e 'type == "array"' "$_dest" >/dev/null 2>&1; then + return 0 + fi + if jq -e 'type == "object"' "$_dest" >/dev/null 2>&1; then + _msg=$(jq -r '.message // empty' "$_dest" 2>/dev/null) || _msg= + case "$_msg" in + '') return 0 ;; + *"couldn't be found"*|*'could not be found'*|*'Not found'*|*'not found'*|*'does not exist'*) return 44 ;; + *) return 45 ;; + esac + fi + # Not JSON at all: Forgejo answers an unknown ROUTE with the bare text + # "404 page not found", which is how a missing Actions API announces itself. + case "$(head -c 40 "$_dest")" in + 404*) return 44 ;; + esac + return 45 +} + +# Resolve a branch name to the PR ref Forgejo will have recorded on its runs. +# Echoes `#` when the branch has a PR, nothing when it does not. Returns 1 +# only when the REPO could not be read — a branch with no PR is an answer. +gitea_ci_pr_ref() { + _repo="$1"; _branch="$2" + _base=${CODEV_PR_BASE:-$(gitea_default_branch "$_repo")} || return 1 + _resp=$(gitea_api "repos/${_repo}/pulls/${_base}/${_branch}") || return 1 + case "$(gitea_api_error "$_resp")" in + ok) ;; + *) return 0 ;; + esac + _n=$(printf '%s' "$_resp" | jq -r 'if type == "object" and (.number | type) == "number" then "#\(.number)" else empty end' 2>/dev/null) || _n= + printf '%s' "$_n" +} + +# The jobs of a run, normalised, plus GITEA_JOB_SOURCE describing where they +# came from. Echoes a JSON array; returns 1 on a hard failure. +# +# Forgejo 16 has `actions/runs/{id}/jobs`, which carries the job `id` the log +# API takes. Forgejo 15 has neither, but `actions/tasks` lists the same jobs +# with a `run_number`, so the run object gives us `index_in_repo` and the tasks +# can be filtered to it. That fallback is why `ci-run-view` still answers on +# 15.0.2 instead of going dark with the log concepts. +# +# On the fallback path the identifier is a TASK id, not a job id, and there is +# no way to obtain a job id on a server that does not expose jobs. So `id` is +# null and `taskId` carries what we have. Emitting the task id as `id` would +# hand callers a number that looks usable with ci-run-log and is not. +gitea_ci_jobs() { + _concept="$1"; _repo="$2"; _runid="$3"; _runindex="$4"; _out="$5" + printf 'false' > "${_out}/jobs.truncated" + + _rc=0 + gitea_ci_fetch "repos/${_repo}/actions/runs/${_runid}/jobs" "${_out}/jobs.raw" || _rc=$? + if [ "$_rc" -eq 0 ]; then + printf 'runs-jobs' > "${_out}/jobs.source" + jq -c '[ .[]? | { + id: .id, taskId: .task_id, name: .name, + status: .status, + conclusion: (if (.status == "success" or .status == "failure" or .status == "skipped" or .status == "canceled" or .status == "cancelled") then .status else null end), + startedAt: null, completedAt: null, failedSteps: [] + } ]' "${_out}/jobs.raw" > "${_out}/jobs.json" + return 0 + fi + [ "$_rc" -eq 44 ] || return 1 + + # Forgejo 15: no jobs route. Recover the jobs from `actions/tasks`, which + # carries run_number. Tasks come back newest-first, so a run occupies one + # contiguous block and the walk can stop the moment it has gone PAST that + # block — otherwise every lookup of an older run would cost the full page + # ceiling and report truncation it did not need to. + printf 'tasks-scan' > "${_out}/jobs.source" + _acc='[]' + _page=1 + _found=0 + _passed=0 + while [ "$_page" -le "$CI_TASKS_MAX_PAGES" ]; do + _rc=0 + gitea_ci_fetch "repos/${_repo}/actions/tasks?page=${_page}&limit=${GITEA_PAGE_LIMIT}" "${_out}/tasks.json" || _rc=$? + [ "$_rc" -eq 0 ] || return 1 + _raw=$(jq '.workflow_runs | length' "${_out}/tasks.json") + [ "$_raw" -eq 0 ] && break + _hits=$(jq -c --argjson r "$_runindex" '[ .workflow_runs[]? | select(.run_number == $r) | { + id: null, taskId: .id, name: .name, status: .status, + conclusion: (if (.status == "success" or .status == "failure" or .status == "skipped" or .status == "canceled" or .status == "cancelled") then .status else null end), + startedAt: .run_started_at, completedAt: .updated_at, failedSteps: [] + } ]' "${_out}/tasks.json") + if [ "$(printf '%s' "$_hits" | jq 'length')" -gt 0 ]; then + _acc=$(printf '%s\n%s' "$_acc" "$_hits" | jq -s -c 'add') + _found=1 + fi + _min=$(jq --argjson r "$_runindex" '[.workflow_runs[]?.run_number] | min // ($r + 1)' "${_out}/tasks.json") + if [ "$_min" -lt "$_runindex" ]; then _passed=1; break; fi + [ "$_raw" -lt "$GITEA_PAGE_LIMIT" ] && break + _page=$((_page + 1)) + done + printf '%s' "$_acc" > "${_out}/jobs.json" + # Truncated only when the walk ran out of allowance BEFORE reaching the run. + # Having walked past it and found nothing is a complete answer. + if [ "$_found" -eq 0 ] && [ "$_passed" -eq 0 ] && [ "$_page" -gt "$CI_TASKS_MAX_PAGES" ]; then + printf 'true' > "${_out}/jobs.truncated" + echo "${_concept}: this Forgejo has no jobs API, and run ${_runid} was not reached within ${CI_TASKS_MAX_PAGES} pages of the task list; raise CODEV_CI_TASKS_MAX_PAGES" >&2 + fi + return 0 +} + +# Fetch one job log (Forgejo 16 only), via the cache when the job is terminal. +gitea_ci_job_log() { + _repo="$1"; _job="$2"; _status="$3"; _dest="$4" + _cache=$(ci_cache_path gitea "$_repo" "$_job") + if ci_cache_read "$_cache" > "$_dest" 2>/dev/null && [ -s "$_dest" ]; then + CI_LOG_FROM_CACHE=true + return 0 + fi + CI_LOG_FROM_CACHE=false + _rc=0 + gitea_api "repos/${_repo}/actions/jobs/${_job}/logs" > "$_dest" || _rc=$? + [ "$_rc" -eq 0 ] || return "$_rc" + # `tea api` exits 0 on HTTP errors and prints the body, so a 404 arrives here + # looking like a log. Classify it before anyone treats an error page as one. + if [ ! -s "$_dest" ] || gitea_api_error "$(head -c 200 "$_dest")" | grep -q 'notfound'; then + return 44 + fi + ci_cache_write "$_cache" "$_dest" "$_status" + return 0 +} diff --git a/packages/codev/scripts/forge/gitea/ci-failures.sh b/packages/codev/scripts/forge/gitea/ci-failures.sh new file mode 100755 index 000000000..93fa3aefe --- /dev/null +++ b/packages/codev/scripts/forge/gitea/ci-failures.sh @@ -0,0 +1,167 @@ +#!/bin/sh +# Forge concept: ci-failures (Gitea/Forgejo via tea CLI) +# forge-executable: tea +# Input: CODEV_CI_RUN_ID (required), CODEV_CI_JOB_ID (optional) +# Output: the shared ci-failures envelope; see github/ci-failures.sh +# +# REQUIRES FORGEJO 16.0 OR LATER. The Actions job-log API +# (`actions/jobs/{id}/logs`) landed in Forgejo 16.0, released 2026-07-16; on +# 15.0.2 there is no token-reachable log anywhere — not via `tea actions runs +# logs` (which calls that same route), not via any other API route, and not via +# the web UI route, which is session-only and rejects both an API token and +# basic auth. Verified against a live 15.0.2 instance. +# +# On such a server this returns the `unsupported-server` envelope, naming the +# version it found and the version it needs, and CARRYING THE FAILING JOB NAMES +# it was still able to determine. What it must never do is return an empty +# `failures` array: "your CI is fine" and "I cannot see your CI at all" are +# opposite facts, and they are the same observation to a caller that only sees +# an empty list. +set -e +. "$(dirname "$0")/_ci.sh" + +CONCEPT=ci-failures + +if [ -z "$CODEV_CI_RUN_ID" ]; then + jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' + echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 + exit 2 +fi + +REPO="$(gitea_repo)" || exit 1 +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +rc=0 +gitea_ci_fetch "repos/${REPO}/actions/runs/${CODEV_CI_RUN_ID}" "$TMP/run.json" || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "GET repos/${REPO}/actions/runs/${CODEV_CI_RUN_ID}" "$GITEA_TIMEOUT" + exit 1 +fi +if [ "$rc" -eq 44 ]; then + ci_fail "$CONCEPT" not-found "run ${CODEV_CI_RUN_ID} does not exist in ${REPO}; pass the \`id\` from ci-runs, not the run \`number\`" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" '{runId: $r}')" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "Forgejo could not read run ${CODEV_CI_RUN_ID}: $(head -c 200 "$TMP/run.json")" + exit 1 +fi + +RUN_INDEX=$(jq -r '.index_in_repo' "$TMP/run.json") +RUN_STATUS=$(jq -r '.status // "unknown"' "$TMP/run.json") + +rc=0 +gitea_ci_jobs "$CONCEPT" "$REPO" "$CODEV_CI_RUN_ID" "$RUN_INDEX" "$TMP" || rc=$? +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "could not list the jobs of run ${CODEV_CI_RUN_ID}" + exit 1 +fi +JOB_SOURCE=$(cat "$TMP/jobs.source") +FAILED=$(jq -c '[ .[] | select(.status == "failure" or .status == "timed_out" or .conclusion == "failure") ]' "$TMP/jobs.json") +JOBS_FAILED=$(printf '%s' "$FAILED" | jq 'length') + +# Forgejo 15: the jobs came from the task scan, so there is no job id and no log +# API. Say which server this is, and hand back what IS known. +if [ "$JOB_SOURCE" = "tasks-scan" ]; then + gitea_ci_unsupported "$CONCEPT" "job-log" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --arg rs "$RUN_STATUS" --argjson f "$FAILED" --argjson n "$JOBS_FAILED" \ + '{runId: ($r | tonumber? // $r), runStatus: $rs, jobsFailed: $n, + failingJobs: [ $f[] | {taskId: .taskId, jobName: .name} ], + hint: "ci-run-view still works on this server and lists per-job status"}')" + exit 1 +fi + +if [ -n "$CODEV_CI_JOB_ID" ]; then + TARGET=$(jq -c --argjson j "$CODEV_CI_JOB_ID" 'first(.[] | select(.id == $j)) // empty' "$TMP/jobs.json") + if [ -z "$TARGET" ]; then + ci_fail "$CONCEPT" not-found "job ${CODEV_CI_JOB_ID} is not part of run ${CODEV_CI_RUN_ID}" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --arg j "$CODEV_CI_JOB_ID" '{runId: $r, jobId: $j}')" + exit 1 + fi +else + TARGET=$(printf '%s' "$FAILED" | jq -c '.[0] // empty') +fi + +if [ -z "$TARGET" ]; then + jq -cn --arg r "$CODEV_CI_RUN_ID" --arg rs "$RUN_STATUS" '{ + ok: true, provider: "gitea", runId: ($r | tonumber? // $r), + runStatus: $rs, runConclusion: null, jobsFailed: 0, extracted: false, + reason: (if $rs == "success" then "no job in this run failed" else "no failing job found for this run" end), + failures: []}' + exit 0 +fi + +JOB_ID=$(printf '%s' "$TARGET" | jq -r '.id') +JOB_NAME=$(printf '%s' "$TARGET" | jq -r '.name') +JOB_STATE=$(printf '%s' "$TARGET" | jq -r '.status') + +rc=0 +gitea_ci_job_log "$REPO" "$JOB_ID" "$JOB_STATE" "$TMP/raw.log" || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "GET repos/${REPO}/actions/jobs/${JOB_ID}/logs" "$GITEA_TIMEOUT" + exit 1 +fi +if [ "$rc" -eq 44 ]; then + gitea_ci_unsupported "$CONCEPT" "job-log" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --argjson j "$JOB_ID" --arg n "$JOB_NAME" --argjson c "$JOBS_FAILED" \ + '{runId: ($r | tonumber? // $r), jobsFailed: $c, failingJobs: [{jobId: $j, jobName: $n}]}')" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "could not read the log for job ${JOB_ID} (${JOB_NAME})" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --argjson j "$JOB_ID" --arg n "$JOB_NAME" '{runId: $r, jobId: $j, jobName: $n}')" + exit 1 +fi + +ci_clean_log < "$TMP/raw.log" > "$TMP/clean.log" +# awk NR, not `wc -l`: a log with no trailing newline makes wc undercount by +# one, and logLines would then disagree with the from/to the extractor reports. +LOG_LINES=$(awk 'END {print NR}' "$TMP/clean.log") +OTHERS=$(printf '%s' "$FAILED" | jq -c --argjson j "$JOB_ID" '[.[] | select(.id != $j) | {id: .id, name: .name}]') + +if ci_extract "$TMP/clean.log" > "$TMP/extract.txt" 2>/dev/null && [ -s "$TMP/extract.txt" ]; then + MATCHED=$(head -1 "$TMP/extract.txt" | cut -f1) + FROM=$(head -1 "$TMP/extract.txt" | cut -f2) + TO=$(head -1 "$TMP/extract.txt" | cut -f3) + tail -n +2 "$TMP/extract.txt" > "$TMP/text.txt" + CAP=$CI_MAX_STEP_BYTES + [ "$CAP" -gt "$CI_MAX_RESPONSE_BYTES" ] && CAP=$CI_MAX_RESPONSE_BYTES + META=$(ci_cap_file "$TMP/text.txt" "$CAP" "$TMP/capped.txt") + RETURNED_LINES=${META% *} + TRUNCATED=${META#* } + TEXT=$(jq -R -s -c . < "$TMP/capped.txt") + jq -cn \ + --arg run "$CODEV_CI_RUN_ID" --arg rs "$RUN_STATUS" \ + --argjson job "$JOB_ID" --arg jobName "$JOB_NAME" \ + --arg matched "$MATCHED" --argjson text "$TEXT" \ + --argjson from "$FROM" --argjson to "$TO" \ + --argjson logLines "$LOG_LINES" --argjson returned "$RETURNED_LINES" \ + --argjson truncated "$TRUNCATED" --argjson jobsFailed "$JOBS_FAILED" \ + --argjson others "$OTHERS" --argjson cached "$CI_LOG_FROM_CACHE" \ + '{ok: true, provider: "gitea", runId: ($run | tonumber? // $run), + runStatus: $rs, runConclusion: null, + jobsFailed: $jobsFailed, extracted: true, + failures: [{ + jobId: $job, jobName: $jobName, + stepName: null, stepNumber: null, + matchedBy: $matched, text: $text, + from: $from, to: $to, + logLines: $logLines, returnedLines: $returned, truncated: $truncated + }], + otherFailingJobs: $others, cached: $cached}' + exit 0 +fi + +jq -cn \ + --arg run "$CODEV_CI_RUN_ID" --arg rs "$RUN_STATUS" \ + --argjson job "$JOB_ID" --arg jobName "$JOB_NAME" \ + --argjson logLines "$LOG_LINES" --argjson jobsFailed "$JOBS_FAILED" \ + --argjson others "$OTHERS" --argjson cached "$CI_LOG_FROM_CACHE" \ + '{ok: true, provider: "gitea", runId: ($run | tonumber? // $run), + runStatus: $rs, runConclusion: null, + jobsFailed: $jobsFailed, extracted: false, + reason: "no recognized failure pattern", + failures: [{jobId: $job, jobName: $jobName, logLines: $logLines}], + otherFailingJobs: $others, cached: $cached, + next: ("ci-run-log CODEV_CI_RUN_ID=" + $run + " CODEV_CI_JOB_ID=" + ($job|tostring) + " CODEV_CI_LOG_TAIL=80")}' diff --git a/packages/codev/scripts/forge/gitea/ci-run-log.sh b/packages/codev/scripts/forge/gitea/ci-run-log.sh new file mode 100755 index 000000000..6ad082f0c --- /dev/null +++ b/packages/codev/scripts/forge/gitea/ci-run-log.sh @@ -0,0 +1,105 @@ +#!/bin/sh +# Forge concept: ci-run-log (Gitea/Forgejo via tea CLI) +# forge-executable: tea +# Input: CODEV_CI_RUN_ID (required), CODEV_CI_JOB_ID (optional), +# exactly ONE of CODEV_CI_LOG_TAIL / CODEV_CI_LOG_HEAD / CODEV_CI_LOG_GREP +# (+ CODEV_CI_LOG_CONTEXT, default 3) +# Output: the shared ci-run-log envelope; see github/ci-run-log.sh +# +# REQUIRES FORGEJO 16.0 OR LATER, for the same reason as ci-failures: the job +# log API did not exist before it. On an older server this returns the +# unsupported-server envelope rather than an empty window. +# +# Forgejo 16 serves the log as text/plain with `accept-ranges: bytes`, so a tail +# could in principle be fetched as a byte range. It is not, because `tea api` +# sends no Range header and the log cache makes the second and third window over +# the same job free anyway — measured 142 KB / 1.0s for a 1599-line job log. +set -e +. "$(dirname "$0")/_ci.sh" + +CONCEPT=ci-run-log + +if [ -z "$CODEV_CI_RUN_ID" ]; then + jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' + echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 + exit 2 +fi + +# Window first: a malformed request should not cost an API call. +ci_window_parse "$CONCEPT" + +REPO="$(gitea_repo)" || exit 1 +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +rc=0 +gitea_ci_fetch "repos/${REPO}/actions/runs/${CODEV_CI_RUN_ID}" "$TMP/run.json" || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "GET repos/${REPO}/actions/runs/${CODEV_CI_RUN_ID}" "$GITEA_TIMEOUT" + exit 1 +fi +if [ "$rc" -eq 44 ]; then + ci_fail "$CONCEPT" not-found "run ${CODEV_CI_RUN_ID} does not exist in ${REPO}; pass the \`id\` from ci-runs, not the run \`number\`" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" '{runId: $r}')" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "Forgejo could not read run ${CODEV_CI_RUN_ID}: $(head -c 200 "$TMP/run.json")" + exit 1 +fi + +RUN_INDEX=$(jq -r '.index_in_repo' "$TMP/run.json") + +rc=0 +gitea_ci_jobs "$CONCEPT" "$REPO" "$CODEV_CI_RUN_ID" "$RUN_INDEX" "$TMP" || rc=$? +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "could not list the jobs of run ${CODEV_CI_RUN_ID}" + exit 1 +fi + +if [ "$(cat "$TMP/jobs.source")" = "tasks-scan" ]; then + gitea_ci_unsupported "$CONCEPT" "job-log" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" '{runId: ($r | tonumber? // $r), hint: "ci-run-view still works on this server and lists per-job status"}')" + exit 1 +fi + +if [ -n "$CODEV_CI_JOB_ID" ]; then + TARGET=$(jq -c --argjson j "$CODEV_CI_JOB_ID" 'first(.[] | select(.id == $j)) // empty' "$TMP/jobs.json") + if [ -z "$TARGET" ]; then + ci_fail "$CONCEPT" not-found "job ${CODEV_CI_JOB_ID} is not part of run ${CODEV_CI_RUN_ID}" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --arg j "$CODEV_CI_JOB_ID" '{runId: $r, jobId: $j}')" + exit 1 + fi +else + TARGET=$(jq -c 'first(.[] | select(.status == "failure" or .status == "timed_out")) // empty' "$TMP/jobs.json") + [ -n "$TARGET" ] || TARGET=$(jq -c 'if length == 1 then .[0] else empty end' "$TMP/jobs.json") + if [ -z "$TARGET" ]; then + ci_fail "$CONCEPT" bad-input "run ${CODEV_CI_RUN_ID} has no failing job and more than one job; set CODEV_CI_JOB_ID" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --slurpfile j "$TMP/jobs.json" '{runId: $r, jobs: [ $j[0][] | {id: .id, name: .name} ]}')" + exit 2 + fi +fi + +JOB_ID=$(printf '%s' "$TARGET" | jq -r '.id') +JOB_NAME=$(printf '%s' "$TARGET" | jq -r '.name') +JOB_STATE=$(printf '%s' "$TARGET" | jq -r '.status') + +rc=0 +gitea_ci_job_log "$REPO" "$JOB_ID" "$JOB_STATE" "$TMP/raw.log" || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "GET repos/${REPO}/actions/jobs/${JOB_ID}/logs" "$GITEA_TIMEOUT" + exit 1 +fi +if [ "$rc" -eq 44 ]; then + gitea_ci_unsupported "$CONCEPT" "job-log" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --argjson j "$JOB_ID" --arg n "$JOB_NAME" '{runId: ($r | tonumber? // $r), jobId: $j, jobName: $n}')" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "could not read the log for job ${JOB_ID} (${JOB_NAME})" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" --argjson j "$JOB_ID" --arg n "$JOB_NAME" '{runId: $r, jobId: $j, jobName: $n}')" + exit 1 +fi + +ci_clean_log < "$TMP/raw.log" > "$TMP/clean.log" +ci_window_emit "$CONCEPT" "$TMP" gitea "$CODEV_CI_RUN_ID" "$JOB_ID" "$JOB_NAME" "$CI_LOG_FROM_CACHE" diff --git a/packages/codev/scripts/forge/gitea/ci-run-view.sh b/packages/codev/scripts/forge/gitea/ci-run-view.sh new file mode 100755 index 000000000..625bf7361 --- /dev/null +++ b/packages/codev/scripts/forge/gitea/ci-run-view.sh @@ -0,0 +1,76 @@ +#!/bin/sh +# Forge concept: ci-run-view (Gitea/Forgejo via tea CLI) +# forge-executable: tea +# Input: CODEV_CI_RUN_ID (required — the `id` from ci-runs, NOT the `number`) +# Output: {ok, provider, run: {...}, jobs: [...], jobSource, truncated} +# +# Still no log bytes. +# +# `jobSource` says which route answered, because the two are not equivalent: +# +# runs-jobs Forgejo 16 `actions/runs/{id}/jobs`. Carries the job `id` that +# ci-failures and ci-run-log need. +# tasks-scan Forgejo 15 has no jobs route, so the jobs are recovered by +# filtering `actions/tasks` on run_number. That yields TASK ids, +# not job ids, so `id` is null and `taskId` carries what exists. +# A task id looks like a job id and is not accepted by the log +# API, so handing it back as `id` would be a trap. +# +# Forgejo exposes no per-step data on either version, so `failedSteps` is always +# [] here. It is present rather than omitted so the field means the same thing +# on both providers. +set -e +. "$(dirname "$0")/_ci.sh" + +CONCEPT=ci-run-view + +if [ -z "$CODEV_CI_RUN_ID" ]; then + jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' + echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 + exit 2 +fi + +REPO="$(gitea_repo)" || exit 1 +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +rc=0 +gitea_ci_fetch "repos/${REPO}/actions/runs/${CODEV_CI_RUN_ID}" "$TMP/run.json" || rc=$? +if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "GET repos/${REPO}/actions/runs/${CODEV_CI_RUN_ID}" "$GITEA_TIMEOUT" + exit 1 +fi +if [ "$rc" -eq 44 ]; then + ci_fail "$CONCEPT" not-found "run ${CODEV_CI_RUN_ID} does not exist in ${REPO}; pass the \`id\` from ci-runs, not the run \`number\` — on Forgejo both are valid ids for different runs" \ + "$(jq -cn --arg r "$CODEV_CI_RUN_ID" '{runId: $r}')" + exit 1 +fi +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "Forgejo could not read run ${CODEV_CI_RUN_ID}: $(head -c 200 "$TMP/run.json")" + exit 1 +fi + +RUN_INDEX=$(jq -r '.index_in_repo' "$TMP/run.json") + +rc=0 +gitea_ci_jobs "$CONCEPT" "$REPO" "$CODEV_CI_RUN_ID" "$RUN_INDEX" "$TMP" || rc=$? +if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "could not list the jobs of run ${CODEV_CI_RUN_ID}" + exit 1 +fi + +# Read back from files rather than from a command substitution: gitea_ci_jobs +# has to report three things (the jobs, which route answered, whether the walk +# was cut short) and a subshell would discard two of them. +jq -c --slurpfile jobs "$TMP/jobs.json" --arg source "$(cat "$TMP/jobs.source")" \ + --argjson truncated "$(cat "$TMP/jobs.truncated")" '{ + ok: true, provider: "gitea", jobSource: $source, + run: { + id: .id, number: .index_in_repo, title: .title, + workflow: .workflow_id, status: .status, conclusion: null, + branch: .prettyref, sha: .commit_sha, event: .event, + url: .html_url, createdAt: .created + }, + jobs: $jobs[0], + truncated: $truncated +}' "$TMP/run.json" diff --git a/packages/codev/scripts/forge/gitea/ci-runs.sh b/packages/codev/scripts/forge/gitea/ci-runs.sh new file mode 100755 index 000000000..1b098cd9a --- /dev/null +++ b/packages/codev/scripts/forge/gitea/ci-runs.sh @@ -0,0 +1,105 @@ +#!/bin/sh +# Forge concept: ci-runs (Gitea/Forgejo via tea CLI) +# forge-executable: tea +# Input: CODEV_BRANCH_NAME (optional), CODEV_CI_STATUS (optional), +# CODEV_CI_WORKFLOW (optional — workflow file, e.g. ci.yml), +# CODEV_CI_LIMIT (optional, default 20), CODEV_PR_BASE (optional) +# Output: {ok, provider, runs: [...], truncated, note?} +# +# The cheap question. No log bytes on any path. +# +# `conclusion` is always null here and that is not an oversight: Forgejo has no +# separate conclusion field, `status` carries success/failure/skipped/canceled +# directly. Emitting a fabricated conclusion would make the two providers look +# identical where they are not. +# +# Branch filtering is client-side and PR-aware — see the header of _ci.sh for +# why `branch=` cannot be sent to the server and why `builder/x` has to become +# `#123` before it will match anything. +set -e +. "$(dirname "$0")/_ci.sh" + +CONCEPT=ci-runs + +ci_check_status "$CONCEPT" "$CODEV_CI_STATUS" + +LIMIT=${CODEV_CI_LIMIT:-$CI_LIMIT_DEFAULT} +case "$LIMIT" in ''|*[!0-9]*|0) LIMIT=$CI_LIMIT_DEFAULT ;; esac + +REPO="$(gitea_repo)" || exit 1 + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +QUERY="" +[ -n "$CODEV_CI_STATUS" ] && QUERY="status=$(ci_status_for gitea "$CODEV_CI_STATUS")" + +NOTE=null +PR_REF="" +if [ -n "$CODEV_BRANCH_NAME" ]; then + PR_REF=$(gitea_ci_pr_ref "$REPO" "$CODEV_BRANCH_NAME") || { + ci_fail "$CONCEPT" forge-error "could not read repository '${REPO}' while resolving branch '${CODEV_BRANCH_NAME}'" + exit 1 + } + if [ -z "$PR_REF" ]; then + NOTE='"branch has no pull request, so only push/schedule runs on that branch can match; Forgejo labels pull_request runs with the PR number, not the branch"' + fi +fi + +ACC='[]' +PAGE=1 +TRUNCATED=false +while [ "$PAGE" -le "$CI_MAX_PAGES" ]; do + rc=0 + gitea_ci_fetch "repos/${REPO}/actions/runs?page=${PAGE}&limit=${GITEA_PAGE_LIMIT}${QUERY:+&$QUERY}" "$TMP/page.json" || rc=$? + if [ "$rc" -eq 124 ]; then + ci_fail_timeout "$CONCEPT" "GET repos/${REPO}/actions/runs" "$GITEA_TIMEOUT" + exit 1 + fi + if [ "$rc" -eq 44 ]; then + gitea_ci_unsupported "$CONCEPT" "workflow-runs" "$(jq -cn --arg r "$REPO" '{repo: $r}')" + exit 1 + fi + if [ "$rc" -ne 0 ]; then + ci_fail "$CONCEPT" forge-error "Forgejo could not list workflow runs for ${REPO}: $(head -c 200 "$TMP/page.json")" + exit 1 + fi + + RAW=$(jq '.workflow_runs | length' "$TMP/page.json") + HITS=$(jq -c --arg branch "$CODEV_BRANCH_NAME" --arg prref "$PR_REF" --arg wf "$CODEV_CI_WORKFLOW" ' + [ .workflow_runs[]? + | select($wf == "" or .workflow_id == $wf) + | select($branch == "" or .prettyref == $branch or ($prref != "" and .prettyref == $prref)) + | { + id: .id, + number: .index_in_repo, + name: .title, + workflow: .workflow_id, + status: .status, + conclusion: null, + branch: .prettyref, + sha: .commit_sha, + event: .event, + url: .html_url, + createdAt: .created + } ]' "$TMP/page.json") + ACC=$(printf '%s\n%s' "$ACC" "$HITS" | jq -s -c 'add') + + [ "$(printf '%s' "$ACC" | jq 'length')" -ge "$LIMIT" ] && break + if [ "$RAW" -lt "$GITEA_PAGE_LIMIT" ]; then break; fi + PAGE=$((PAGE + 1)) + # Ran out of pages we are allowed to walk while a client-side filter was still + # discarding candidates. Say so: a short list and a truncated one look + # identical once printed. + if [ "$PAGE" -gt "$CI_MAX_PAGES" ] && { [ -n "$CODEV_BRANCH_NAME" ] || [ -n "$CODEV_CI_WORKFLOW" ]; }; then + TRUNCATED=true + echo "${CONCEPT}: stopped after ${CI_MAX_PAGES} pages of runs while filtering; raise CODEV_CI_MAX_PAGES for a deeper search" >&2 + fi +done + +printf '%s' "$ACC" | jq -c --argjson limit "$LIMIT" --argjson truncated "$TRUNCATED" --argjson note "$NOTE" '{ + ok: true, provider: "gitea", + runs: .[0:$limit], + truncated: ($truncated or (length > $limit)), + note: $note +}' From 346243ef54dcc7d9dea94d5bd1cc8ecbcd07d50a Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:27:18 -0600 Subject: [PATCH 09/30] [PIR #13] test(forge): pin the CI concepts against two real captured logs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 51 tests. The two fixtures are verbatim captures, stored gzipped because the exact bytes are the point: a 2528-line GitHub vitest failure and a 1599-line Forgejo Go failure. Between them they carry the three traps that make naive extraction lie, and each one is a test: - every payload line is ANSI-wrapped, so a matcher that skips cleaning finds nothing on a log that plainly contains a failure; - the first line containing 'Error:' is a PASSING test's fixture string, 1214 lines above the real failure; - 'Test Files' appears four times before the summary that says failed, and the Forgejo log's last 25 lines are git credential cleanup — so tailing returns nothing at all. Also pinned: an old Forgejo returns unsupported-server and never an empty failures array; a timeout reports as a timeout through both the script envelope and executeForgeCommandDetailed; ci-run-log refuses zero or two windows BEFORE spending an API call; and gitea's list calls always send page= (Forgejo ignores limit without it). Both SKILL.md twins document the concepts, the envelope and the version floor. Co-Authored-By: Claude Opus 5 --- .claude/skills/forge/SKILL.md | 115 +++ .codex/skills/forge/SKILL.md | 115 +++ .../fixtures/pir-13/forgejo-go-failure.log.gz | Bin 0 -> 20000 bytes .../pir-13/github-vitest-failure.log.gz | Bin 0 -> 37718 bytes .../src/__tests__/pir-13-ci-concepts.test.ts | 843 ++++++++++++++++++ 5 files changed, 1073 insertions(+) create mode 100644 packages/codev/src/__tests__/fixtures/pir-13/forgejo-go-failure.log.gz create mode 100644 packages/codev/src/__tests__/fixtures/pir-13/github-vitest-failure.log.gz create mode 100644 packages/codev/src/__tests__/pir-13-ci-concepts.test.ts diff --git a/.claude/skills/forge/SKILL.md b/.claude/skills/forge/SKILL.md index 54618c7f9..603ff988f 100644 --- a/.claude/skills/forge/SKILL.md +++ b/.claude/skills/forge/SKILL.md @@ -22,6 +22,114 @@ Forge concept commands decouple codev from direct `gh` CLI calls. Each GitHub op | `pr-view` | `CODEV_PR_NUMBER`, `CODEV_INCLUDE_COMMENTS` (optional) | View PR details (JSON or text) | | `pr-diff` | `CODEV_PR_NUMBER`, `CODEV_DIFF_NAME_ONLY` (optional) | Get PR diff | | `auth-status` | — | Check forge authentication status | +| `ci-runs` | `CODEV_BRANCH_NAME`, `CODEV_CI_STATUS`, `CODEV_CI_WORKFLOW`, `CODEV_CI_LIMIT` (all optional) | List workflow runs (no log bytes) | +| `ci-run-view` | `CODEV_CI_RUN_ID` | One run plus per-job status (no log bytes) | +| `ci-failures` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (optional) | The failing job's assertion, extracted and capped | +| `ci-run-log` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (opt), and exactly one of `CODEV_CI_LOG_TAIL` / `CODEV_CI_LOG_HEAD` / `CODEV_CI_LOG_GREP` | A raw log window | + +## CI concepts + +Four concepts, **tiered so the cheap question stays cheap**. A builder asks about +CI at four moments and they cost very different amounts: + +| Question | Concept | Reads a log? | +|---|---|---| +| Did my push pass? | `ci-runs` | No | +| Is it still running, and which job is pending? | `ci-runs`, `ci-run-view` | No | +| It failed — why? | `ci-failures` | Yes, one job | +| Is this mine or pre-existing? | `ci-runs` with `CODEV_CI_WORKFLOW` | No | +| Extraction gave up — show me the log | `ci-run-log` | Yes, one window | + +`ci-run-log` is a separate concept rather than a flag on `ci-failures` on +purpose: a window parameter on the main call gets passed by habit, and then +every status question drags a log again. + +### The response envelope + +Every ci-* concept prints ONE JSON object on stdout — **on success and on +failure**. Errors are values, not absences, because `executeForgeCommand` +flattens every failure mode to `null`: + +```json +{ "ok": false, "error": "timeout", "seconds": 60, + "detail": "GET repos/o/r/actions/tasks did not return within 60s", + "remedy": "raise CODEV_FORGE_TIMEOUT" } +``` + +`error` is one of `timeout`, `not-found`, `unsupported-server`, `forge-error`, +`bad-input`. Use `executeForgeCommandDetailed()` (not `executeForgeCommand`) when +you need to tell a timeout from a failure — it returns `{ok, data, stdout, +stderr, exitCode, timedOut, unavailable, durationMs}` and keeps stdout on the +failure path. + +**Any response carrying log text also carries `logLines`, `returnedLines` and +`truncated`.** A trimmed answer must never read as a whole one. + +### `ci-failures`, and what it does when it cannot tell + +Extraction runs a ladder and names the rung that fired in `matchedBy`: +`vitest`, `go-test`, `tsc`, `runner-marker`, `first-error`. When nothing +matches it does **not** fall back to the last N lines: + +```json +{ "extracted": false, "reason": "no recognized failure pattern", + "failures": [{ "jobId": 11952749, "jobName": "test-unit", "logLines": 1599 }], + "next": "ci-run-log CODEV_CI_RUN_ID=6554924 CODEV_CI_JOB_ID=11952749 CODEV_CI_LOG_TAIL=80" } +``` + +A builder handed 50 arbitrary lines treats them as the diagnosis; one told +extraction failed reads the log with the targeted call the response hands it. + +### `ci-run-log` windows + +Exactly one of `CODEV_CI_LOG_TAIL=N`, `CODEV_CI_LOG_HEAD=N`, or +`CODEV_CI_LOG_GREP=` (with `CODEV_CI_LOG_CONTEXT`, default 3). Zero or two +is exit 2 — there is deliberately no default. The response carries `from` and +`to` as line numbers into the full log; in grep mode `contiguous` is false and +`matchLines` lists exactly which lines matched. + +### Run ids + +`CODEV_CI_RUN_ID` is always the **`id`** field from `ci-runs`, never `number`. +On Forgejo the two are separate id spaces and **both resolve on the same +route**, so passing the number silently answers about a different, real run. + +### Provider notes + +| | GitHub | Forgejo/Gitea | +|---|---|---| +| Runs | `gh run list` | `GET actions/runs` (always with `page=`) | +| Jobs | `gh run view --json jobs` | `GET actions/runs/{id}/jobs` (Forgejo ≥16) or a scan of `actions/tasks` (Forgejo 15) | +| Log | `gh api repos/{owner}/{repo}/actions/jobs/{id}/logs` | `GET actions/jobs/{id}/logs` (**Forgejo ≥16 only**) | +| Per-step data | yes | no — `failedSteps` is always `[]` | +| `conclusion` | yes | always `null`; `status` carries it | + +**`gh run view --log-failed` is not used**, despite selecting failed steps in +principle. Measured on run 32515040122 of this repository it returned 2528 +lines / 293 KB with every line tagged `UNKNOWN STEP`: it selects the failing +JOB, and its filename-to-step mapping had missed. The per-job log endpoint is +used on both providers instead. + +**Forgejo below 16.0 has no Actions log API at all.** `ci-failures` and +`ci-run-log` return `error: "unsupported-server"` there, naming the version +found and the version needed and still listing the failing job names; +`ci-runs` and `ci-run-view` keep working. An old server must never be +mistaken for a run with no failures. + +**Forgejo records a `pull_request` run's branch as `#`**, not as a +branch name, and ignores `branch=` on the query. `ci-runs` resolves +`CODEV_BRANCH_NAME` to its PR ref first and filters client-side, so a branch +with no PR gets a `note` saying only push runs can match. + +**`gitlab` and `linear` have all four concepts explicitly disabled**, not merely +unimplemented — otherwise they would fall through to the GitHub default and run +`gh` against whatever remote it resolved. + +### Log cache + +A terminal job's log is immutable, so both log concepts cache it under +`$TMPDIR/codev-ci-logs///.log`. An in-progress job is +never cached or read from cache. `CODEV_CI_NO_CACHE=1` bypasses it. ## Configuration @@ -88,6 +196,13 @@ so a PR targeting an integration branch needs that variable. | `CODEV_FORGE_MERGED_DAYS` | 7 | `recently-merged` window when `CODEV_SINCE_DATE` is unset | | `CODEV_FORGE_MERGED_MAX` | 300 | merged PRs `recently-merged` will resolve before refusing | | `CODEV_FORGE_SEARCH_MAX` | 10 | PRs `pr-search` resolves for one issue number | +| `CODEV_CI_LIMIT` | 20 | runs `ci-runs` returns | +| `CODEV_CI_MAX_PAGES` | 4 | pages `ci-runs` walks while filtering client-side | +| `CODEV_CI_TASKS_MAX_PAGES` | 20 | pages the Forgejo-15 task scan walks (it stops early once past the run) | +| `CODEV_CI_MAX_STEP_BYTES` | 2048 | cap on one extracted failure | +| `CODEV_CI_MAX_BYTES` | 8192 | cap on a whole ci-* response | +| `CODEV_CI_NO_CACHE` | unset | `1` disables the log cache | +| `CODEV_CI_CACHE_MAX_MB` | 32 | largest log that will be cached | ### Exit statuses diff --git a/.codex/skills/forge/SKILL.md b/.codex/skills/forge/SKILL.md index 54618c7f9..603ff988f 100644 --- a/.codex/skills/forge/SKILL.md +++ b/.codex/skills/forge/SKILL.md @@ -22,6 +22,114 @@ Forge concept commands decouple codev from direct `gh` CLI calls. Each GitHub op | `pr-view` | `CODEV_PR_NUMBER`, `CODEV_INCLUDE_COMMENTS` (optional) | View PR details (JSON or text) | | `pr-diff` | `CODEV_PR_NUMBER`, `CODEV_DIFF_NAME_ONLY` (optional) | Get PR diff | | `auth-status` | — | Check forge authentication status | +| `ci-runs` | `CODEV_BRANCH_NAME`, `CODEV_CI_STATUS`, `CODEV_CI_WORKFLOW`, `CODEV_CI_LIMIT` (all optional) | List workflow runs (no log bytes) | +| `ci-run-view` | `CODEV_CI_RUN_ID` | One run plus per-job status (no log bytes) | +| `ci-failures` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (optional) | The failing job's assertion, extracted and capped | +| `ci-run-log` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (opt), and exactly one of `CODEV_CI_LOG_TAIL` / `CODEV_CI_LOG_HEAD` / `CODEV_CI_LOG_GREP` | A raw log window | + +## CI concepts + +Four concepts, **tiered so the cheap question stays cheap**. A builder asks about +CI at four moments and they cost very different amounts: + +| Question | Concept | Reads a log? | +|---|---|---| +| Did my push pass? | `ci-runs` | No | +| Is it still running, and which job is pending? | `ci-runs`, `ci-run-view` | No | +| It failed — why? | `ci-failures` | Yes, one job | +| Is this mine or pre-existing? | `ci-runs` with `CODEV_CI_WORKFLOW` | No | +| Extraction gave up — show me the log | `ci-run-log` | Yes, one window | + +`ci-run-log` is a separate concept rather than a flag on `ci-failures` on +purpose: a window parameter on the main call gets passed by habit, and then +every status question drags a log again. + +### The response envelope + +Every ci-* concept prints ONE JSON object on stdout — **on success and on +failure**. Errors are values, not absences, because `executeForgeCommand` +flattens every failure mode to `null`: + +```json +{ "ok": false, "error": "timeout", "seconds": 60, + "detail": "GET repos/o/r/actions/tasks did not return within 60s", + "remedy": "raise CODEV_FORGE_TIMEOUT" } +``` + +`error` is one of `timeout`, `not-found`, `unsupported-server`, `forge-error`, +`bad-input`. Use `executeForgeCommandDetailed()` (not `executeForgeCommand`) when +you need to tell a timeout from a failure — it returns `{ok, data, stdout, +stderr, exitCode, timedOut, unavailable, durationMs}` and keeps stdout on the +failure path. + +**Any response carrying log text also carries `logLines`, `returnedLines` and +`truncated`.** A trimmed answer must never read as a whole one. + +### `ci-failures`, and what it does when it cannot tell + +Extraction runs a ladder and names the rung that fired in `matchedBy`: +`vitest`, `go-test`, `tsc`, `runner-marker`, `first-error`. When nothing +matches it does **not** fall back to the last N lines: + +```json +{ "extracted": false, "reason": "no recognized failure pattern", + "failures": [{ "jobId": 11952749, "jobName": "test-unit", "logLines": 1599 }], + "next": "ci-run-log CODEV_CI_RUN_ID=6554924 CODEV_CI_JOB_ID=11952749 CODEV_CI_LOG_TAIL=80" } +``` + +A builder handed 50 arbitrary lines treats them as the diagnosis; one told +extraction failed reads the log with the targeted call the response hands it. + +### `ci-run-log` windows + +Exactly one of `CODEV_CI_LOG_TAIL=N`, `CODEV_CI_LOG_HEAD=N`, or +`CODEV_CI_LOG_GREP=` (with `CODEV_CI_LOG_CONTEXT`, default 3). Zero or two +is exit 2 — there is deliberately no default. The response carries `from` and +`to` as line numbers into the full log; in grep mode `contiguous` is false and +`matchLines` lists exactly which lines matched. + +### Run ids + +`CODEV_CI_RUN_ID` is always the **`id`** field from `ci-runs`, never `number`. +On Forgejo the two are separate id spaces and **both resolve on the same +route**, so passing the number silently answers about a different, real run. + +### Provider notes + +| | GitHub | Forgejo/Gitea | +|---|---|---| +| Runs | `gh run list` | `GET actions/runs` (always with `page=`) | +| Jobs | `gh run view --json jobs` | `GET actions/runs/{id}/jobs` (Forgejo ≥16) or a scan of `actions/tasks` (Forgejo 15) | +| Log | `gh api repos/{owner}/{repo}/actions/jobs/{id}/logs` | `GET actions/jobs/{id}/logs` (**Forgejo ≥16 only**) | +| Per-step data | yes | no — `failedSteps` is always `[]` | +| `conclusion` | yes | always `null`; `status` carries it | + +**`gh run view --log-failed` is not used**, despite selecting failed steps in +principle. Measured on run 32515040122 of this repository it returned 2528 +lines / 293 KB with every line tagged `UNKNOWN STEP`: it selects the failing +JOB, and its filename-to-step mapping had missed. The per-job log endpoint is +used on both providers instead. + +**Forgejo below 16.0 has no Actions log API at all.** `ci-failures` and +`ci-run-log` return `error: "unsupported-server"` there, naming the version +found and the version needed and still listing the failing job names; +`ci-runs` and `ci-run-view` keep working. An old server must never be +mistaken for a run with no failures. + +**Forgejo records a `pull_request` run's branch as `#`**, not as a +branch name, and ignores `branch=` on the query. `ci-runs` resolves +`CODEV_BRANCH_NAME` to its PR ref first and filters client-side, so a branch +with no PR gets a `note` saying only push runs can match. + +**`gitlab` and `linear` have all four concepts explicitly disabled**, not merely +unimplemented — otherwise they would fall through to the GitHub default and run +`gh` against whatever remote it resolved. + +### Log cache + +A terminal job's log is immutable, so both log concepts cache it under +`$TMPDIR/codev-ci-logs///.log`. An in-progress job is +never cached or read from cache. `CODEV_CI_NO_CACHE=1` bypasses it. ## Configuration @@ -88,6 +196,13 @@ so a PR targeting an integration branch needs that variable. | `CODEV_FORGE_MERGED_DAYS` | 7 | `recently-merged` window when `CODEV_SINCE_DATE` is unset | | `CODEV_FORGE_MERGED_MAX` | 300 | merged PRs `recently-merged` will resolve before refusing | | `CODEV_FORGE_SEARCH_MAX` | 10 | PRs `pr-search` resolves for one issue number | +| `CODEV_CI_LIMIT` | 20 | runs `ci-runs` returns | +| `CODEV_CI_MAX_PAGES` | 4 | pages `ci-runs` walks while filtering client-side | +| `CODEV_CI_TASKS_MAX_PAGES` | 20 | pages the Forgejo-15 task scan walks (it stops early once past the run) | +| `CODEV_CI_MAX_STEP_BYTES` | 2048 | cap on one extracted failure | +| `CODEV_CI_MAX_BYTES` | 8192 | cap on a whole ci-* response | +| `CODEV_CI_NO_CACHE` | unset | `1` disables the log cache | +| `CODEV_CI_CACHE_MAX_MB` | 32 | largest log that will be cached | ### Exit statuses diff --git a/packages/codev/src/__tests__/fixtures/pir-13/forgejo-go-failure.log.gz b/packages/codev/src/__tests__/fixtures/pir-13/forgejo-go-failure.log.gz new file mode 100644 index 0000000000000000000000000000000000000000..47e49a62f4df3c8ec34f0144eaa5e79fddff61f9 GIT binary patch literal 20000 zcmV)sK$yQDiwFoSy@+Z819D+^EoN_WXJu+{E^KdS0PVfqk{id8CV0&?+jq!S_jKtG zK>YvIcFhb)Q4)6~kw#Lddyc4)C6I|Kh$tkK36L!H*|nXEz3w*Ko87Tp>nE72xtZrV zFEITG>mCW9imZ&tNK|G`kL}d4qhbM>UwHic=kDRb2vZ|sMvT6q_EhpyuE&aKLYe+! z@?yJMet# zzir{M|A93LNtscE{xNx1lyBzC;$5;X^K}X+zQ?W5P^Xh}yQ)UT>LhuSUnQ$zlVtgP zv4X-ED<{-qbv~J3f=GlvX^YwSn{$@ny zbCcPO>6tcK>gk-3>6GU)I;Unno6Q?UFk@mhBNqBPfwuXF|CA(I0sT`ZvocRNd8m!1 ziw(%Im|QMai{ccp#n_CgPS(qG1J7QbEf=fp`$>A4DUrayE3ekYVzoK@>E~p2nc=Tp zRn9(%uEHn2I&ewV9qN!TQy5IMDo@MV#p!>II#9~thuo?I%z*!9DxhdXCq!1@zy2ta zMhbh|*y29veT2BhTHCHe{GVra?3lyC&zY;vpZ~9a z|6l&|-zUlWVw24C4fOHJ#b&dvPA8LFRZ3?Y&_$|=+ir{P<|L`!EY|Bh`=3E8nV<2D z&jib5n$F}jHAEUFvs5cV$&4D%efTuBWJ+Ot0YcU+nNWfoSPcLCAO8V0rR4h*HvDLb z{&lPjp2DqQirpT!u)UYFv<qnM1Ek7wKv;c9lQ@Ce!JJxxN*-8A zaOfrjiIgp>&1A94^7rFktKAY6TA}P%QURT3+<^Um|IZ}Nve9J-d%RkwvwZTAFehF( zBVf$_G%GGIK_UKmvaL!`BUY31YjM17td5LQ2hizC)e+8j&=D^hRK`Bk*#*Nj4@f(Fc z1)jG^my6%`mAWpu8G#X^IQ2C?NWbaaxQqdthiP)LKx@H|6^e3l3H%~2PjFPASdM)n zL&ZN9wb045Ojomuaq{W{W;QVObi3T_K6EemiUaO;C=TANwsjN37_G-SAQroEw1ru5 zyU1D`cpf$pG!3N7rME1m)V?ejCs)O`1cmC|bvrNe>dWNa#R5>@EtX4=X1y*^1Kwb* z-lZiDh%Z5VSuRh#g>bnq#QTUqMY5LTLDM|J8jg33_^7VsUt=}@r58~mDF_#;1lbq* z17KANj+$Zxs}(5yRgTLzbaR@yDRk&^Xmj{uu{!xOIoZH|_T~gF5#B<8o$L!a0qr$k z7VArxmnS$d^5oZTwHehMAlr8zPz_k}pbxNuS@L#~)~&r8)BgGxT%N;Kwv2qx&buN< zmo3c+EDNU9VLpObuvTua+@^P9Ie^jTpI?(PlsNH}z|nx@o493uSpZ4Gr4okZ`3>!N z1}fP!FVCIk8#kY&)L2scas*FWqi?Q(iz=NiaRErTn*tWR#Vk2b%PDXj=$z#;cS?;9 z%vz+AE!)DfnBm?w8)FFx@nC7ASX>pfF^b-(!pohqvRz{xE}Gxsgc$>Gy?|zai%-9N z`R#u69af&yUsp~zM}uWlT}LAvcq2FMV7lj>qx=RC!c4)C)rXghE7q=G+Z$tPWeI{8?tljN`dDrrR6t}1vB7RLIe58nCH zpOVvN8Y%^7&g5p}fBWjyb08s`GEFXEG0IENdy;@=dDlkAdG&hiX4mV}*OS+W3HBOi z7<@2xk^}`&Fm34l^++d?qq+;R#*Hh?hfslDl$HuyWe&Z#Sk0E(EPpz8#tqn36KCIW zicNu5pbS2TwYAzyhGwP82d+SSziU_jyMsutabX7a)3a?T3L6VKu0tCIsxFFmquoO8 z6|Y|N$Ms|5ZRycrwBUxftJ}^<=O_n!Ji5``BbADIrcyexd6teuHWi~O=jkY$Sz~5X z3mcyq?{MM2E7&ML|NL`L4YabTy+`%tq0jbR*0SA> z*Xics&J}MK+z?7E8Ax#V^`^}8YV!G@l~9y3adai%_r-R4qb@fZefM4Ie4D=@xg0=z zZM4w+ir>)m_fi$hsWvfM4Lr%|O%u%5Xrzthh{l@0{si04x-2#Y=yT(pRpgBAtiZ5V z6{AghUctZC&=z_9Nxj{p_UplpXWlS&>no2TEa7F<~BzI zESb$SG+dvS`MjFci(fTC=k$bfkavP>%k2sYITlP2%V7wAT^5%|D#kSo8twj=B%dcg z)f{mCbF%-56+apM%Op-{ZJ6b4y}f%^4lb(!eS}Q~v>8IZdi}i~XD}l9LeH$=HX-g@y6z{Z^qj z=ie4vN7-Qx0{z}NFH7?6lZRiOB-I93_t-swlzUkeZ{X2lyUM;qTVoFWu+GZ`vh{T7 zD+Qx7{bhX(jttd|F zEJiy+0(@LTD?@E=*06r^%Ux}+yrpmkV=*g|sz^Rv74Oiix`Y?Y&#+QkXL^J68SC$c zbzRLpGaV;S-lv!AWq#_#m$>eGzcti3f80E;k)g1^hRTiV%8fi0lHKdp1O}sF6#DO3 znZW2OYI9BjgUgXdA}m>wete82S>Y zHcs8lp&VfMk8Is`I{wTHZ&6o2z{4ZG+G7JWa zsOS&H%1vtA^4>NHXvYLrLn&c^yv&^~7I4;O{&rDpD_9s%zrv4U|A&3^4H|8KuQu7Z z5sFF%6CLK~r`Y$-QJZd3VDfn~9*>j%vH4^9FFapR;{C?^4)ealyzg-DJKXz@@V+Cw z??~@E()*6`zN5VFXzx4P`;PIxW4!NJ?>jcor~VF7M5@|?=QO~*YZuLaz!l4Sr$X-% z4EH>1oETOOmZL}Eh9jZN`+@;m(agOMubvl&(1U87&k{OjYOH;4f=Um| z?86P(ANH2h40gI@0a{1>YGb*Qz|z2sz$Mjfkoa!f^MW|L(8C$7Y1c2{igeAgYrDu8 zG=W1z%Xu-{RS6l*GKVoyP4-ewYLF2$0+Uo5A{4%``_zTNjow2KZqjmm{(FODe9R?m zf;cLkpvAFX<-RGJ>j6&#>w=#qZ3W6m-v17o%Fc^>wkm*wz5$JEHF}d@0rSk@ON`ri zS1v%$A3d&3a8_pXS(?%j&9ok&(Pv~8n%56!2DENsrlTy)_()`g zj#4$9kLEm^>TI4Wp0NhO#TdANaN#B|w=U#z4y5p^c7wPj97YcGezZW|G9)^lcd;S6 zFc25T);xlx8e2-G=Jq4O>hQh2RKh4I%_8k$5RA1)`jz8H}P%+i27f) zjU?863vGp}^1(_1W#p^3%@11(iY)fTi?froJn=nb39`D!kIzodXEXm(43(zG-#q)} z@z>uy{O0A^$>7}AG0_#zIhDLuEEa>@1xYw?Qfn@ zO22>h)sNpjX>P#!SwRdw`|$g(lsG$i>u(AeK=qR^9)Ant0!O0%SzvhX*&koQv#Wel z&iv0h)4?~-9zFdbASZK2cD{M`_~GMkF@CNrip@Uc7WRc4_VUPFbyp zFhYFzF-)A=#{l~=lmJCIGf!Umr51HxfBWp^tC*pxB?Q z0a-Pv$IlPXUjF#~SC2wZg5r-)wl@_JNH@%%Kg2a)FSP%0rrl_J{^IE$9=-}Stmcx` zPr6F&wb6B1yuTWs7t3_z{61gSyu%M8>f!z3*)y10?OKZ;NHRB{c1>4pFK39#_uL>H2o5utcEZr(H|a%7A`}HtKc7=ymyy>VN)kzWL?xvmd^G3flunKdRf_{}OLD`0l=So`;a#Rh&{bX1F{` zRHU4G{&MkqewzGmMp>48|0wwsRAC`FJRy@upEUw-XgMaOj|z?6?-kirqZv$J zpe!>sN+}~Fo0)t*lUCB4Od=Xj!IE23Sq9d(9>Q0dyY}X?v{s^|1EMHeU-nkQZ zG_q5}r4S;sCRHL+(@ainJ_Bt&<3i5nbY{qWDuATusj#V~A{PeMI$&B;!6k2yOpF=y zv&A9y#TUtWnXi*k@jN;C^k4n;nVS|D=ra0j{KZL9ZwAd66jVUm3%*DWFP-g2w&x9M znBA3H_|8Kd9#t4K43EyXo8)AaB`2_Fd#b~UF}M+au?KAtgcIvxBELv7m@zOWa6F`C zelQm7?F-9I;>A&D1u@FVyPE6wSeB#a*}J*^0535%dD8XSmn)E{6Mh3{qgOXwvW*?)L-v7d5tHJEC7qTTJ&mSkMV@`@1&X ze>fwOPp|Pl^TP;DB@&Yk)9%;bXmHX=LIpa00~H2{U+7qm=}3J6wWNH@W9lqH>lYu_ z`5+Fu9X^2cRojeFnb+tGjji^tzj=YsWACr*ufKJ!D4=2(D)OQ0ivIdrm+A@4t^Uw4 z(|vT9F=N^)%09+9ro}mERFdispgPrwbET=4*wO{+y{@V26h3uM84zqX0tG(#l#K-) zQPz$rtkF)j{vvt)Qkguhy4YML)9tyhWJB?|W4L5#Pn(JZA?naW_a{AbrF+X^ zsNjUlQ6!=qcDApdw1FPj6rWFwE@Hs7&9DitT0%{Av8OdISsE z8b(!~VQ8|aJZqi#v1@US5oR?c?C*k;c1Y1ygXg8E0HLTA5pASfN;nzU-Tc$fV~j04 z@f2sYJMP!Dv>N;D#6PhC+7JPYeQ-|VQ_hDRoKEGY5-0j=b7JW}KV*NUjtDbHwp2LhrbsvS{uK~z?>Bg!s_wq2oRR5j^oqgC?o zVe+VnB;)ar+h^u?Wm-WooTL+3a!ScbxVV$#to0GENYbgpL6JnP-JsQ600k;0?cnAD zET>+U`Y;!SwP`3y5#E!zAk=WMkoROM@D@>PqkWqR1eeyfF9@=mnLr3`Ex$iAfuyi# z&|b|1K;vVS>Eq1=5||#5_hu$ggt_310n7vdEp4j-0G@K0mHfc)NB`+gMH|%MIwG+-p(%n=_&9xi9 z@e06NjWhJh?cKhT-%G?~a8G!s!8>5-W$|^avj_ zVUKgyBYI2(%#juYgcnwKw-|)RIo{NwdNe`+A|v()BScWhItNW}L>m#MI>O5x7SguI zQZPn32+3J{92jG?1EL@_LPqAJ?vESX)Vl?#@)fDwy%n7p%#Ol`N~@`S@Clzl^?~7_ zK%8{;nui!7y3He^((laV6_9k|B3K9aoMTbWr`HRU4>z6yT~7?@J;x&4rCWV7c*Tri zv~@eRzBHeirFFjY>({V{k@TVf(pk74kr5n+y|@O+e% zQq{`M;i*A9=m;#(L)9vz#M5{!LNY1ofUFzc=n;|Iomsa8%4X=mtXpwqBZyCD)~&b} zlK08F0Z~VsL+H-B0obx30^|YNkii1v5uK4?SvM5H1s#la1H6dfe#f$IwafT05bFkb z9Rq(9>jrS6?O?1M_A|nU3X(^38WDVPEbB(Iam2x`V|g*|^_KU_ivdzb800(fVn9@! z^va6?QVZE9F9t-TNUyvYAf5KoA1}s>$0GvDSqE)YNlPOl4{mLK2V(rywNE+BhcP$1 zZSxg!+Q^3+3aM#u(-+?2m2TrwxB+!zzuv+O89rVd%dwK)3+q*i)B~jt*6S^T2{S0& z#h7=a2B5o|2;FYI<)Rzi)u4*sW3V{hmSZgGmF{XKxg3)28ln~B_eOW!O$mGux@%y4 z=Qo1Iy%Gqfhww53u%NWPd<|TI3`bIpl`7(}XlIgYG4HMzn50@|=m4SKh$|zUiXBKQ zAnJ&2>`qbv*u(^6cPFV9`j8J5>W%0$h63M#q(TwQ4CfjKctHm{#f|V0zN4c_s&&mX z7>@+-S|=NXM*_O#YA_yYG0U&@_nw3Hb{?W@+N<9gAYnL&9{DvQn)XvDUHCOXvWO!^xA}D#lC~QW z8Pgs%{Ktpad~|=a<^Iyibk9)naz+flc9YU4hv1^sRV$jfa(59NlRn}#pXk!1c@K-O zItPx29Hm$&YY(ozGvCH?j3)OHFAhZk8&>z=WyX?d`{DMx&~7Y3vjNU%YXWLMY2S;F=Ej=Sm~&FyxjzDH*_O`^pR#ybI@}1HT21;V@R~EmDI1o+YrOX@ zV&Zfq@QAA;*kwF{*BtNJ9Du+hu!zVk*`2^6veZKnctqC`wwF5)cr3!`!P2lHytQU1 z0uS)UT}M6)fk${I$S_`J1m|Y3RBQ+@)nLie5MJ4#qOTF%L`10_X&pd#%SE459w04| zeI5ZvM7KZp$bvE0#vH+d!;peT5KP;p-jcPmU{1Oqs&;LZ-CF+lB>2&dlwLuXFqWis z8*SsdiRLyH;)rbH7vAD6ZKGJ+cBZPnY*@~3!qxCJ70RK?-V05IqL?uQrxC+a1SNw~ zRE&RXKXHEqMHR^8TNOV+x=~bEMR?3P$G#}4fW{%c9%aU$OKyjxs1kHvc5i{u8gD@1 zY!Hg7h!N;{3C~5$(j`J^#DK!mn6#(e=_3Fu-AW(fDI;iDKq2LT{26GxWdl%CApX*& z%NUZH8kjE;>U(EuYPb*)QNTbNF#=bJG1pYq!9!4L-1bHhykJ`Mhufi45%=tO!vyK0W!jL z*@a&NVuVI=n_q{8R62sPvo1pF_O+kg+FlNn{n0gP4reCAqHHTq5~Pu{H$eu9LrFYFnq{BZY)9(Jt$p=BDA0b&~;2p)PBZ7@NRV7 zYHOvvH@a>JXD(q>e{>xnIW+^)bwdoJQ5UH7Le~vu{)@=$djwraXpIT@_7l-7fmC9E zKyCy!+C-GE2pukv+u$8(1LXKJj1bBOzK+W} zVgRyjFd32J1CecmS#!jo*Kt7+oW~q!K8|byyx?XqvJLPu#)@(z*#>weBBVc(6XS)d zlnv&pMR;!XK%5xSWyG#?Bqv7rm@TD)s}>=(G5tbn!6Y9xklO$gRuRi$mq2d7v^3%r z&T)agKEyq;9vBv0H2Mfn(oD z*>#85a5UC))5?E&BLVL$Le0**zvdO)-E$KbUB*)Pc~PF{zZR42A{ZTx#acEziASeL zyRj^m#9vp=TNtzQ9^%8XB$wc5I5v+($cXH1o!Pv>OE}uIkO|d^%^N}S!iM`}^TH|_ z;l1d>=7rTHBAo7cHV+*t>HV>J1vD}uOUyBB9=+CRRd1V-iZXXPH5eQXPbnLbk@;xS zt(Z~~fsEaa3JNc%W)aQ6G$@0u&A_2e2e1ubrXxA!-un2GH#(TxmR z2rk5Mh6jKjFh(5VW6r-HYZ?IfnA|Q(H7*6AM%RR>UhM+_NtNl*vyX_Z-`v~?DI%Qh zv|GI+?t{Ngt%HyzB6}X=x@^YTe@^IL> zF_@UG)rGB^7_`4A$CLwK`$VCRXS>&}1?Sw-9gbQ}Xl zbVCPaz*vM;gN334yd-9@G-n9UBd%^fk^v*Uh>*T>C^`abJsA5%_?XitN3vglS24Gy z9m#$Xo<_v7bY;JY%OlRt#8Xv7jJV&eLlC-U@TtD9Aau(NClRNnpzr*QPVs6P^tpa$Om(wAw0#b?HzH={D0fo7< z28nh@X!D_C*ee6(Dj9(ED`uPtCqe=f;RZOK(<`Cn5S$+4X*n5~(<_UYp!Cn_T|#i( zC#T2c!hC?^

;;HSq{e56C>?u5nXvSrw91j88Hl} zS8=wyW2ra5Q#RPu&M^JmRoNXX*{O!rDF^&j_iQ?itdun1H0a#Ja=UVtK^U z-6huDVu~RN!LWn_gq}PS0h%4W_Sqf@g&HB5EyGAq-_wRPJ z6x6^`3Eetc3LLT|a+nwjq=|^RzjY4B zzwU5qIE(Q9D-Xf{>S6NTjTO#Yh-p2GEM*gK{|MjF@sB_A6>5oS<5f~evuy8S>$I&f=5I zGlWE>dX47RHKeqIT^tSt2l)hKNgg7a~SN@F9!R*q%V5(B%Fz26z*3ao4eA8sKf8S9~L+B9zkJ!s%;7W?8SK5g-Mzy^={k5^A{fzsaIH$C%Ua5xToDw64_yC+! zQD)mulPKDab1I<7)Z8EEREiTWd*z$}DVZ9Ob805I)%V6Z-Hlsp5YCC$VIvRl9H4ej zC?UlFd=r2TA;F?St73XMu4x2|IZfD^YZ?LiCLNY*Qo;p^Fw%GBnuseS@-oMBO+?gm z0IrF^hMNJnCL-IxytjZZNzBRcZ2K>)y^-J48WS zPZ%C`#@c~f)*gA+g)k$e=rc0AhE$sLNth8*M_ee^fiNSaZBM*;n=pqUVK_zH!zM}e zYydO;*m7mTZWZla0(9P za7y~3;NBu?C1NoC4MhlM^^p7lUWIJyw5iQ#V`G?uxInpv^U_kGB#8*f?aG`1SJ#0- zLo#O|Urd_*nKR(32>)IO=8Vqp{!d6FFz5;~?%(dr8IY-wLosIwDoIQr_#K!t6u|@^ zj5%W*f*LID8R3N)%u9>#D#EdMEWrhM&BQ* zN~Tf?%}uXS$$%+YOr-NsQOSrdgzD4543H`!^sN(L21spauY4IGEmghpWeEg4B6hk9 zUj|63RIhv)AX!9PO412Zw(HVNdQV3`NuF&k^78mY(X|j-w(%V$8V^NxE5AMq-SHi- z(6%*TTB%lM?n5`EJ&AcFC-)X1ts0b$LlH5TW#5I4LlGt-L)l&ExKx%9GXNb|R79Lw zQMwx)SBy}h?vIYc0#9P%r904ZfK*%#NXLO{M%~ub6CKy?>gc2CxCaz?xFW~8xjm8^ zw4e>=zy^GyRK(TevXO=dhKv!EQa&&bw*)pS);|xou+~R}W8UK7f%dQ%D-c2LeYb}G z@$2O2YPQ^Fd4|Wz(c`jS=9_$#R~77y8|Ql2R(Uy^r!%~u^W;-_v&^e6Pm+^-GaG;A zt*aFgbAjDDuJwMgNr(p%NVaaKA8&EJDfk;l~ zi}#=InfPb_(`m9N{mK5reL`5TAgtX*$IQpNh%5cD!=BVcsRvn`E4Fy| zTkBS=Nw=-oN(@%Ix2>1~hoPMJxfLU%8t%SMD1hQXH}|y_!${^j;xJNo%Q$1iYBpT< z5XL}R)csrD8O&T05kA$`GR`<*Mhv_oL!0oJd*}T-@*Ni>5J1qF^psr<-U!Tv8o=O< z$ifa4Du(DW$L;Q5@Wvvv94yfX!kdUQrAHdP0bY`bdr*!vcq2TG@CF=d@J4tRbJ>SE z8lKz1(r_TWq`Xh-G(sva`i3-=4Qudbm=-Q#o_Y-4cV9O_tBA-;e4K3@(RoZh5Zy6e z5`ksJkxp#_OHU9zDh;d+tUMufE8EZdb_hOX=(3M6HT4sG$Rr09UG^A!2#CA8e+CFX z1aKX3q48~Jiodx5QILpmM$rXP>h|0zRCYnE52>{v`m(V+e3(4)EhV0qJSGJ-tj&_G zNfZ4K7w6c0%OgIKjFhb+v5w^ZG;LYDs4E(kSiK8TR}^n+8IY(uN3OazqOQ4ORu4hc0ogFsKas|Z79v7Tjy3>j$ymhg zhMkGJhHaIZfr+|CBZKIRs7EhD+FITo@p=S32ViOJ5cC}VIxz(LIC>7~CgK9zyU=qe z!rH;&(U>8G(!sKZAbiZdt;f=HfR7nB$I^3vHxb9&j-}^@(1^>tkEZ7UAG0!d;nIdM zrghJd`()Jv(sLsPiEs%<(evol5(*WX4@JsBBeoHP$8dWdu0}edng_F_;Jtisf4A zo=9=Wk}(Nm|76%=`li;UNvm#T*eb&H{hjCHgpim(rVeD7ql%~p^jc$X*_|`?KFBc6 zmnbiMXEMxTCsGj#V^=Z^xSB+0>Bo{`&Iz@=f3G#*Dl&8&8Rt?&+^QWS!(EmiEKWt} zlATE~0_%vw9^FYWBHM_(q{oq9K$kJ*w>ywvEP}-t9*!fy2+z%6hq4ep;#k13BpBgi ze5OZ|V1(CdFdsF-N8CzvEC~j9MOd$gvJjF=+Bc-2VptLkT9Yv`30Qn2xYH^rl;siG zXpbYmh%O`28(DrtwpavL5tr*&ag0}*@6&AechC$WEi-){G~+_!HCWvbng!|&8jOs% zF_UybWbF$RDoP!dzEF}qqdalEEN9K~z=!c)BA^)2?n;!TwM^X3c= zny;`_)t*Z<+Uwb=U_<+lO6HQ1h>PpB?ZodmUc?oZi=rof2MV<|y+$PiBx4b$b&uzF zhA6J@Eh-s{i?~Ag7=CAnCZ>O9C*m?TAiqN+yX-&HCE^M_u(OjJp|vr0b9TnBK1LW8 z5y0J@-T^SzL)0#a$@aj!8F_Qd8;1Z+rr1LfXisp>f{Q(-tEb6vkUmQg`r?MwBNR>zz3ZB4bX} z8`gJfiOc6RGY#4p$Ko~U}ts#o)I(HnO%gZcCa(M2+t$7ks}E+!i$K7 z@<_sr@KW?jm=RJ*(K95E$|!Xg!i-s+BXSA`{h@7ZBNP#1*gB3U13HH4wmT~I5nfOL zR}tpQE`CaY1ZAdIKP5uyh*Xmu5@G_R;llKp5EHX}llF*#E(tLaQc2b0jf;YFB8(dD z#zlc)Fs-LJ4eRB`MJP-$)kcMki1L}9hOJ>xmga)><|%D2&d^qO@DVJJ8zUoFp6tf* z1ko(!d`Dj_PY5F;a@cfXc|ycY-#f58Xm%m)kL4-BViMIK!SWP_MwW|*}- zdMg3Z(pJiFq_efR@%Oq{?u$q2Os|(GBBUg=$IBBD zQMD&9y~E{+SVEmYy9-@A`tn39&a|FWHoBcrSH)$%xj<(vUKIfoCs~)de|@!z65bQL z?p`$2qJO3hxV)XC8q@GJJbX$M0JQEQX-kEhHu#LDl zqC4xBjF1@L-f^rO(P_lthdZ!tEP~m=SU1A+m{8~ASU188JJ|hy2(QRsu2qEB#0;D41Q$%A=Op69<7WBB#pr`4{Y`C^$@KR3Ls#zrdGTtK+u&4+JKrrX6b zo6L$Vhfgol%>z0nlvT|r7~`+}D9goaGjf_zjxdt=VVwT>aaC@^jJ8#Jo<~$syX(>z z?$7PQ)u@x{Vg+Jy-c(x(@*9RH6vKcO^fUmLd9^N9RsKMZ88Kcw;jRn~Rr@2n$?P&~ zuRY~s4K3;}V!b6SBCxS$y7IkfU2?;w7bm;Fv6-ax0|?A%cdJ^nVznt3(@DO1yC{p* zWe)Y*6r;sz5m6-pY?~;pnk2h~(YmZ2@UgH>LyYOM6+~Iw&^+%>(iv*<)dQ-=mia-n zrY#_E7n?<1K?@tA{4fS{1>b6*Jxq29BeAtfYAEBZSzd{%tIId}6^KIw_ljejq1r1> zjd9F0+`)4q10rrW7Z32X0rSfz&@ki$c$;q061r}zl=Wjs5oo7u3ZsWo;eTC)dMukx z7Ax33mdofiaRIk}aVG2Ze6dP5MG5uNocaZ`M1+9mb;I}pENNUOb^z|9d7c3XmHy_j zRFW2uUyDhVU%t(;LW*iX6jOoGa=zHWU3uu48g_RKk?_60sH$!5rnzWb=XV(H z-YQ>j&|A^kM@!VkrpG@%2bhWc3z=mN%f>{{=TBox&FfZNeqq!FZNO7Qcdyq-G zTrXC6Y!x-NCJJ;DjmRmBK!jZ-jqV67@VHnEW|l@{y2}Y;FgKtB!}yQIRmeiyamx+jf8i`~NJj zaP~2#yzOQL39Ps4viLQhq1J7g@MB0^v%R(2M(sm2l45}o@!Pi8q);E&!2Pml9zZDb zb@2emkhOjQXO#8Q8N|#r^BdMmE`9YXZS*Ss(D&IKukocDxpcm3JwQ3O5-s8rcA;JVJ z1yGY>ov#*7+vLVCleDx&CfF=4qvg`1RjxI(*1Mua$$(1SlFT9$B6;=T+fb8>w7Php z)tE`@2UJo<1Fqk#JiU2<)8PS%a4=u+_sWef<%crVgrFQNdq62{9?TD71rPP41DPzc ze6?B3+V$-{ER0&y3R?lW%A*k}*MZtj)5^`phSmIlz<6kA%+lFK?%5Cy5S;NKL_IzQ z@gq3>*N*@@B`gYP-U)$78d?%|Ad~cRx;Wn!+bX(^r8N<_i`CmjwU{pB`%cMF-(4%_ zXzKJ2c%zvL!BlSnfEgA5(4onfUlzYEa1ashhcGC&rU4)y6L3H` zX}<`F@QLcw|pk1sDH7rto(P%Q*16X2g7yv7j!wf6)dCLlEsR^w}hy^-V zUcwF>JN_^dv;}g#Sm(_a*p1U`PRwA{kySs(FLt2dsk4IqxR9V3Rq_d^kQDj()9*bDz}7MKZ*-g z_oFP%FP89sw-T#B&uJ?{Fh#YwOhKG_ZDan4Y)x>8UTiMoN4wR*2HI@a*L$6}?Vs`+7Cj5EB|7Z{SHh3PRH%V&B28dxdK){FVX+Uc=(m9zCG-2Y(RIZz^JuGbr2Pu7m?Cpk99%*a4^IcLOJhJ$Ys7*E;<&$ z2hne<7kWxG8@GeVF5Htc-ufpek-0e+arWo5i+?GAds1n==JgW}^JEYY~H6 zZPHEEvhz_xLQKJl29>^CFKbO-P!_;y3=G&~{aQs+d`fMQO#S+9^Zo&a4)!Zyu!!hT z&EDi!i`98p?WLHAEP&xck(`p^Z(!!misP3PVkV3xyNCycL6tSi9OlC+TNg0M#^=SU z7N`3^@jS(kFTVTqv*f2IKPT|u_=9iC#p=2)jR5$zRZ9dbP=uVvaKW2}pmsTx1CRC-@uQU7Vj|Fc^^c{Qc(RXI6h| zh?_dpR$>&-|8Humkll?&qvY#{Pro}&UcuaY0ir%H+>(XE6e~rdbPr$+Ie?sq`XkK*Z)dLbTJ5w%rX!BSWxa>>nU-r2SYH*in`DVL* zpvN|_v2u6Uw0ASCF(zp>TP$Xv3&n1X++_=F5p9C!j0)_^gfLc|Bi|`b} z_^}L+5z84-x%QJIZaHgjHCQA&(Kz7(589-|Us>UtEH>i(YI^ozc+`mqQI_RXmyvnZYg>=etNW-~)aWI~6Yfp%-RVRKG1fuKojr2`+iH zsX1M?n4*L+p~l`DtYcfpno#Q;;^9Ch?KOT<>Fy?LMcD_$!8)h?>zt&FMS;~MPDmMm zvQcu)(()WbvKbH5ky#yub>TcD0XG(vJV>^01iW3Yi^@%98kl}k1U=jz#rff~5Wtcm z7Ar#&Jre;e74&U2#Pw0U&)tuwgB#avyvaqryo{X*(i=qe5qeoSq7L+LI1=XLC_$*T z^sgFpRtPRnDuC~T3@<$^KWp9 z>!{IJhgaeZ3rHojiowFLB#im+tE@^Y9@GBotMwwf{gsGXnM&7YG_bcQE#r{3)2s9R zvT{*Ifi5s{aMioT+=cRq0G3rzbLwph`}xXsXkaVWZ5!70(E$=L9c#)%n&cX0HX23`aZQ@x$Oy z*iM&=S&J5>-MMxz#&6qvn?F!vX#-=(9cA;jj)Ryif9vWG(?-cw*vn$}CK}Mv5KAue zj_SGg9s}Fq&GRx}&1$95_>ISg2pYsW&)=_?<+jE6i~jN;45&~Gn95fV_}EIn+G)ae z=v@>wt_S`hVHgdq2Ng`YMZAIDA1}W{+icLhF2*V_hwF&9fJhDmb*A~^JX$gybR9r@ z=a=ak!`}S?RMZF-8kbcS&AN?CvEdg7dLIM3dC+HUWN=FWTD_gs(cJzh5NZ?&?ReE& z=MCW`0ANCsqPkkm98nF3ptTGl>;msafKeEuLKu5r6*|5&3rtNLG^8D1Ci&YP?e$Mj zOH>?IH7nM6wt+vfl7TJ2(jfJ{MtobJjSSHy%YuVv7s4+B^VIU-_KygQOUwWkJ1H#J zv;}Zkpid+kFdPB74My_$3gYF?o`njqf11_TL;6ubd8?LtgA57qaqB&f`Z3UCLCvZq z&dEv}T8wU@)rUo3iHmPJ6?EEew&eggd6QTA+uZMGLapgt~OQ&R_z} zF7nwM2Nhb)tqI2CZOcWALDE_p8W`JZR~zqggPE>EBVso=L0vNG&<2M6Jl!68Dj*pPDqRi?wC5Bza|C@?x&a!DWoLCJUN5Ngjvj2I{=R2K z1UIdB`7}&47?CPu*#Usb=!XoCP^3UDV0A1D*>l z3E{cZREWF`q^KCvMnvJ_cXtyv5i1op8p`|!Ac(uscj@= zHxp*ecG5qIYoh()%IaRH6U?mUsxT!f!% z0iua+gT31!QdrOxSS)l09|U7ADStHXI(}Nz@riNBwTB}Xn8lB@)jGZe|=$+!=9f`$O@!N6jDv1Slm(9ij z@BJpR(57|Ksgp)HtE->0vF2!on>g{R60@!Q!CU%44goq=#4j8tM7IhTmu&1BQjvBM zFBcWA(b5EDb63u_$yTR8j25n4F9ghp42cIo5z)Jy)mJWC_Yg$QOK8Tb3*X?XBe1ig z)!RvYAzFy;g~Y=f&f5o=J4q@r`@4UDS*F_M-1!fp#S=tC46)tP5sOKAM2lAM(hk;6 zCK!pR{PT z?P@{?*LXz9^2=yaDs)gYM7TQrK{TM@q@|YV!o?&ZxY7CzP{IZoA_C3x_ys|dR=A5i zU2ZO%X*(X(nif>IZ^yz>L)+k@( zBisaYn-OPYp;!RMs0f&Yg%Mr~p*06A-$I+FSqHsDO3PwU=gTyH!<0&zn2KNbhGv0^ zyHs;Q5xecZQaAb~ZUf%P|D_q}U{nPqI8NlwdZaW#PInV|ms2nn*yR|F0j{G;q9s-& zsFd6Q?xL9BwPh`R=-~KR_;a8zMKz5;xiL^>K(kOxk(Qd?M4RmIVu`LZZ;T-W$hV8x z&1`kBLtFpGrFdh`mTMSHy%Q@^^O34+tV1ShP5r8n5{+qzu2D9dbasK6!}0Jo@w+dT zY>eK%2CDTJ#aRHNF=YM&2t3YlQDm@Svw&4rDcLH_WS6#y1v$Oajaf08knm(aro^xv zRx3{gZ6o|LOV`dCZMqcDe2Ap6fCB$uu82SpOYtoqY<%0r#$5heAzbwLL*Cu3fnZp-=BmE}{L9e4hS#}4T zU1$5(D=osKe}^*JuJPhTce*A7rgQ|@KKO|Q8cFgtvEJt826v!9^oEwa@D%==)eJXo_A#n=;{g8)^RS(>hIqr zw9&y~d41i52@OjtXbXxvY#?k1!v?!mS~DL?9&EE+t8{P|O6kRF{_IfeM|A0R$CnEj&jaD~@R7IR~iA6kacpCfWe8 zGt#Jb6lkhhaaj+BKujhx&6gPX&?dY4W}~aDtc-wn*~+3Zoy#ZG0Pb8pOh}GxhxbQ-?xAYz;Z7gnc zivDX{mt+sB$^uo@GT3;Vw89H4gg=V~$L#Q->TMTpr(MjtkStDYc#t^LD&{%z4-863 zP^sHTjPtSdlWagSLB(l@(#-*YVJIR(ww)d3Vv8%B^aliS%4m=Wyj#3kJYZ@p7ztpQ zrjB%+t!AX=;Ow=6ZBqG~+UE-1(&>`KN#UhyRo$S%K%2ptAWanO$aQ(8Ew=Jidnk{bD+)F0R7Mec6GNWLkOEMG%E<*V5uuabFL zTqdp#n7|OJR|0iT@Kj;xJ_pk1DMo67Nx(Z)ZGUz_*+~x65+Ks)lyEx}xe-jT8JXoo zrdEsTbe?Oba@dPf!E=?Ssel2-XBnB#GdszEgiWgSZQiYIkj`?Pc#mC;FBY4mn9a5& zEG9{@N;dG>x?DgjTqSSvtK`ET9znxZqwf~0?fcPgT#QC`YPb|aWY(lgWNMnpsm*61 zlNlFsHm5T~=2MYnxt%+S1B16ZRX_|6%rvjQ6M$E*5E7-@w028NHr;|yRZqo71Eoa|-E@-9L zZvHjT_)M@|rs+&hQ$wU-&^=lSN@f(Zjq+(~$&`8umr~m!2hemv@|4+J2|b+)3EK{b zX_~SbCHYJ#ZOKf|B?gSo>6}YgSL8esbCu7iG?|t40O~Yqn1msZ?!=!^Q%_+p1kI_$ z8>dkjPJjzwuw?VAigAtZ-Wv}ls0xm`=Xoi#c4<&yCrxlu_B^L8!%SYUay*%xmW$<8 zQa}Oa!X>r+_SyGO&Q9EI=QOiAlSGZQRnw6$xgAaOOpH=BRg!5~mgn{)nc|J%IjREg zw)15%O_xsSIy=Mu-gd7r?l{cD%oTo37LuA-;=eY9TZT6~d9NJ!uAE&NX_cg~WX|*J z*F5DJ>F7Xnf9%jkqaFHalxCU39bb#%&&1kXq^7WY!7}{<_^DgdU%vb{*%iMlaa&u| zJnu7ZaoKGV*;x$X_bR>2N5u*{6{?YMMs-FKc&^M}kE`n9agE|BOIVl3uDPsgJFO3B zCcA5OpokJf7%E`3Ndp2+-CXHd6MJZ z@}SnZ#K-s*tQIp+-tgwQbXn~sb6T?p+;U^sUAy|<9YlIPcAl~Fw7EFgm<2l5ch4{8 z`^N(rp7D|KI^A4&A(b%QA>G%TG6z2Kc^KZ{RmZpC;rGRMdZUzfK`q+0B9UU{W6a$o z7TJO4yMftf2XNzVFL%2pJUY5D0!Khs<{7A?Bb#UGNMutnnsT0wvY9nzHnkFT|8Xs) z2VH=oBBSf)rrvS2OI~9QUL)@7n{{}7sKjemi`T9iuU$P}e^8Ov;hNz7GJH?PjvHew z>XHNXnl;YQI%u!ms_D=@2vgXlUAS`(c3nC+R}3A``xk2~(J!bJ_TP70FwV`L`_5m- z#| zMY%X%to$lY?O+?%kBu9zl|kVY!H&M6tTm`(CwE-YKhwt$>oo9T52to01{Pb0!U0IP{q>x*s-;>G25HHI5U^WY@cIoFZNyf z1$G}{-6Ikx6bVlx3ak{^J3V8P3i0O=`MHO?hlj`i^MC!{f(g@Mt|RzM?v9l`X1=SP zU`CnmlZT6<$fon8SWKr`k(_5mIm)NU$)}<#rRlOy|NG6&m9Rp6pM3Jk52r=GnEm)X zwtqZHPqXA%k)MySpRAyoOU>o?$ya$f&j!iEd@@_iv*f`kkV3UkjI-~PZk zV8pZed{!PG9i5KmXNz99pHGg`{v3KPk0PsfMo3zD)Eno$qe!1nN9&^=RW)Ma(W|Ns zQ=R8oI+$eL$$%nD2`Q?{1G>gpTJAzp5nGf;^YnB@+TV)buMqXWtvUyVGi=q!7 z9VDaaFyGkHbL$zd+M{LFswt3=mm{P#`E&`j{#n+4lP~7KJ6E43cVB$<;F$A)_J*ZW z7^~SpJL#BpnKb>ObUmljp|KXe%Dwjo=d8z%!@PVQ(OZc3@zqXM$}WUO<^QF6@!BaRyuPpYvWaO(%e^SJTlvd6|{-a&uG9 z8EjwQCr^MLkao$aOnMo9_dzxTguwIGy}NJ^8_}USze^YXndJ0%4|8>{ZOf&P7_#^WPS#^!^4-4pFVl<`oY&< zKmG3Ys~0D)Up;wo^6=I3lh;q4K00~*?aA{OP~r1=vB;8adOj-h=>#>ObLc$njk9j@ z`@BddIba@*r%pjs#>lBK_Cpvtg~GEP9K!p&IA^zzPS-DkT}fd z{WC-de368cwJj<%%i6Ge!aRmB8?t*VSzO+;{Gu}}^3x(Kx6Iu-8=8!V+d6ka1A@*M zJ(%cXoNa61nTQRtd8bT=S!cD?)RP3;Mwm8iH873EG_;)+tHrJ#Rx-uxwp|ozO4uAM z$6fSt+D8$zGxDJaDfzliyxi)(1W{R;PQ*4Bn=5HuGy;O4TrLj^#!rvV@<~={Qb#B< zBPH>lpHe*nz<^?;y z&Yp3H{Pl88bmuTcF&f~kzk2%3$>$qp`x-_4k%cW&%l@zf+Zk9!+-MM)N_Rkz4E>M~ zeb4w$VWSzMoSvj%9Zx|8PRC&;z?Q?&X%b~~Nzo@QipK|o!F3p{eGuGvU{hJ{K9`~M zYMg3}v`o?jp<1U=#ql&pzTW(is@^)Cs`pMO9SlOh-79ixu|bRqNX4)42$b=FQJhb| zQg@Slcr(CxshQ$y9Pk*l1Q58RpO-^iEhp6j*mdM;j8N&=eYT2$rSCwlosFg-E?$*L z&8L$T?9}2@{8C66sx=AVCj2pqbSJ$co%YWvMWs?gsvkoGYB-RiZaLh=2z2BKIRws~ zNjjR|ff0gOkX;77=hb&cbN&6{Ye#Yx7d(yxy`vHYU87o_2UV#6UFqU#*+*GwQ&`s=_TZ%f9}*O;E>0=vHg~ zMO=u#$Cdak)uXU#+o00Q)oML~zRCn~mQO(j_0K@SqkINbfTE2outErYmQV4seR@Cn z6t=4A8~ANHpaubLTP_Fr6fg-|;0$%iPZ39;LocE-f-3Ox3!|#qL+-!{4wCax8b3(I z1>}5;LBpta$`)kvwRNtB>`b;SrxR9g>Dx?DGOdx*emw}Deq%@$vRze(gft(|15GBz)wORTU z*(3)W2n8@O(kTqI!ROIPRZv_=&2@wA5^5@tCTk)!u5+-U7xNs=kbZKS7Cl_3NT00$t zawjk{2oTUPi|jOeJA-;f_NOk`3lC#2#8}=6P3}4?4HvhEEJfad?)wYSiq11ou9Hsx z8pgL0_367*eVY8;-z6J+0CgAIp*RnXuC@86e@c!CZUL^hTJO7f2=7021IYh${HLQo zRXG0>0v-PBW^d<3dIr*RlOrF=a3GYWXm4G2Ah^+zV^7@;On$win(yGfw`eBq_Z|M{ zQS<;m9{vf17bt$4B{CSU%u9K*_1;KOupBF4N0JxiVmDUnE{gGH;)GyI+lFh=WaX0w zuU>xj^!elOAE0mNIO(M&$U@59ppE5Pn+5_)8^?@Xm$|=3i@Tu;3$2A?B<}&H!4jY6 zAga5SVfDF)COxij9_)JfuNiD#S^On_qpWjfX~XZqS@1bu?nJ-oX9LChTJ(Fpe%2cZ zBYRQ{X^rqh*f>4w3)bVp92eOTeen3n%A?t0JU-&?Xp$A1&N{g23I|&T%K6_H`Q(Nh z9`*v;H%fx!*U1l2AszmhEdSS(_N4RMqzT1NIkDXq4K)q->-Nf{Y>sCG)0IV3`4}j( zJ@jGq_CtwS)73D*J1!(J8-qTzxhpV}ni!4mi!@-LY#L5Jx~LeY7^ra9}YrR z3-l1l!_`3&BR33*rj|)glenB_XXTiJ{{*?z4_)qpP(0^V%A%_oPhCsUIV)z(lTYcs zKJAjdfnNK4LX}E}cFQ00MKBA1^+3g+gD?es^sAExk3LPxIcWCXPy^Naah|_{s`+9% zxQ{N048FlED@JG$Y&wUnMp4Kum7@ZUjbI9l!kSO+haZ+MZU9O@Ee7!0WmTyRWnnC! z4;6B97>`FXf#RU62dGbG=u1Wg6MBr+0?^RATwB^++JA%Vn=UP6Y5=8Cc95lD1%AKu z>0hMNIrbS%Bj_6CL}jeN&jqY9%w~OtvF811wwKf>z@A_@4DzJRle^RW0%T?~hK9vG zoYW$C13)YXTLmEN7E~h2l4robnu6khGXypSt>YtmuRxCR5hR2Xf(=SE{Z9b)Wb+( zGynslDodb}Sj)qfD-$tt)>mke0|qEZZyc$1r7V~58IVKKor)+(=gW;J`340-qQT(E zGRdX1V!&l8`@O#ExlD1_GrXsUnQ`2w8kh=b@k7qN6GPL_oXdt_H!8t}^8?=Nq4AtL zDKh2;1VOO#2-{%SfNIV%M#<4}4%p>Ubb>AT`QLs;^r8iqpp-P-YB%=+Myth8iT^m% z`@n~KVQr(ycI{Aa)5$W`T!r(M8m5OTA|AmEvT}Z${7ZFYjDwIZR6Gj<+Fzq<44r<- z6!uZFfk7?=oemlIxQ=6?t}|*?_cg2|utGNNU%sBDco2qkp$hY+dznti0X26)RLx>c z#p^$p`SkU01oC?MQx|_Cn=9Oc65Io)gLIrvw-K$aQ4RRkDzEf4ugCL>4a#gh>{M%+ z72WQpy;FCk_COiT>@EW87(|V`m;CVPG`+RbV`dqmpcJ8;wWvQC5<( z*OtLL-^D?H!X*G12bNzyTj#d~R_`aA0@|s`#aUMQ&nrJR{JX4_gu%Uy;~4K{IQ4tB zIvs_7=Hqi9o!^K52Kh5cK=drbRN`OhApFP1E-f=GL@!2IH`kO>F~6NLV@+!@crUVG1ky6XW$>Ri zH`1W2qDKB7JKqQ1`NG;dX*m(jaxJQfY_BnDf!g|Fl@O$tB!S5Dt1wKmzGrN0A0zh2 zP}MV|V}Qa%dRcbRrA_Z4v4ub{T@tO}4%#B+yuJwstyF9cs);ofx-{UUACom~j0<`8 zf`@xcA~`jN?%T5yER^}6yp9d$wLB>VK5HQ+wu77GvBT(=phG zQ|RtITWxR@S)f1bTi&TGY!+IC1F#2ujJCu!uvYm0Nhf)0R5B#|b4x~A2mV?R?sDai zs@^&<8NprHdX6fNjs@e!gqk0xWNm$s+L=&L8xh^VSF6{P36#0&^XFD?qFyxAXY%> z-*iDl;7OEI!9CEt|Iqc(IcFO?G+sM~N2w3PM?3>l9Cp#;xV?lA+xdFQFJXj^G@fNl z4131Hzu~U|_o>f%Vqlqb&T}h_9i(2Wl(}>$b+4yw-|~UMqt=1nNQPxHBI`Y&_*ZWw z0RHe;IfO?)&-&R2^L+o^+mV7V>K`9(d$|XDOi6qu~^f z7}80ym_i2|JaWQx9ZXfT6+=TaE1%Bh)&_P2XEsTi9^Qkc8X2M3bvYBN zZQlO{0SHi2Nv7211JYQkLCk-j{5JVr zf4s=b30QUU^LEO?o8QSUe66^2TS6XJ*nno_P!3i0-zM|2PgMa`-E5>46>QMN1m6V; zv600C=%!G>h$<qJVnd0{Pe38o(9(ZED8xRU+`8dFP0(eXK%I^%^#Wz$1}I=$ z;7zcA?a0A2?tK&(unL&&YU2Si=-8IwF1wRl2#qZFJIDoWVk8r8S1`ne;n1D{L#$yu z`+#5weJZwH`#^CEieHFTqe?*~wj04Pzno>|P0$b;Qp2eb=S{&OR?$on^lB6h-fn_J zbbyAjkLQ~&lU5x#49HuD#(PZ(n8>bSAi6LVX*Za}h9L%-fJvyqnf`!EYSpLYwgF6> zL(e(2><%3i7^}1eChsle{*b|;EjmR5WI`0<2?j59X83oA_2^>jP8g3kpRM}w#Qvy2 zEd_;23sTHrgzwc~$K@n{laU>1Jo0)^^la*az!`R9wc#93O*N5ix~2lID&p590>z0(Cng68PnL%gBp4v>p5>jWLX zNTMsBY%EdD+E{8enBbLJtR{c?U;j_|x1YqnsIB5yjyW1m=?eKL8u9_ z0ARO-tDV36{DDK_|lF=3J8DmWG1`2P&v_qh$a{rTGL5)K*$F*z_yOzqsb z#=B#F*H13jzk8r(S>V?`KL;-b}cYzCrl?aPs1tE$lM6#x6^S5n%Vx48#4FH8WN@zZ*p>Y9geT zEF-$6j%BueJmVB*qu8C0L3eTu-F5J8f^XP=9ozLAy@T(`W_%}g0B?eD%Y;(fXjAVZ z+}p0Gkt>!tJoRY^al`5$t_-JQ)MKd*ne2AcZK9(1;WV5;h^ZU^!AZDu5r%;X=U{9l zl_MsH*|ZFv{RWylk>;m=I8H$JB)>`E`-EM7l$8l;QJ*CbDncIyI`JMuUd%HnKwu?l z2`Xrr;E4+~>uw&>C`q`9&=ersgiG%0+Bw!H6me{UX-vefig;}3Eg3x%Yk2NHq#_>s z3`)rsMci~T=Sr}osZ!01-{LRCmD|r<7)P=$u*LZ(bO>FMH9g_Uuy|?+KQp=%vt!_g z=O~yM{6m>Z^8D2kRDOLFR!hkw&E#BxR;`liXU`_hr%4HjKm`a0N6;vp;5MW?s-)4s zMQMa8@$1*YynX%psL0Mo*+plPVtU82i$8;^9K`9}O6-b>5J~07sD8M2C8`sy@R!M& zklsiRh75B&`>Q2qjy6iCKOO;}RzioON!HOM8_}dOb~*S+n$koN!(AOQ+%=;{OB<3V zZ24edg-kP$$+LVs>R(b6`8tY3GlHySCL~>|BWQ$yq4Q1;uaTt%MUSdophn~Xm@(hp z7IH9C3>vfW$;w3mdqPG~WNIjC_Q9+a+~H|Tb4bJfGa%@(HT3wV^f--9=URGJ2gL*( zx`qzZhz_TWW_BA}g&Uw4C`U1P4IQ!}9WI9N5~ZHo=skcQF<)Tvd(`!krP$_iFrUm& ze;8be^dP{KyZ&C?(3liWRh@_bg^YyncFvihx`(3r0i@JyHNXpWEa1hlIKZ+V)mC$> zS8X3Hs%v?%t+gsFEzmG8(rJk?*kE?J38=wAgGdKi0##`J}u@ zso{YXP;97A5vT zbYQRwDgEZf4W%k%qOCu;+eIlO&h6q@L0{UUm4%v`5q5zKN`&EpD#f5pE^c2d!+BDy ziDW3DiQ{O8XDGt5ZFZcd5qfPD_C!kk4;oEAVmmeQD?6u1uqTUU{lu(Nidd4 zNv)N@!%nwVL<>rWihFfKTdsuPDxx*jx!%&bw`PlYc2rx*wV=eaU>n4<;hUwi^H;*z zzbl$qjc5*JXyeFF19q&?YzNUC+bE|Si)LuyK<9J22Jux5L%u9RG{+i-*$*h1L!X>& z4=;zBu0G|`EBy8`>2 zDN}qKC{kjxJW>qpMOht2z}V${kQGJX#D99vNW-)qohsmXWx~DoJuq=joKg0&29ZylM!;W($iyusr#SG&;G4Asn z-`Qwdjs|$+D13Ij{e$7~7e$IOpdo`}e}1>y?cVEBqnU8YbaN^2b2!ob?kt;z5iqG^ zf}bnK@Ap%fOqo2-V3aERN2IjG`$>!O0Fx135rc_7Dsqm=0ljxLIYJ-+v6H*^lFxse z{3BV>ns9ym=-0_`RAdP(J|-j$R9A}`>HYN))rAxoaP`mkJ|imxbioL?mEvE6JCSJU#?ssAOE=-Pjel#9U-yGqmim?Bkoy;W#B$!VDdr6J=;d(d=B)kq(wEV_xeL(}OhkI9XpX}M9gy98A57&Nde zG%dHnGHQwzk7ALp#yS1Rzz-W%KnuT`3tZ$1y4W7a}N0Z~NZ@`~GGZW75!=Kni zNbXPZCsg1914?$}Pi&{1B&Uc;F8QJJC#IB?%+QliRm@bI2F9NklUcb+x&U?Ii(`Z? zS{%X9fFjH&469dCbQ|G)YpmzAb(UO1P~b88sXf+nmT8*x9DH%!g!P~1kA^0Oc=;PZ(<>wBeb{S*=|2u0n#SS@^X)>@hS$`{QimyRTZE<~T=G?ExRKTX znx5_F5ydX_7HImwC%ZzEV`da`W8jR3H;oS*nw$xamzQo6nlOejH47yLZ_y2d1BL{M zRH`jG>r@?gm;Ff0iC^O#g(xi12nbQ;FgdgWA99R zTDW6M(5vs5SFTN_bzITLBIfi{jrJ3~3b>ya5xOv@c0X~F>l>UcK|AH>9o#92!Gwp- zv`%wQekV+_?OTr;9pu>+`jOG_EcePhMsqni3k*`_e9x29eB%Tx6IRH!c+fHpYBjZ( zP#S4ziwAK)&m>Kb9}sG{o^B%e2XH zFrc-TCdcz8XJ=S3T*2If;{?j?8Ur6A>V()`RmFCS)bgN|hkJzw&T+!U>}0nJO{Xmx z{f5VCAxT4f&~&gj@fvbu6Wb&Aw@zS;;<_Fl&s;%8ad0;}4C!kcL)2lTW#f03XK>?+fxh5aquTN_`Zna>3G?ojBi=?^Tl)y=pCn%ckkp;)37Y&U2TICVKnCiy`-p8 zx5$ZB#=CT6sYjRjR!&5ZzT3;e2wi|oU^Scw6_mPxJ`3%fpr@#W_lK{~g32l|qUlzl zsZ@w)qDr-tZR&9rQ$#A79HUUhZ2(QLsJVU_JpRE!pk~nY8jG1?hH+>YNx1zd42HE zG(z&*hNiLPwOS~(QutP(X}zrZZcZ@K2%64#u_rWrP!DUM>CFvpqgl-F#_1^<=Goy} zzZP>Qn_Gsa5vB$UMhf1h)dvGQPAg(YG2cjP!p3jRe$X_U8CM5QgGmr>0!_n=p{Hm( z>MREjO&8vsyIp8HW9bbYWRGI4LetY%{w>IN-Xt>#Ry2H1Xj;yNt%IiJ8V|>IhNk62 zw(D3boILz9-e72oTZSfBI-cIpLD{sTO=!Y4DQO=J$avlinkER}J)r5Bkaf4UpaOcH zcZa6q_>MI-MJRQLf4$tI%w)GLH65mtHS|IW!`hUZj!WX%&8jqH(%TA6sA$A~Zd}Z( z#OgM5s34`i2{e@<^Ep8&W(52Qpc$XqxLs&s9!PqjY>3%t5t>%BI@&JCbxJ=;@i zau;4lp`i)ejktkd)8YFtITlnHlkTZE?Z zinbB7WvY?X)Hc{KdqL9%b+rbXUq48=!QvogVP{r=V~l;R75`l3mw?@w!@X3k(|*=xdGwR=0PP4 zXlnEeD0QX@G>r_ox%Yyms=vDe6@?Lt#KUxNiBtZx;X zc)72J9P4x=XxeaWv==mOxbs;9O&bnFcZa5Ru0|NRaOvjYq3N|Xw+&726}^L6CWUJg znwDANYsd-t^tMA2eO`N7=$0#47kT6`8~6<(kD#5iwCmOiZ9f8C(@Jh>iRM6^YW>FF z=yaOnYK1UV{20@G+Sw4RTBNWrxMt}zDY8Fj{ji=RYNj4km$qJT!O<*z8tTAEQLVD0 zuX)}C-G|IIIRS%;lI#805iAQ?tHgLvgl}T{dFALZTmZ887#wEa5HBXjQX8_k-^PWG zadw(b|G1*{T$ zzVYStyB@GygN&M@lkHu*g3%0p?FvdmI=L3P)bqNp@42^)xD>g4Pc8)@)*c38LI!mc zF7=*~Zf(y%O*won1u_kJz0)zNgEal?Z%QJad1R8G(;Je2AtH*>I}m-`v4Qh8mMa0@GV zcLRe}g4#s07@7=+|N5oFeD(TvA7;fdoE7v!Dd|L;&~(ak8k$aX<(oj$>F|QNlhNF4fc2@vB-zL|A$|T76Lo9Voj_$)%7qeH87k1%9&7mMvupw8- zu3QK0w5wThr_|vaNO%$GmaqIf%@}=O4NEO+G5!z+j8DLjr6D=xR%k*+Z{1$d)J&+l z_fW7xc)Dd*Xlf=o`7$})(PHNC-AbB)2)T7=*4(j&Qlq?W6`GdOLR@2TqHhFEOwqU} zG(mONwb1k-;mPjM^h(fE#Aivx;X{*i$w+$?Gd=RwUz9Hwy~(JGkeFX0;s!-JoJUh4 zyw8)&(&7#L4qqs6&AMJg&BPnlnghxzCK(A0D0OXS$b$t1wpKK6<ObTiE<>aJGx8!{qKtNf9DUzg+!rZ>d_yNun(auRoSo z>c(d3Bp&_qwiT>?I_+oUGD)Wi9&KHnYGIp;VlN?MDDj z0oJIxWoRlBJZ?lO<}_~;np#LnzTg6-s?oa?9u!1t_R^-jcAD2pO{m~*kYU~91K;Ga z9h6#r_}7DxOix}9hEg9=Xnkb*iuNpA+-OH%(RdM_%E;&q|}k?V8x$8YyX28lSBe@xZi<-J^do>KY#NN7L`?nu z%N$@;I!SU7KnrUS&#`blC$stG{bZ8P`)A8Fe3M=F^0XLGGx0%Ew4=g+G~{Fs4J@@n zwaJX2QCV6xc&=p=6~^Kz(VqJRC{xYwk}$saN9Q?TyM~H>zo$T<3mFbMNoa<=RJ%vGLpxPt_&KK8;lH;C zWP;2h^)T4>mDUGveiPR`v{SU}5Z_2p2cLM&qc6$c?sXFw z1Ctz)3uU=AZSE6z@r_JHg{9i7CeXB!nf**%=puwv($KWh+Z(7b-YQ9k8TpWz_G5si zQE0y0E;NmIHCV9P_*S8bxm##*Fb6{;Xu6PIc`s;sS(ji3D_E^=08P+LX#)aEz2S!s zO~);fCxUkwht0PAbd~F2#Mh6F;KQV_DE%B8Z&qQ5i8$v?JJ~D4Xs{K~i|UzbFLi7f zkaI$Y#?o*(sh$^OMYqpIsL+Lw^z-epLbwdE6EB8#nC*(5qV)d2L;3@mn%`Sk#p)<;#e9Y#kcb!7mDfqc##WN ztHB5^g%vm8MJ^4ar5{&Hd<5GB*T%A2-X^#aOwbEutQ4(6(+X1qO&gN1Hh`w}TI>l; z7jhBTKog8HCt!$D!5Ei^4^2$QdD|Kcf`Q&oF#x(%XbRz+rpbwKux*DX*lg18 z2Th^4s8g`=Ff{HB;37(BBXZWE#&%4MrL6Wb0SCt1Ol2G)BCfm`*GsVp8qH#44p}Ok zYXnVW)xOZQA){OkG_5iICeU<@mcUTzwD}028MBz*Dl|Fbk6b9unQ2qQfdNB)GqTjl zCU&9bpegO=KGYcJSf}A&9-p0GNssN6_w*E%bBC|tXih#b8cMBZ=Klv8j>a6wvMmh< zM$k?_VQEWh0`Q4iGxISBT9KLulseHS z4|>O`{Bp6>m?pNAcY@3P6jtaWgw>Ng=wUzBYDGD4HB=w& z>GK0k5f*?QIy8Yy7exQ9LQ`VOPHI=sBG=ioH=M8&7(^yd} zH5C}A`XMAQ3}a)P^$0Gd8TklZdbeRxrZDAgEX)Fw2;VNeY@&dVmy)aWPJ51N`= zUJFh1OtKq5QwybO?$c68b@*VM-BciIVE zSmtakG;Mf(YuQ973lHNQJT$$w_LiY(7$coWSgILs6PgCyCp0-+2-^gjh8u47 zgQfv{tQML^Nb7F^O(Trs^b}*l=tGC5QG#?c;73xX^?0r#o8`qEUAXfznDsP)VZ+)K z`Qr2pzRpSb8#rd3rGq3NCRy1}XUoMr%S#0Gc-%`jK9?Hi+Ky?mWYG+SE)DlvsQ4tSJ-|=(_qL zt>D<{2R#0m1pFUIp!dev1*|sJ1y@mgCP+~}1r!ZnX5Rq19s={7W|6h;9-ZZr?5J1( zI~GS5dGY250eD{ha|Vo*o@V9IJCqy+)@ddRj+N5K+=k)qQOW89$s$CFR3y*y)6;Rb zM)n6Hcr7jeb#4!|8X!hDAXQ1_SELq^$POOqb|T%a5X%|U$5J&Q*Mxhg6}eA)f6f4x zKvnGPVI`D*0QpwEIQ?Tw^ecbt8<20cy;|*1|F#J;=2%#Y9OvF&%`$@(Y>lNfKW@NK z8WTo+*mY~d?phuTLy^SQ8sj`G^0$}WDBxe>j$bBshof-@va_DhDg5&_u-jssA#rNb zolR$`La50R=oX{jnXc%H_p4!9tTmbrvbWtom++arJJ$CodW8$md#uwt$>wMI;JdK# z7VQ(vEn~-mrWmZPcPak+bec~uC;6iM?FXRucjuTAd;Xj3?F^9rJJsb~^?rk@)={h# zTMkC>!_`EcF~zEUW) zx{39uv=w&0lM3j<+B%<`f`zlY$I4o2O)1=Q0N<;Kj zd(bo>uQW{#7#B^fCxy>A>~~V3sw|`=G&O8mYWM5C+#vc$>&b+doDUwF#`$ne)AUKM zcXS={MJ`kGB9{}k!HZl-oM-x$Z{~2k6y`~h4yDGo>8mzg(07lq)Hk{Qz%jp{-0v6# zx(N3$Nxm^Y+I2hgjaQbuZ~~?11wMSf@y1$9Pr^FcdaCn59^YO;pk+lro6VD#X^*1N zGbh^8n+)doB$*6UbRk;|*~9@It05=uq+6K`D(XGG?lz?1sbMmxpw#ZSsIi?TFZe^L z4DrYZ&SVw?EXW^M zdfp+2Gm0GGcp)25Bf1!y;?)jY`|{0$&NMxz)`|0)byC(hqRc3K0ZKNfY`7SYI{>jx zIvnQ30AHXY31oFoflAYe!0=?a-VCk$JS)cOWqA!_(WAKxlBZ~MwI9_rAWe3qkz9N7 zX5Gf+TlcuuY%!ryxM`EzEoWIj!Mhdrli68XW(mLd-u;0r89n=-2Am0s2XSKz zN;zYOnS?V&g8hau^P_g%mq)^uAB+&lxFl4eMnJz=$t9F)07gfykx=G`ipaB6$UiIV8(IU z%fnjGb} zzmpm&YJq#%9$X1scp+=J(n}*H-<>O=opGL?qSfy3_e&4Pu_Y}IuUW{d_SeLf)@bWm z;!4dJzUxjbuVxZVHPs$hYNm}fiX_LV5ZBz)7rOCN@8mufs*$G=C^I3Q)HZpc@l_nL zmlsN7+77O0d7-%gL!9j_FR+~vl->lTTp#`}j=AiVu9*aE^Wt#i#SB#IY>>rN`MrE_ ziK{cn`-=$%g5|{^#sL<<+w-&+^322_Fj(Nxbb#O~E|cCk@4vZ6%ox*{w557umOoDO zFO*u}qU#t26tvC(rIH!bfN7zk)_ZO8n8uLNj~B)YA<^N^ObeQdWAqe3E~&$3T4|g$ z^dy|5A4C7^WLoc?X^m+W(~jm61uWev(<*L_rpXbkiH{j8>V2LR2wlikqnloqe=Tl zBzdM_-B{BT|7y`Dqv>dp&TjHOIn&Ju!zVy=gG{=xZk>7Ffq#*E!#g7Oszq}5S=b#E zh`{N*K%qA)@_hK-X7SC_S1(TRMjb3k@|S=8uStJA>c2_u{N_&lE2hp3{~fdDhW{+` z1rAk=LcYOic9F#OOJ08U{OPMNze--7{O6a{?7bDXl}!Uj6O0kJ=woSk38$S+BMitH z;q#59+-}Y887qdDmTtOf#4fBN4<auA z2m}A&wrrf;u-a zZ&SrdP7=v^17$#W6)FRUH`P)w=oNs{S!T4SNb_}iCu>HUHeNeCs;*t8(F9Xgbb2UI z(&Ca(=<4)2+lU$?nbm3suj;T!Ps3WBX89zWlcf2*N%NjFCU{fjw@fjf>*o}2PZBiM zshP4VQTW2)W>q7U8WlZ^(B(^%&A%UwvocC`_$QB}ahuFgdJMuq)sV31C3m^KS39_s zOjq59|KS)Ly07Vf%K$EDC)EHi<|zocaLF47i1nnW+IkzaH(R$#j{}+G2L8Pd@qK6!z@dkIxs=1aBh4CO5n6lyA`b%%@ai!(t8_X+GnV_nZCd z2R0#^x_A~|o@L|ladNaMi=*CXdelqHv!s(H|H%HCY^YH)g*7-PYuFIupFDf=?3>qL zJ^cpEk878)!zy=>-0QchyFdZaCm8W~;sVPA{1#VgV@=5!Xm*BJZYY%;9elq{e%Bu_ zvJ%v#vv?u=cd{$H{QB}G4U-9kKgj_@l=5#A09mIPCab>x?(dTR41P+w-6Qz_(8txt zwI5z$x=$i;ACwBOnSU2kpMcJYXMV|SG|PtRXq=Rw3?=andxAf}PLJn?sJ=~77$0<} zOTc~hDd+)jL9IjzyX1&L6{&#ufyN@y3*#B?`Pz7QzZ}iKTJ(|!@yhzANp)i*`Fb>6 zyp1Tw0e@P<_fH9`7@#bz$h==WV#wCD8WCo+Y(7U-37s85;la+rA2^XRRH|l%`X1`< z|K;D4bbJB(e!}pN+CjZ?YlGrNG9)zUUqK?3Ak?Sx@nw**$-~Fpdv)E}5H?wIFgK-R5eX zhcS=Ax(*$0?q&Tq$!NG5G8u-QH12El1DLo+UWw=IZPs5h0?3u51DYOafgnZ_@!P-S zU||AQlB4-#b`)!MA|>7V+xch7w~t?*ym*=8C0jN3zVXjRYf{78wZ0 ze@O0JsW07M$L{YCON3JOKV3CMT+@}e&0aEmDshdbB+2?`d2;9Y_yTMhz~u39#R?V1 zBen`P#VIxXfy+8}|;GxZvv_ytcKo>kB^|)m7%gg&rcpcczN;&epHo~ zn*=lQr=b3$hG! zndh_4I6KeANgj*3$uq!MncasjhXBBN9vE&iI;~XR)5U0z9k0yo$OFON4kbscnxEi+ zogzadQ);>CddZbBlzIiFrn-|7^nZp6WG2b;S5KBJBiEM4OwL1UouvA?dG@C8XDyXh z>nbr+63l>yODUTL&14YwY^AIUHA7vNR z(TJ_P4k@i@U>&-yRN*xB%7qDHZ_S0dk+6Awkrf?)KK!sG?){_Zt2*zN;rA-x8)eDn z^1%p7=dhiYh|{au51u`)d|{y-H6J6@wNr&3M$=*bWAd=bQj7yfALr>f@1^7Jpof;? zH7s@4W*k|dM~!2d<;1tPUYYPDUDIzJmkZN*;sw}}_~fR}MZr~*<1i0462=X?U1_`i zx!(&)tGhdFm1snnV6bdz2Mz&PTueI%ka>|&M~J@eYOdyuD3cQ86>;*qnKE=54nV)h z&#0U{w%DhP}Mm$PV)oGQKOqVerXlnvO~C<8|D}StHtfxNL+8 zF)Gb4c>rpDcarzt+`WT3$Z`K{G#pN*!+-qyKmPsK{O|fqDAy#;E*V+CW@{ixRTngED>az|C8UHon&n0RY>ljG z_Lkc9WL;prZxGR=byo;XeE>U8r_&)Tcm_h^+mq*y9zT3}oIHH`%{M1cUQ)I3);Nf& zSKoZ|;Q1eqlP@!jOG?J+G)u}iqi|du{Dh~M=r|iJ`l#WeEJe$q@=lR`J3RP#`1Hx| zA3r*I@(_&Uub+NLRdx*Bnl9WK~+@amw2osMuLUJZ`^SKMG=!y8}1j+0lW zAXcXa^AZDH);`YtkeWaEa$t~$Uq4RnE_a&rYTn(`JU<<0N1%-zRrR~&*}brl!gjfN zG%1|H!DP6vph5A4m!x?3HG~3qD|bJHSKl5L#%BfCrT&s!Q~|B-5Lt z6sl4$+XSAJJUH1}0iKjXpu@c#;3-&WJc8U2c+hMIiKm#+c<^=`z*Edh6k?5lht5U3 z_g*tEYd60<6iXeKwngBsiFxb5z@Ei-zH{A;GT+9g7 z40w)nSGUzd1-1F|Nh}pinKprknL}7xzzdnD+5(=|A~^eZ1fJITY)Mn#L2ZxuL~jFl zT5H=_;%RLZUh=4!m$93#pvO`xW7e;hfTy(!{`6LWhiR=)k!S~aM#aR%I|2`yNz+*3 zLG5s&cN@Sn7|zHV0uL&OCyr|7MLo_~m0Dz=)_{irbJDd0JQysf8La?Mg3h9li8lkU z*5dEy({eGUtZQB8JQdzIGOsna7>lvdYH4t541TR zp6S@;?w(;JZxW&_hDACF3!Em=AzuZnUw4S(M}GG5k4ZYl3!sp~g_lw2paP&6$ z#4~-ba*@j>5xs#})Zexed*y>i4Cf#gDl%8usMFyPUA8&USS7bvM^J;v>WYh6uX3+Za@d>>drhY}w@+{sXxG2zFy#D-N zMP=ZBkOSHL>iYmwEx(V_&HPB(Xddn(H9rZ<`W;XUS*Og#!%mS6f#S1HKQG8*F;&Bm zsu||3kj8PjE14j!fxPB=~XH2-9wu@)=3V}=T&5P0LX-4^gRh_$) zH#%c9;Vg&k&L>v@#7Q9MrnIwK)h%hG6}7O7`}ktWjwh1gu$H3B)}3oNqf0qwonkwl zxKljAYcHGlMid#3cTL1j@hGJ53s9vZR7L~n3iPD+qEE77biPcMuo#UoJ3`oJmNBXr zYY?=kDJDip(muY%5T0sHi_>gQO$#49Sr1Ej7-e5lksB`Pg&X8l2Uf;JmoWq;JYE*| ztq`<$fkA7L+7cFIRD=at{niPMx)V?v1XC{QUBar|BS#8vOC|V_S6<+r8zR4sn~hXt zu?-AH^^bBd&Ljh^HCg=+;~)#N5#M}YwklNuy8L6lC=xv530rfq2vOegyf`iJv*K;P zVvef!Y zAe6%Amt3{5+*e_$U^C-G?C(|>=lPo`h>IMbt6pAwo~7Xt%~kuWM>Cho1~oIuxYSu^TmU2 zPT*sf`S?6|sM2{7_2xk+Uh3A=d`t+Fzd3RRd~VYUa)lG&=KF0S7car8U6Ct1SBgf9 z*BBqt5^^z^gGR2A*7_EZ+fS!KY6c4ojI-d7OaQIrhV3rqvwMjqc8sn~2Lp8QRoGqv z8OH}CQ!}%|?1f=t&GO5?gW<9XFsq2F_kjspcs3kOGm>!6l-w0@_;e*LD~&f8|I`xV zEUQtkgkvt+*@uLjC{W#L*xC7?PmENrRau4Dzh2Y7aQ zKln+XJbjtqW!Ch9niD4e-v?t{OJ{C~F>Z`8H^CUUT&f)~#;xQv-_=?S>S_sN+yz6O zD1{B({o*1F`iN((xI;!|EqXI68)Jn!yfr@kXd|mlT9<{gNs7dGZY=X0450HxnKfYL| zu6gHVVm+3Cnu=w9FJ)RqE#dYZUm%XOleg#il3=m1UC-8_kyf zl(XUhKpUVM)$}*n<(ki7nkTE{9zb?T$#p;Mp|E?tosCC*kdxEA69)+2o|>xRYA@{_ z!^&GZFqRr@r`rUzw%+gRTZ8?TJ6aNJ7kuSyf!YVR3{h%WNCW$egBtaK)kE$)vNRkgY%T>2l&Lnnz7(e%52fXxM`8Y8=3PoZZYI=&9hlk_Q>DfHy zJ1@y~G%SWU-33xd!%;s{nudd#GMn_qG0gb0oIjtW5)HzrQ}2a5;JcQj_dMJ61D-raeRolQIDJ{QmiaZ%)W6 zH6aC1>T%!t`stG|sTz*in)W_Kc>ID)5p+vD{|!}ETR7q^CRaxtow)UmGq8C6ep{Si znK$ybIKgs7xJepuf(5%*?!XCFFir2`7HzUUIl&4;M;c-&eEHcE zi$RDDDN3j5_;Nn#lQ|vgF2{-{`2jFdw$(96v01i>82Z_Pub0x-p8dNY5X9uwf^o6T zOtQ>?e{DHA=h`%*9d`C`ioGoxMF)sUd!SUDRm$%&;REVcXQP;zBw1$CKq&*}d5RHU zRZ0WHMMH|X2uA0&Y!Cf$T2!=DOA0@xgDZP%Ma&3K{PAJHvyeiiqK7s7x>5=o9xAS- z=QP{}UcDxhy**18@m<4J;_9VM1nY8|qHYSCPOr${0J&8vh$VYA$`S1`=Zg>f?dE4i z>+*VMA|o^MhZ@O=I9bsdQbp%wHyte>8MXHainQS3n$KmeZHTftPemYsGlV6bP6RhD9#RoVCx>%>vli~VuxB~uck`mg*Y`T7YjW)U(4tk9uRbV#7~VyI5SLVU5al36*tF z@P{qS)oy}*Y(dW*s4>*ZldK!v;ZeSTP* z&yUN=#CiNqfkyI&FOpt(=4RN@$>)i%uFEGS*}EaGC;6^-@P9-K`nse%wJGSOsC!Rf z`#$H68S8Qjdb3tr-mOHH=~+5I`+78e^Q_3vM*~uL>$;pNLNp>uiOPm-l1!7J-P zt~LdUV~waWn%R&9Y^PKoHR^yddu62nqIit^_S>FijgR9}8K!IZf94v|A;7ScTc3vo zTl@4zt6^CqDhx*l$B2?NVZ`tNI1k9?L*ahL533Q#_}y+wH$o-P9g3Dlw?UX#b)a9cY0 z_T>2=VapiI&l1J{@?ZXqE4I=)d{dH~VYVSj6BueyOm9q-YrIb0ukw^_U8hkx&QEVl zlhO@o(y+0}hBS=^96S))H9WR$`4hJN*!V5I5Qvv_3!w(CJzy^T1E&!n%? zt8IVxcS)xVFCo<%0Q+ks%ja3x_Lr`ggelKmW5SSv2VXq>_9UKejR)72tw>yWfbr^k z;k@Y?{uD)Z#rSrZcOus&&YL8e?E@B z+2vEC&w=VM7DIf7?JxiOUz0Chz4&8tcb*s1d-u_ITjFCX(|Ix(g?CM`N+kX6478Kw zl^cwE>WE}Gdb`T~CmYWR^fl9LPNP#?{$FrXrit_tU9bnd+T^znrJP%5l&4CigHlPA zDzBX&+Zx9FCWlfe6#}pIP;CW?r1UciK49`C7rk7o0z33IV$ulIP ziRhPIhKr?laHmbcc<<$JDU#}3e4H`Wh$!LEC$)pZc$OAZI%}(`)9|@(M3O{*w%C&B zurkHgS1X$rwCj=17Z{X`W{YK+2R7RQT&vU_9rR0`(&)mNLwYChL_1kq`_|OncBfv>V+h&9S2>lo;W4*8`dGNkMy_w*R&Kf{-&v+#gwSfrTp{Mqy5)$^}6_XB)qF+b>9 zKM$WiI{EhXv*)j0K7I*`cZiA#0eu6a3n@bKr%hda@$}Ij*Lz6r;$!pieZV_+Fwi_) zM+u62FCSdqBZlzCpr7v+hp@m1c#3kJ0d9fjyG!A5}3mfg)^cK_Q z1r$tiHG};>yUG9?=lw=bYlphsGxZ>rJz4OnTDe?-P^f#pF0}&BC&4S&Ci0R_}fXCRW zsNQ(LL8qHLlm?8L>r{e({KO{TJ$R8ke13w@x8Dyr2d=tX4HMH>2C3RB@(Xl3fN1R! zBU!`b^^Y>hs|ab#Yk-GPA;SD3U!c=thM9tr!2<8*p@TAcUJbhr8+-rGd^8_t@UiC9 zhtOOJ)Oz3J=yY1bW(D*sro!<3Fmfa*y>sHmB_wdxz_OfoVJLs1u&|)ZidqucU1sFEq?ZOSRUDR=&~pf=~E(M<%o(&!A=) zj0)RI2V@F$qWick7g_QN)QE9h#p}@IHGIL>XA9^z8IOip|FS=RFK_aL$Inq)x{C|Y znT4bAklY4-!^461PIlqDmKz56d|m(rLH!(HByqZ!=bdU}EUR?3p+jnIjAOMo9ai-` zJ{R;Q2&WLI|M>aIBiM!us=DQTv+pRDiWrRid@+SiLda}56=#fbvW*I7LCJUNC}fvT z;sU2L^zWtP)pO8;^K{yW&A3Z;WCSBg)7D6FXju8;)tA42{GZ=}%l)uqpTm#X-6z0; z9~?8u-AOvfs~!oU^&Yii&RE|7K_DD4X=GCbY3D7CAP{^;Z9-5bW@><>y$tUW>UN1CZicWC*6X-#&Qq@bQx`lb27A@z5vfB)=cMB^!Bd>F?!O zyu-aFabl96?J@@%2X&Yzl^Ab!Xix|t6qw^^3K}TBQ?#+d#_eU8j?3&SnsT|Bpk7%k z0TibC6b(MF^bAjzzQkVF^I93Ce3_PvV^l+ALoodAWXL#RNr`6qoN02wq2ucR$$(* z|F6Ao>5(Hz(!2Lx&?PVpEXm!DEPp?0G}_^?*_zgNlLT44jn&L7H!?G-DyA|sQW23X z)?x(ANk^R)SU}JMI<1cTAEbZMzeHy45m{MedZb4r*;7p^zpdLjM+b3H85>-*6DN8L2 zoK6RO5Urhbw{mcdzLh3QK#O_TlnzaZz%2E2S;ABtrEy~t8Y!88C8xD!k&#TwH6BUl zS&IL|x(#((Z$gCOQKBg>w4#S94eD8TeqJu-vFG-?k|-G^h!IRX=I~xYdXMURbJkge zW~FpOjHH&ch_r#we6h#&1)VU1NUj5Hn_0GclhvRfG0SmGc9xBj%POrD8?YocczV{F-I1+wVgh}azXgxbhpjd%9BB-M&I_a`ceq^8t!HQ7B#Ufacb_U^zm-!h z)!tUK^?FgXAL|;$O!T&WXbxd<=J^`4*;mb?{ZfjNwD49TKgUF22Lo>vSN z!zDWJ5u?FVN`D+$RhxCQJK0=V7?usVsF$9hY~``x9jn=(_7UADuvmZZV8<~|ddq*| zrXLQ*#j2=X@(C=hflVqc_7D3->1gNks%ZnZMU6$bbEI-9+?Qh105u1UkXwhtzgG(( z_crbze0G;LMw0x&)W9Z;L-g6xMBS92lCCo+w_8eGIv=KhY>|t*Y3G4|I=w}rSF_Qa z28PtPhTphtq)zG*6>+^KY64#Y;h>WBrYoHTuTtC|T7y2zW{txJ!8mF; z#R}vlEF^UDW}C{bZKN;Dt8EAH=(Y{p-!|rk-ukiFtV8l}ELDjIvwBCl`n;}T699ZP zwSSJep2hUcUJ*Wk0LF%90L=0ZGJ8?Zu3+#zeKJB%`TmAKR(?nZwo0W8?fbJuRxDAO zxi+&Y%>5D*G^aPH;HnH}*}R@Ua`gx2L1!GG>8qBJI>q8LHbOw z&vBAHdH(IUPoJH+Rp9nfkF(`S0+BWX0mjl` z4#H7+XTf0&A&%30n7M{4B1Wwy8OTY?0-<@sheDNduY7V?yb@EMbZ`*c+<2K z57XRgunBNFwWhHTq(+Yw7*_#96+tciAfSKY&BvWda%YO*DP?>9020ls60_VNb@}?2sNu##M1qBDLyPzm@(n}tJxy{oahS`Z$+hOJa0q&UhJ ziu+a}ZbANCRo!Og(+KOl)oQJG9O(1qQi{b`3 zVx2NF;jp5u$B~N$rfI<%7Ud=gd)IiPTD+xWyRSRE9{FhR-B0l5SME&ny{xm#^4cx@ z)bonE#Mkz`Y-?Y)XA9o43Vf^LLeTz0V!TfvVfmbtpCMz~1#{U0m;Htaq>K0D*) z2^X_`iJll}jJ;J3__fjjEFT^%$rNKM)Mru}g_Tc!5T9(3!UA{0TEGL+AO7i&gMRIQ zo1$Im*WJl;+Z%!Bm}-*|JyqN@(DbKnv8S&415bgW?pyOuef{OBpN(lYol(XEPiaM@ zd+JqQ!ESt-bOjT)lXd@-m`UNSj>j*)PW~ppIZbBG)$6~6|Fp3r^pF;oh6W3~4L%T2 z`xBnvTKYQ09V=~Tf}Y+Uj0PSzoO+y*@75T(I!z?FJpt7R+l_R!xyfUP3XL@$`-b|Y zIk}x@CxLf?vAI95&0RG_lTTeqIetz*gih1<4WlL6(H$MqQ`W%+1iDQS%^aL~l%9gS zkNG1pCDX?9^J5&Sp6h5VV5PpE3f~P1kWAqe0DXAK!r|=&O<;d7Dxe>o%=Y5ZYFSJ&D5PgI$l+q+TR^T}3!BAwo12yRQUFj&p z8+Ldm4`rqVaU8slk{0t3NhwsCND(PBOdNu=)CnjMAY@9w$QThzC8f4t{l~e@qGRKEeoxKM$Rf<_Kdf61lTKolmr?Mltp>9S%`(-_x!`iuD$iJA0GgqC2=F+FdFcJSRo|k<8I>cbV9s zFV;%)gZ&2g8Dd1R-zXJK+}kDVGkO+T-F%rf*<*L0yk+yglc3Oo>EU+n(L<2MBJ|Bn zQyS58loM)#J;$K`)xBPUBE-&N7^GcpeKHZj&L}PmAyq`eLhu zj_5aL0~N0BDyA6QdpqNHGA58|46Mk~_u#&481Kt?_YwNh9Ydm52vach@zO*nJB1Xx zJ<3iHW#~O$wElRw^}dqsyvSvOOp)0n^l4NX7STaM@clKY{;D#T+|v$%)d%(k^iY~& zxC|3D9BM_4wFS6Ol*zJKEQkFus7AR zHeGN&hfQ@8`fADDxz@qnk%Eez-Z9eL*WTF@N#8Ql;*+UENpmTEsdIk=pN0lg%=B>! zGT1?yIWW|F*+HdGQo&>xEc#q+mU&gol5*{KyOfqSTfl$|nu~ z@IU{#>#*+^8vupK=DB}VM#wsQ)A1We| zx{FykM5s~W(wN8^n-Ij?l%peSQ>-1xKwbC7VtR|2XjE{LO{mpu5Sr_DMjk9q?yUfh zDdYh0gkeIlA>!r1sn;8ajG>ls@>oW+941ei!9W*H?&3q3N*4_K^_A|#R5XBuNYF6p76xy`bf-8CMFE|5i}nP1?tZ+4wYc_W?fMp zg8>a9g3W0yA3Tzji}d=k%$CJqz2W%p6pNEiVyd+|hMdOj%poCr33N!5h)PqUX*6w? zqhXjjf`u`-!GgyLWr-$=hM4EVXcdcW6*A;Jzy{s{Mru{NdX#tVwWi8u%?r529nkJO zenmh=AK2JZPIy?p;yn3Jubw~i$?R{kMKKS283ixkj#K`XtK~yi73IqP`)dC-=ggwa z9lY#Qk1qSPLl#|EykZ# z?1#2Kwd54p2jIPr`sNSC^Zc!3?*0!V#{RhmI-A4)Az8rpWm{PsHSWntzcefO{Wa*( z%>pF)cfS$K$$GWMIVhGVzkGvLjm=;AXMZUsbRyn=puMYY!9O`K;0LqI&FU)cj!q}P zuTME)r!+V}wBq_63r{V9u2&)H?Uq~fUP{k6Q>Tmsq-SJ{zX8&(@@s@0S{9LulQE{v z_Gu92`=Ysgf*4hc9kNv#?`lF|mwPQ(Pcu9c(%QJr>l z*AQfZDDzrOK;yy8RYUmh+$o-`iywn~62r5?X-1|U|i#^59} zgB*z9dJm0-p6fz$zUv z%Q$fqSR$~)?+Nv-o15(b_R?6=zV)eN;1WSN$%xX-Wklg)$q)bIeif6gaVGb$8|cBz(mR&T0&iQ27L)lIcohAHBi=tF^(+LBlqG01U< zRj|#x&S5>a5L1oI55}>*F4_z+#LDl`9awDZhiST(B*RHP0JM7if9Q&e`1arX37TmlweSTk; z7`^wtO5#z`Y6Ougn&jaN)p0&+CSygEzCxsT+=MA1BB+=ZgDZPv|KH2#qnrISi)0ZVK4#YYG6tw4_h#zu4kyFDpn$WuxmW# zpUaHFeC^G`0bI?K>#Re-LgO$;n3qMF;YdLrF0yqeOcdUs7sgO(`-MS}tLX-Lk&}yU zUaDJVE#y{xSrx0RI`O4=zj}W5Yg`phQ`BvpL$bo8;1;}V`I7?h_FN{eh9Gnyn=Ot=PclJp7;9fy+#|2+s~Ael+z)k z5I%8jl`X^Re|>&boQ^hv8e|ma z2N(P_>0WSKY3<*DNJq{4Sts4im|fL@kvP-eXU6P(OYl>zzQc|63d6~;TyBMOXI@|= zCwII+JoXH|Q&Y9KzQ$centAdGbZ#>*N>AS0GHq0KM_A!;FVRe&R^=7!0PA9%hd!4I zBR+5kGqCDKu!FTyf!x%Qz5DJO>kA=9tuSjSv4S6t2;(tki@Jy8ISv+KL`a{Rn3XRl zR0u8y=k+?)p?0f*-VB|z_HBSRN<2L&sbNX52Mn zjsm01UOcemh98XSptXX|bn5Uhd(XycUmS%6(8PVT_3(~Ctp%@zsVv{^Of2C=!jctn(vGZS!xxIF0F?xdcnL-q!zN1pD2 zR9bPXwHcKc;*!}%)ha|7X@X3&GU2<}5t) z)o@vKH5wjP0u9plAr{UzSgLT6t&0}zdY4c!yU6hT@6OBcHKA{ogx>M~+Io$b*os@L zW%AwBxjeImkr~~{w;6v1(t_AaPTDy|2hvW z>MBDoMlyYgFi1Sf42+MCs&BAnC-GSHZBqhg0!%Mb^DMP)?s#ldx>Yu*pTOi z?CxoS5V!n1^(1*wA~}x-RB;1{v;+PNJ6I(As4uX%1&hzULVWzqH_yL+efIR#+3P3Y zJpb;?*WZ43_W11U=g)pYu@tAd#6G}xz&zU&(91W;GS5~X5n|I;J;UdDfqZ&TXiUMu zzO8^jAcd+qb0hcd#zPG~UMxV;`FA)vXV~blk{4VLi(=|`pnUZP3reUK@c*NbNMIh{ z76uG%?5M?e3ckwTM0b?TpnAGZ(|_R-2HP&Xbhv;cDi2)RKIqJ>_@$b#3A=k0e}01G zR&QMzzP9F6J_<=k6?s>>W!_wt^Y1G#L2)>=A$T zv3=zZU8#e=!&f=nip{8k--2QbrsO+fcN7$$^uGOZdsvcF&V!<&(SIL%SW0$ERZwir z?x)yP!o&#AXQK_j&+E-v^(WM$NR=9p47~jPP(%JIqmq7-rqwvR%204j*39q4^CsyOVZ`W`EJZke{9o>3);FyzxPssz>a6Evy> z^W5N{m{BDJz1L2Rkh`|(eKM*9VnzpzDxo=5_ui<2?H$Ply=8@uc;Y{#wZj6@FECwo zcG*#1?-&Jvua=Z4I?}Q_>015k6|yk?;_aVJt$S!{VSt+6FH=hjv*Q>3*eb#?3Wg+8 zz6Sb!np(1jFSxI&UmE)Yc-|Xjz1Ip z>+`Y#+lv0#wz`M5l|~MN`(;~cA?%+nj3s&4w$jj7{NCDDS`%*Xv93pIQ2OcFR@!>7 zqffxLGQ@K7>Dg8W!R07yQo&+y6gVRhD9rO zy%hu!I%uqb=G+DK|FM;m)}S{-k}1Y+H_?_}{;Zf?)sI*6m-+8KrA&l*g0=QKS>nOdJB~TxKhfe4aXRt5M~OEJ_;a0Y#`CT2$5*N!Q5G)4(XE7g-P6 zoH(+!76b*mT?nOzfW!KN?L~Q!*4KHy{#dv{f=x7`Mra)kKhcU-&k!>_1AS!hOtY#(!rg2^7nXW;rc5F0rBn?SmIbF5BnCVBG^outmts{kb2vZ zKNhR=G9)UKcPV>2w6gqd6MRJLz3jd2Dgs~15B}{hH>-Dl+*HNIg)=Vj!yQGYgUAbV zShnI>c83OgUas=cy<8|7{7?hB(Eb=7>OTuH!8R2dC5TZLlF6Ue-+rCPms!>q!3Qkm zp-EKd+%G@mCg&i*c$mmbj<-As(OWR=_E3ZGli6_d9jk(e@n5Q%GJ zxi|#qv!zr`G4ph@?uQ}l8#xhyi6Ha@tS>=!Nv7`xMc%iqDdoG+dLU<=hoc~HchUl2 zeuEDDi`AS@I7T1$a7n zGrRnUTHPV;{ywW#4&dyz3Gn@k3j{TTd4int zse>5*2e)hH^PhXiE}uJiT_@h^#)mH!H|Z>Mz!kG{eKSeG$XYnz>SUQYq~iHzmM5T7 z7J25LpF?lelf?P*iyJ4$c?M6N!W%Y=`RW1scPnIVT$YXh&bv~DK2r&{m#rPH3X0aq z0Jm#hRG<$z--0PEvJH$O=)`h828{}YvT?H)b`{E)IfyF0S+@D)p2KxrtPpCj6*tMw z8;oNi0sIQA-YQ!-G`%nU=;33l&HN%q1ZxK@oRsH@$BXI!N?M%oR(#56tn_@#`__Zt z>hSqmAszSl$|2f;48Ei9eBraF-y^o{8!%Sgg8Cv^AuPB5`VH&|b&d$;2nBuH+p*#b zr4A_7IWA-mOuofWZ^5+^&okf3+yxJjF0PojpT`|6Za&*I#hV;vI=U3@+d7N1gUMBd zN6!!?*#XSy?aS+hiyAhUS*zw)o9{CrKTIemDCbg?a@Rtccvq zRwbCTptMnh`?8ho$`trSMWix;jZ9N8naLrOD`rj%F;bCnI%?K}c(4f{Hd6fzx^wok zXRh58IRfE7fA-Daxm#YmY{@A)$afhnm860kR+t}d_HM~M;v_T@GsE@Zu&^}36WlL1 zi>7dnFZh??=;pD!YQjtq{#?jGp6?=0n{1lmM(ucOX89=Rmqy8l9BW#V*Z`*|N@}PG)Lwp-o;C0u24|bduv#P9-9pTB#$FF|v zXj|GX{sxzf$JxKebJO>1(d61e@o#r&5i2Cy=KY2>3>b!>+(Czc&E{=1FU{=A9kwFT zP1PC|?Xf_eeD(6_t6zIc5JzG%1>2+CHLAD9C5seQYoUfPU)~_@bedfw#51%x3aUS@ z8#OXt(t&Zn7DE~7ct)_-vAC6@wPJ>_!7By&I-Tc^#csMOkiIT0=0nw>JEe#@9J3Qb zwDw>*!?37{PLVYut+O|I>NsG=(CpbL8L1YM6Ei5t?%l}?WI_Oo2bFioigb6v@Z8MJo z-wf6hy#tvax=#x{fOv(Ufx7Hhe%d76^g-132I-_%S68#-;g(h{^g6=@EcPG%>5upA zS4dz>+&+oPSnkVClkSs)Pe{+t@+)VcopxPya@$pZ^EXcd4=X~sd-(enhxar&X_o6f z6E6LVTt25?pG?}<*OBiJ6PO;r@@UZoF$&)H~$Mmf)FqusDzJL*;n4p1SXT)k@*-xrV8D&@p+t;NX5O%Or zVVC0QFYUj|5H?!;v^te(qZr%MsoxZ=0B*P8$ zwv@!!pO#jDZ)TL-fsad)Z!v!?d4;grXThf|63n$D^PNuzZD}n}Z%GrkyU9tn)BBk2 zWH}GLm^&`gch^zpPIeALa6Q2b44yu8bEeEN%vqHfN0@GqyJjBj3BfQ5_-@m9rZ=E@ zAq)>pFisJyo8QX`HUrN$IK6h$crUXG9VQ>8$7e6J*y9<&efK8=Y|5%yst z)`#pvjyxmu6S5CcU^i)_?L&A1!;}5?VIy7yA6AZ#BEmlWJNfI)itn`#``!=6MraXm zYNRzkokhTz-{HsFUHAfHWw1Lf1GDTW)SXrkBagAU@D(y>b8%s`{3&&)Fj$`TQS2>v zw^c-kdK%qSqklR*t(ekU4Yakm;!1G(;cYE=v8Lz{j$NjR9(7hjJ`hOiS*FvbJZ#peEE4$Ni^!<@SUua>ZI1V>d2n-)mu z;gjisljO^?Nt+_b$0Lu;qy;&&G;9bB6;j+`W_~))nk>zl!KDb;M-lOJNhU-Q!v~SK zVcHn%FG#nHr%v!EGP#X7wc{~SLlhA!*@YZOislkcwN(pg=?_DWzHsTe#iC zON&)oLpEj#$|wPuvE1Vz4rM}HlqF`gwtL%0B9sFOxwVwrQG{|e7z6n{Z42DSaG#PW z5e3XbOUZDJw@vHrISaGL<#U60804@+GAs(T!cvxQ)0ao7z_{U6(sC)qkiJ7paGRXk zrNP~d+JED8p|ap42N)v{5RrC_bEz-#u z`50Q>IMr>LCSnMmWw&CmV5`$P^_U@#8*wxT4IgL^X7e}MYL=&%)tPR+v9toFYBQJW3Os(m{O{wtS|i5WU={`3>4)lPjzd(wKlPoF)1 z`s^(9PO!3Jf_{iM87B4?ZM&;r1)4*2=m+~{$ewGX#*8b+UOlAJ*kMO(nj)x$89lPd z809j`vf=e-u)X-)7*}xEmm5Rvv1Tcq2+1(4Hik^z19QNJ<`Y8m5r$vf04rgPkmG2Y zsX(e;WL3@E_`x#BbS;n9!5CiGC;*;q>9FJ z;j{~Fg;ot`M|#2_92lr3l)2gy9jfxCsPbjFFt=Bgy954e)!JEzNx#5 z=Q*a+=D~MMjRy)}VM&CGW4?6KVc2AE_|iY5h}jz(cUP&_LwqP0x!IP(>Tpa6o zvI)n1gYl6i5%12+cQ+a*-+q-7Q+)vPNh+nHGM=Iv+10^@SQY0^r2YEl$N(V9#RcI9 z)g$dhUsp9>F$2zZ+;)5 z3|7>tcBOph1tx#HQjntdPdVVPYLh3^O_Qw31P`8EHhaC-?1k@Q;F=J%=OF7@3F@jS zt2>pA$$1GaJ5p~4u+_OFKTlkay34q4^Vz$v4K*t`Gl6vsR~4h`-gLGr*~_mHRm*p1 zTOe6ZG4z)_^!EFsz)&D^iF>S00_DN}lVNM+caOPRu}<5@)6EZm{MY2+wzgq+U>*hE zstpZyvn5>I!#REKrrkV$o8)hxOI^QaIK>uu>pI<;+1Mvt{;Nm!3v~x4g6%s7Kf}q~ zol0)82Pk2s@)WMP9jNaRk*rw5a=)vGj^FPsGaH(WyFu>_DgX(Pi zM`B2bCT?d0I~*JZIy{`DWO~U`@CX{PJst5h#X$^3q9TfeYzcfMAR;B zMQa#YsSd#j)Hkcy$Ew%{A&ez@5Kdlp?!v?c7JH6Zg;9RHm=H?9fEqD)f`Y23uIA`781IC&Fjg$o;SgYl{|gwXrm!JR|Yb*lURFTnO#&|tTu~9;5)4$Rv*#V zog=va^_EFPLY>ShkyNPRX9xRxc!*D>qLvwlti)rQkJj#5 zw2rWI%z=57T!tvl7(MJSyX5uOKI}EHj|II&p|k=eJ;2UrvGdbe4jov3w9c{wCYn(r zfH-)ESX1Rqa5Q^oS&UAR6R>=kp=>B3yhDz4AhPwx%Gi_>3iC`VI_5aMOY@o+HA3-@ zIn|Amk%4z2G#{?&IOnND#!l-^b>0U!b!-X*2x*m)2aoLM}`#si-xQ4VZCQ6EjAAM5Qx;Jc41yXl%mhb)D9$Y+Ylx1-8-Q=dm_! z(ksifu{8QP)}zZiqh85tkFq1?)`&i(5dZI>%a;c`<7~R!CA#EW?03A12$X z1e0w{e?`uAnCqYy1}3Mr<$_~E88Ka@COBgp3eez60vG2&cJdL42qsQ8r4%`Yf%Cpm zHd_>6H98WXj}(Sdn8Km8QiBH*MfGVM&Vw4I@@bTWpz^`_5|L3*%hU6+y2#V~Ee!9p z7?Z`p<3?3lMI4;LD~EUIqvalgxXojF(54AJ*!#SaQgGy!8u`5Dcmf`oQHT1p%c#R# zrZE}eG1Y=77Z_ib*dgT#p2(>BSUa@Bgn!M=n1~|Bx##UWwKS&4d_7 z5A>0W6{SWXJH*8uQ`d2u)nB?op;ufP5}Au83GvcZw+GvH|D;=m< zMg}=ec>|6To+8&bb(1fLZ=O1;+pM;P^8tcgHp@k~W=GVt#Ykl)1l&a_51!tj0o!lC z^1fidwz0x6P7Z)9*ZJZ+y)0p<<_@6Yv;%#w5+^3|Pu4D@%Yh>9T-3x)dHI z4Ydf`G(cgjN~~IPvEUJEOhpOB41%r>i-db{X>Yi09i&ho91)hwl|#(RDm*#7u9T(v^jfiP+xx9i&qaqRu3Tow+0-pzbZ%iux=bqAo7 zx0)mD2jh>UAr?q%m92jE~d?rlo)MHZE^fxq`Nkw;*a_vD z2rCEL)+0Q09Vd>EM5}=XA5#}SC>I5mqonPShIZua{B1IyCOc1TvjQlk=DnnVIo5L` z*((y%&+{~QxLN5rZsN#mu+5alGt}fFZ~o(=oMwwJrw<9!hJTi zcL`%d_c4Zmgim(GZ$*3*e8fdWKR6nse>vyU!FRuMun4EaSjgQVrSA^la(?1L7QE`1 z%%`Eo1$TLx$(ifg57zi4()2iHxoO@O%Fg?wFm^9OpT8;{dYGdq@3PZQ6N;g z#mB!^vw%JCuPSzHcYfI}-g33LnSk}?{^3%2(llGI!L&x3*`-MPiXjiokTUVW_0+9x1V+b|o4`zCs$K6{50X)=`!?QrFc*`>{9{u8TZJ-EA)<*IC{6%{(+N zhFe1SR6n>o%l6JpvFN4WbG`J^sLz?aO);wuDAbEJF1A@bCttP?Y zR)mOQ1b}^GJ)~kc3S+@eIJl2GknpnqNWIHt8gVifXUB1dc%6|EZ^?ApCLo9DxsQ#) z+lf#T7LPt4Q+uG&@p;194yUouV}pNoqNp*9iQyC^xaK1mNYv3osTh-kqVHQ2CALj1 z#KQ|Tp10wU5vw6?Ef`{~ux22A_!!TW#SO3}%2=j{pRYS8H_RGM^VOR%RYixyv!{ei zh@qNT8+kNE7)#&OsK`^$4IE zlT0zq4H=;U$yNhqs~aq@8L^Jz##Vs^W5l62>;=|0tQ-=bOLrY{1}US&!4y&}MlhFh zx$QE?A$f4ikVngEIn`E-*x)Dy-(r!Xy^vR_N8cG!corw6pj=8i%uoQEJVln_H^Fjq zc1&>73uZMRM3BA1Tk6=|Q`}ER96^IsB4Q;JRGR~5(2kSTL6&c+KetTujuG_Xs$BhJ zUX_lfw4?Hsb4aReS6+2N21f|Itsxm|)IB#fK&m>LC&i%u|as| zn@I7DmF93>DJ(urvMbh2{gJG2L~w+j8p2NDN3cKa(bGd*M2V%f1lNTQsv@q&U?Ilo zDs7Zdk`FyD#Q~g^n^hA`Fl9#?)>va{joep<+g-JmIlw|rmp}aXe|!H|o1;hJe?gZW ztnL|#l~8z2v{`}&-mt%UMF=B7;c%65C!C;8gygHoUw_lZBE@0n1w$pyiN153zZ*ck z$d|>cNEX=&@!P7pNv8Q__NFK|Rr2t2@IiHQ3$1h~^jbOi)M;Mt#LG7;Y%MuE`v%^) z2B-6jj``eI`p*2}!luUDw&G%i$y1?E_jpLISMk#y*YFTJh)=P`%1Lkvw#eGi__zXO zjImGiIFL!tnPq07ck1uW>L+|*D=h;;b zKX~MuowQm89%#y6-3rH+-E@wYQF5}|gYf5$j4}VYsrL8%ZCH7?A5{M+!4u@p{dz|x zSf}5iLV$_2UgVhI_Z+(k^D47d3HqW+u8Y;Yy#9F$O@JV9ZGl&U$>H@6fBe^A%hsCU zM3>qnZ?!sx?bqHTe*K1?(BC=5e`vaEB~(y6rI-l|Z>VU26_#!{O1{Eqn%B*V->=}a zXmaj0tlMn%yV>Z}Z>CHuPoWmP5#}9#Cv6>P*F$E=qn=OHU=2&&is}CIJLfntr4aJ4 zoQmud6sDl+msAPA**ZyIp}~abvR+cWPoD}EcvceLCa~bMUv661V*jkx;G+0l_hD0I zs~Vl!L`@ih{n2FmAz4r`nVgwIjr#Tup^L!2F zj?0(vfRbGe3*Le!#9M$G|ENf&HyuCWB>DP0xhXfvD);qfh^}5%Sye1<&|vCX`@-Ec z%T_pZoI-s3 z8+4CMiA||ZyKl$Jr}8{q;VB6tDg4Vnnr1G%U#I@0ygmu_EmD8~=j26+z_;ZzX$$}N z4}~I<3CxFi0dIB3_=qFA%5hu~)p7qrOjxJ)|2%#leEbxX#3IFCNj+yU3tDeyyb%Dba^Vd+4yg`x?*nHr= zD*x@It}mbXw*=bK;K>Brsy9>5|Au(OIcy}*1bi6yz?eEy!#`)q&xQ!~;9aR6B!BT2 z$=)|?R2_KIPQ5CXwR@D-DH6b!uMB;pMcTnG{UA-COiM3a^atbO~jCc}VUj+!Si zU*0!)EG1a5dF{@yE4eOX?&$Bbd|@VKwZ zC+%tErrI9>vBH1@3w`kfHz(A#V1k#`8q{;r0!an5G?>j?Ao(d+k=I#;&;AZK_Q~(- z){?kuL*Vx*a~Rmby#u(?5}|@c#fC4NB=4jROFSz;zD* literal 0 HcmV?d00001 diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts new file mode 100644 index 000000000..72db117ba --- /dev/null +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -0,0 +1,843 @@ +/** + * Issue #13 — CI concepts for the forge layer. + * + * What is pinned here, and why each one is worth a test: + * + * 1. **The extraction ladder, against two REAL logs.** Both fixtures are + * verbatim captures, not hand-written samples: a 2528-line GitHub vitest + * failure (run 32515040122 of this repository) and a 1599-line Forgejo Go + * failure (codeberg.org/forgejo/forgejo job 11952749). They are stored + * gzipped because the exact bytes are the point — the ANSI escapes, the + * misleading lines, and the position of the real failure inside the file. + * + * The three traps they carry are why this ladder is not three greps: + * - Every payload line is ANSI-wrapped. The FAIL token in the raw bytes is + * `ESC[41m ESC[1m FAIL ESC[22m ESC[49m src/…`, which no /^ FAIL/ rule + * matches. A matcher that skips the cleaning step reports "no recognized + * failure" on a log that plainly contains one. + * - The first line containing "Error:" is `[artifact-canvas] Error: host + * blew up`, printed by a PASSING test 1214 lines above the real failure. + * - "Test Files … passed" appears FOUR times before the summary that says + * failed. + * + * 2. **Refusal is a first-class outcome.** When nothing matches, the response + * carries `extracted: false`, the job identity, `logLines` and a + * ready-to-run `next`, and contains NO log lines at all. Returning "the last + * 50 lines" instead is the failure mode this issue exists to remove: a + * builder treats them as the diagnosis and reasons from noise. + * + * 3. **A timeout reports as a timeout.** Through the script (envelope on + * stdout) and through the dispatcher (`executeForgeCommandDetailed(). + * timedOut`). `executeForgeCommand` flattens every failure to `null`, which + * is how #12 shipped a `pr-exists` whose null read as "no PR exists". + * + * 4. **An old Forgejo is never mistaken for a green run.** Forgejo gained the + * Actions job-log API in 16.0; on 15.x these concepts return + * `unsupported-server` naming both versions, NOT an empty `failures` array. + * + * 5. **The measured Forgejo footguns.** `limit` is ignored unless `page` is + * also sent, and a `pull_request` run records `#` where a branch + * name would be. Neither is visible in a test that only checks output shape, + * so the requests themselves are asserted. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { execFileSync, spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import * as zlib from 'node:zlib'; +import { getForgeCommand, resolveAllConcepts, executeForgeCommandDetailed } from '../lib/forge.js'; + +const codevPkgRoot = path.resolve(import.meta.dirname, '..', '..'); +const forgeScripts = path.join(codevPkgRoot, 'scripts', 'forge'); +const githubDir = path.join(forgeScripts, 'github'); +const giteaDir = path.join(forgeScripts, 'gitea'); +const fixtures = path.join(import.meta.dirname, 'fixtures', 'pir-13'); + +const CI_CONCEPTS = ['ci-runs', 'ci-run-view', 'ci-failures', 'ci-run-log'] as const; + +const ESC = ''; + +function hasJq(): boolean { + try { + execFileSync('sh', ['-c', 'command -v jq'], { stdio: 'ignore' }); + return true; + } catch { + return false; + } +} + +function fixtureLog(name: string): string { + return zlib.gunzipSync(fs.readFileSync(path.join(fixtures, `${name}.log.gz`))).toString('utf-8'); +} + +let tmp: string; + +beforeEach(() => { + tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'pir13-')); +}); + +afterEach(() => { + fs.rmSync(tmp, { recursive: true, force: true }); +}); + +interface RunResult { + status: number | null; + stdout: string; + stderr: string; + json: Record | null; +} + +/** Run a provider script with a stubbed CLI on PATH. */ +function run(dir: string, script: string, env: Record = {}): RunResult { + const result = spawnSync('sh', [path.join(dir, script)], { + cwd: tmp, + encoding: 'utf-8', + env: { + ...process.env, + PATH: `${tmp}:${process.env.PATH}`, + TMPDIR: path.join(tmp, 'cache'), + CODEV_REPO: 'o/r', + GH_LOG: path.join(tmp, 'gh.log'), + TEA_LOG: path.join(tmp, 'tea.log'), + // Every CODEV_CI_* the scripts read, cleared, so a variable set in the + // developer's own shell cannot change what a test asserts. + CODEV_CI_NO_CACHE: '1', + CODEV_BRANCH_NAME: '', CODEV_CI_STATUS: '', CODEV_CI_WORKFLOW: '', CODEV_CI_LIMIT: '', + CODEV_CI_RUN_ID: '', CODEV_CI_JOB_ID: '', CODEV_PR_BASE: '', + CODEV_CI_LOG_TAIL: '', CODEV_CI_LOG_HEAD: '', CODEV_CI_LOG_GREP: '', CODEV_CI_LOG_CONTEXT: '', + CODEV_FORGE_TIMEOUT: '', CODEV_CI_MAX_PAGES: '', CODEV_CI_TASKS_MAX_PAGES: '', + CODEV_CI_MAX_STEP_BYTES: '', CODEV_CI_MAX_BYTES: '', + ...env, + }, + }); + let json: Record | null = null; + try { + json = JSON.parse(result.stdout); + } catch { /* not every assertion needs JSON */ } + return { status: result.status, stdout: result.stdout, stderr: result.stderr, json }; +} + +/** Write an executable stub onto the test PATH. */ +function stub(name: string, body: string): void { + const p = path.join(tmp, name); + fs.writeFileSync(p, `#!/bin/sh\n${body}\n`, { mode: 0o755 }); + fs.chmodSync(p, 0o755); +} + +function fileLines(p: string): string[] { + if (!fs.existsSync(p)) return []; + return fs.readFileSync(p, 'utf-8').split('\n').filter(Boolean); +} + +// --------------------------------------------------------------------------- +// Registration +// --------------------------------------------------------------------------- + +describe('#13 — the CI concepts are registered, and disabled where unimplemented', () => { + it.each(CI_CONCEPTS)('routes %s to the github script by default', (concept) => { + const command = getForgeCommand(concept, null); + expect(command).toBe(path.join(githubDir, `${concept}.sh`)); + expect(fs.existsSync(command!)).toBe(true); + expect(fs.statSync(command!).mode & 0o111, 'script is not executable').not.toBe(0); + }); + + it.each(CI_CONCEPTS)('routes %s to the gitea script for provider gitea', (concept) => { + const command = getForgeCommand(concept, { provider: 'gitea' }); + expect(command).toBe(path.join(giteaDir, `${concept}.sh`)); + expect(fs.statSync(command!).mode & 0o111, 'script is not executable').not.toBe(0); + }); + + // The important half. A concept with no script does NOT resolve to nothing — + // it falls through to the github default, so an unimplemented ci-runs on a + // GitLab repo would silently run `gh run list`. Issue #13 asks for gitlab to + // degrade loudly; `disabled` is what loud looks like here. + it.each(['gitlab', 'linear'])( + 'DISABLES every CI concept for %s rather than letting it fall through to gh', + (provider) => { + for (const concept of CI_CONCEPTS) { + expect(getForgeCommand(concept, { provider }), `${provider}/${concept} resolved to a command`).toBeNull(); + const resolution = resolveAllConcepts({ provider }).find((r) => r.concept === concept); + expect(resolution?.source).toBe('disabled'); + } + }, + ); + + it('doctor resolves every CI script to its real CLI, not to a shell builtin', () => { + for (const provider of ['github', 'gitea'] as const) { + const expected = provider === 'github' ? 'gh' : 'tea'; + for (const concept of CI_CONCEPTS) { + const resolution = resolveAllConcepts({ provider }).find((r) => r.concept === concept); + expect(resolution?.executable, `${provider}/${concept}`).toBe(expected); + } + } + }); +}); + +// --------------------------------------------------------------------------- +// The extraction ladder, against the real logs +// --------------------------------------------------------------------------- + +describe.skipIf(!hasJq())('#13 — extraction returns the assertion, not the log and not the wrong line', () => { + /** Run the ladder the way the concepts do: clean, then extract. */ + function extract(raw: string): { rung: string; from: number; to: number; text: string } | null { + const rawPath = path.join(tmp, 'raw.log'); + const cleanPath = path.join(tmp, 'clean.log'); + fs.writeFileSync(rawPath, raw); + const script = [ + `. ${JSON.stringify(path.join(forgeScripts, '_ci-extract.sh'))}`, + `ci_clean_log < ${JSON.stringify(rawPath)} > ${JSON.stringify(cleanPath)}`, + `ci_extract ${JSON.stringify(cleanPath)}`, + ].join('\n'); + const r = spawnSync('sh', ['-c', script], { encoding: 'utf-8' }); + if (!r.stdout.trim()) return null; + const [header, ...rest] = r.stdout.split('\n'); + const [rung, from, to] = header.split('\t'); + return { rung, from: Number(from), to: Number(to), text: rest.join('\n') }; + } + + it('finds the vitest assertion in a 2528-line GitHub log', () => { + const raw = fixtureLog('github-vitest-failure'); + expect(raw.split('\n').length).toBeGreaterThan(2500); + + const got = extract(raw); + expect(got, 'extraction returned nothing on a log containing a plain vitest failure').not.toBeNull(); + expect(got!.rung).toBe('vitest'); + expect(got!.text).toContain('AssertionError: expected null to be'); + expect(got!.text).toContain('agy-auth-cache.test.ts'); + expect(got!.text).toContain('Test Files 1 failed'); + // 23 lines out of 2528. The whole point. + expect(got!.to - got!.from).toBeLessThan(60); + }); + + it('does NOT return the passing test fixture string that a first-error rule picks', () => { + // `[artifact-canvas] Error: host blew up` is printed by a test that PASSES, + // 1214 lines above the real failure. It is the single most plausible wrong + // answer this ladder can give. + const raw = fixtureLog('github-vitest-failure'); + expect(raw, 'the fixture no longer carries the decoy this test exists for').toContain('host blew up'); + expect(extract(raw)!.text).not.toContain('host blew up'); + }); + + it('does NOT anchor on any of the earlier "Test Files … passed" lines', () => { + const raw = fixtureLog('github-vitest-failure'); + const lines = raw.split('\n'); + const failing = lines.findIndex((l) => l.includes('Test Files') && l.includes('failed') && !l.includes('grep')); + const earlier = lines.slice(0, failing).filter((l) => l.includes('Test Files')); + // Four of them: three passing suite summaries, plus a shell line echoing + // `grep -q "Test Files.*passed"`, which is its own flavour of the same trap. + expect(earlier.length, 'the fixture no longer carries the trap this test exists for').toBe(4); + + const got = extract(raw)!; + expect(got.text).toContain('1 failed'); + expect(got.from).toBeGreaterThan(failing - 40); + }); + + it('matches through ANSI escapes — the raw fixture has them and cleaning is load-bearing', () => { + const raw = fixtureLog('github-vitest-failure'); + expect(raw, 'the fixture was normalised and no longer proves anything').toContain(`${ESC}[`); + // The FAIL token in the raw bytes is wrapped, so no plain /FAIL / anchor + // matches it until ci_clean_log has run. + expect(raw).toMatch(new RegExp(`${ESC}\\[[0-9;]*m[^\\n]{0,20}FAIL`)); + const got = extract(raw)!; + expect(got.text).toContain('FAIL'); + expect(got.text, 'escapes survived into the answer').not.toContain(ESC); + }); + + it('finds the Go failure in a 1599-line Forgejo log whose last 25 lines are git cleanup', () => { + const raw = fixtureLog('forgejo-go-failure'); + const lines = raw.split('\n'); + // The case for extraction over tailing, stated as an assertion. + expect(lines.slice(-25).join('\n')).not.toContain('--- FAIL'); + + const got = extract(raw)!; + expect(got.rung).toBe('go-test'); + expect(got.text).toContain('--- FAIL: TestReadPointerFromBuffer'); + expect(got.text).toContain('Should be false'); + expect(got.text).toContain('FAIL\tforgejo.org/modules/lfs'); + }); + + it('returns NOTHING when nothing is recognisable, rather than an arbitrary slice', () => { + const noise = Array.from( + { length: 400 }, + (_, i) => `2026-08-21T18:47:0${i % 10}.0000000Z step ${i} did a thing`, + ).join('\n'); + expect(extract(noise)).toBeNull(); + }); + + it('refuses a bare "Process completed with exit code 1" as a diagnosis', () => { + const log = ['Running the thing', 'more output', '##[error]Process completed with exit code 1.'].join('\n'); + expect(extract(log)).toBeNull(); + }); + + it('accepts a line-anchored error where a mid-line one is ignored', () => { + const decoy = [ + '[pkg] Error: this one is printed by a passing test', + 'ok', + 'Error: the real one', + 'trailing', + ].join('\n'); + const got = extract(decoy)!; + expect(got.rung).toBe('first-error'); + expect(got.text).toContain('Error: the real one'); + // The decoy may appear as leading CONTEXT — three lines either side is the + // rule — but it must not be the line the rung anchored on. `to` is the + // anchor plus context, so the anchor is the real one at line 3. + expect(got.to).toBe(4); + }); + + it('strips the RFC3339 timestamp both providers prefix to every line', () => { + const got = extract(['2026-08-21T18:47:09.5820646Z Error: boom', '2026-08-21T18:47:10.0000000Z after'].join('\n'))!; + expect(got.text).toContain('Error: boom'); + expect(got.text).not.toContain('2026-08-21T18:47:09'); + }); +}); + +// --------------------------------------------------------------------------- +// ci-failures, end to end against a stubbed gh +// --------------------------------------------------------------------------- + +/** A gh stub serving `run view --json` and `api …/logs` from files. */ +function stubGh(runJson: unknown, logFile?: string): void { + fs.writeFileSync(path.join(tmp, 'run.json'), JSON.stringify(runJson)); + stub('gh', [ + 'printf "%s\\n" "$*" >> "$GH_LOG"', + 'if [ "$1" = "run" ] && [ "$2" = "view" ] && [ "$4" = "--json" ]; then cat "$(dirname "$0")/run.json"; exit 0; fi', + 'if [ "$1" = "run" ] && [ "$2" = "list" ]; then cat "$(dirname "$0")/run.json"; exit 0; fi', + logFile + ? `if [ "$1" = "api" ]; then cat ${JSON.stringify(logFile)}; exit 0; fi` + : 'if [ "$1" = "api" ]; then echo "no log" >&2; exit 1; fi', + 'echo "unexpected gh invocation: $*" >&2', + 'exit 9', + ].join('\n')); +} + +const ONE_FAILING_JOB = { + databaseId: 32515040122, + number: 42, + displayTitle: 'a commit', + workflowName: 'Tests', + name: 'Tests', + status: 'completed', + conclusion: 'failure', + headBranch: 'builder/pir-13', + headSha: 'abc123', + event: 'push', + url: 'https://github.com/o/r/actions/runs/32515040122', + createdAt: '2026-08-21T18:46:21Z', + jobs: [ + { databaseId: 1, name: 'Lint', status: 'completed', conclusion: 'success', startedAt: null, completedAt: null, steps: [] }, + { + databaseId: 96874679182, name: 'Unit Tests', status: 'completed', conclusion: 'failure', + startedAt: '2026-08-21T18:47:08Z', completedAt: '2026-08-21T18:49:07Z', + steps: [ + { name: 'Checkout', number: 1, status: 'completed', conclusion: 'success' }, + { name: 'Run unit tests with coverage', number: 15, status: 'completed', conclusion: 'failure' }, + ], + }, + ], +}; + +describe.skipIf(!hasJq())('#13 — ci-failures returns a bounded extract with its provenance', () => { + function writeFixtureTo(name: string): string { + const p = path.join(tmp, `${name}.log`); + fs.writeFileSync(p, fixtureLog(name)); + return p; + } + + it('turns a 293 KB job log into a response of about a kilobyte', () => { + const logPath = writeFixtureTo('github-vitest-failure'); + stubGh(ONE_FAILING_JOB, logPath); + + const r = run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122' }); + expect(r.status).toBe(0); + expect(r.json!.ok).toBe(true); + expect(r.json!.extracted).toBe(true); + expect(r.json!.jobsFailed).toBe(1); + + const f = r.json!.failures[0]; + expect(f.jobId).toBe(96874679182); + expect(f.jobName).toBe('Unit Tests'); + // The failing STEP name comes from structured JSON, not from parsing a log: + // `gh run view --log-failed` labels its lines UNKNOWN STEP whenever its + // filename-to-step mapping misses, which on the run this was captured from + // was every line of all 2528. + expect(f.stepName).toBe('Run unit tests with coverage'); + expect(f.stepNumber).toBe(15); + expect(f.matchedBy).toBe('vitest'); + expect(f.text).toContain('AssertionError: expected null to be'); + expect(f.logLines).toBe(2528); + expect(f.returnedLines).toBeLessThan(60); + + expect(fs.statSync(logPath).size).toBeGreaterThan(200_000); + expect(r.stdout.length, 'the response is no longer bounded').toBeLessThan(8_192); + }); + + it('always reports logLines, returnedLines and truncated together', () => { + stubGh(ONE_FAILING_JOB, writeFixtureTo('github-vitest-failure')); + const f = run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122' }).json!.failures[0]; + for (const key of ['logLines', 'returnedLines', 'truncated']) { + expect(f, `a trimmed answer without ${key} reads as a whole one`).toHaveProperty(key); + } + }); + + it('marks truncated when the cap bites, and keeps whole lines', () => { + stubGh(ONE_FAILING_JOB, writeFixtureTo('github-vitest-failure')); + const r = run(githubDir, 'ci-failures.sh', { + CODEV_CI_RUN_ID: '32515040122', + CODEV_CI_MAX_STEP_BYTES: '200', + }); + const f = r.json!.failures[0]; + expect(f.truncated).toBe(true); + expect(f.returnedLines).toBeLessThan(23); + expect(f.text.length).toBeLessThanOrEqual(260); + expect(f.logLines).toBe(2528); + }); + + it('hands over instead of guessing when nothing is recognisable', () => { + const noise = path.join(tmp, 'noise.log'); + fs.writeFileSync(noise, `${Array.from({ length: 900 }, (_, i) => `2026-08-21T18:47:00.000Z doing thing ${i}`).join('\n')}\n`); + stubGh(ONE_FAILING_JOB, noise); + + const r = run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122' }); + expect(r.json!.extracted).toBe(false); + expect(r.json!.reason).toBe('no recognized failure pattern'); + expect(r.json!.failures[0]).toEqual({ jobId: 96874679182, jobName: 'Unit Tests', logLines: 900 }); + expect(r.json!.next).toContain('CODEV_CI_JOB_ID=96874679182'); + // The rule this whole issue turns on: no arbitrary lines, ever. + expect(r.stdout).not.toContain('doing thing'); + expect(r.json!.failures[0]).not.toHaveProperty('text'); + }); + + it('says a green run is green, and distinguishes it from a run still going', () => { + stubGh({ ...ONE_FAILING_JOB, conclusion: 'success', jobs: [ONE_FAILING_JOB.jobs[0]] }); + const green = run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122' }); + expect(green.status).toBe(0); + expect(green.json!.jobsFailed).toBe(0); + expect(green.json!.reason).toBe('no job in this run failed'); + + stubGh({ ...ONE_FAILING_JOB, status: 'in_progress', conclusion: null, jobs: [ONE_FAILING_JOB.jobs[0]] }); + const running = run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122' }); + expect(running.json!.reason).toBe('run has not finished'); + }); + + it('rejects a job id that is not in the run instead of answering "nothing failed"', () => { + stubGh(ONE_FAILING_JOB, writeFixtureTo('github-vitest-failure')); + const r = run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_JOB_ID: '999' }); + expect(r.status).not.toBe(0); + expect(r.json!.error).toBe('not-found'); + expect(r.json!.ok).toBe(false); + }); + + it('fetches the per-job log endpoint, never --log-failed', () => { + stubGh(ONE_FAILING_JOB, writeFixtureTo('github-vitest-failure')); + run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122' }); + const calls = fileLines(path.join(tmp, 'gh.log')); + expect(calls.some((c) => c.includes('actions/jobs/96874679182/logs'))).toBe(true); + expect(calls.some((c) => c.includes('--log-failed')), '--log-failed returns the whole job, UNKNOWN STEP-tagged').toBe(false); + }); + + it('caches a terminal job log so the second question costs no download', () => { + const logPath = writeFixtureTo('github-vitest-failure'); + stubGh(ONE_FAILING_JOB, logPath); + const env = { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_NO_CACHE: '' }; + + const first = run(githubDir, 'ci-failures.sh', env); + expect(first.json!.cached).toBe(false); + const second = run(githubDir, 'ci-failures.sh', env); + expect(second.json!.cached).toBe(true); + expect(second.json!.failures[0].text).toBe(first.json!.failures[0].text); + }); + + it('never caches a job that is still running', () => { + const running = { + ...ONE_FAILING_JOB, + status: 'in_progress', + jobs: [{ ...ONE_FAILING_JOB.jobs[1], status: 'in_progress', conclusion: null }], + }; + stubGh(running, writeFixtureTo('github-vitest-failure')); + const env = { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_JOB_ID: '96874679182', CODEV_CI_NO_CACHE: '' }; + run(githubDir, 'ci-failures.sh', env); + const second = run(githubDir, 'ci-failures.sh', env); + expect(second.json!.cached, 'a half-written log was cached and will read as complete forever').toBe(false); + }); +}); + +// --------------------------------------------------------------------------- +// ci-run-log +// --------------------------------------------------------------------------- + +describe.skipIf(!hasJq())('#13 — ci-run-log takes exactly one window, and says where it looked', () => { + function stubWithFixture(): void { + const p = path.join(tmp, 'gh-fixture.log'); + fs.writeFileSync(p, fixtureLog('github-vitest-failure')); + stubGh(ONE_FAILING_JOB, p); + } + + it('refuses with no window, and refuses BEFORE calling the forge', () => { + stubWithFixture(); + const r = run(githubDir, 'ci-run-log.sh', { CODEV_CI_RUN_ID: '32515040122' }); + expect(r.status).toBe(2); + expect(r.json!.error).toBe('bad-input'); + expect(r.json!.detail).toContain('exactly one window'); + expect(fileLines(path.join(tmp, 'gh.log')), 'a malformed request cost an API call').toEqual([]); + }); + + it('refuses two windows — a default is how deliberate decays into always', () => { + stubWithFixture(); + const r = run(githubDir, 'ci-run-log.sh', { + CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_TAIL: '5', CODEV_CI_LOG_HEAD: '5', + }); + expect(r.status).toBe(2); + expect(r.json!.detail).toContain('exactly one window may be set'); + }); + + it('refuses a non-numeric line count, naming the variable in the caller spelling', () => { + stubWithFixture(); + const r = run(githubDir, 'ci-run-log.sh', { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_TAIL: 'lots' }); + expect(r.status).toBe(2); + expect(r.json!.detail).toContain('CODEV_CI_LOG_TAIL'); + }); + + it('tail returns the last N lines with absolute line numbers', () => { + stubWithFixture(); + const r = run(githubDir, 'ci-run-log.sh', { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_TAIL: '5' }); + expect(r.status).toBe(0); + expect(r.json!.logLines).toBe(2528); + expect(r.json!.returnedLines).toBe(5); + expect(r.json!.from).toBe(2524); + expect(r.json!.to).toBe(2528); + expect(r.json!.contiguous).toBe(true); + expect(r.json!.lines).toHaveLength(5); + }); + + it('head returns the first N lines', () => { + stubWithFixture(); + const r = run(githubDir, 'ci-run-log.sh', { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_HEAD: '3' }); + expect(r.json!.from).toBe(1); + expect(r.json!.to).toBe(3); + expect(r.json!.lines[0]).toContain('Current runner version'); + }); + + it('grep returns context, marks itself non-contiguous, and lists which lines matched', () => { + stubWithFixture(); + const r = run(githubDir, 'ci-run-log.sh', { + CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_GREP: 'AssertionError', CODEV_CI_LOG_CONTEXT: '1', + }); + expect(r.json!.matches).toBe(2); + expect(r.json!.matchLines).toEqual([2472, 2497]); + expect(r.json!.contiguous).toBe(false); + expect(r.json!.returnedLines).toBe(6); + }); + + it('honours the context width', () => { + stubWithFixture(); + const narrow = run(githubDir, 'ci-run-log.sh', { + CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_GREP: 'AssertionError', CODEV_CI_LOG_CONTEXT: '0', + }); + expect(narrow.json!.returnedLines).toBe(2); + }); + + it('reports no match as an empty window rather than as a failure', () => { + stubWithFixture(); + const r = run(githubDir, 'ci-run-log.sh', { + CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_GREP: 'zzz-definitely-not-present', + }); + expect(r.status).toBe(0); + expect(r.json!.ok).toBe(true); + expect(r.json!.matches).toBe(0); + expect(r.json!.lines).toEqual([]); + expect(r.json!.logLines).toBe(2528); + }); + + it('refuses to pick a job when several passed and none failed', () => { + const green = { + ...ONE_FAILING_JOB, + conclusion: 'success', + jobs: [ + { databaseId: 1, name: 'A', status: 'completed', conclusion: 'success', steps: [] }, + { databaseId: 2, name: 'B', status: 'completed', conclusion: 'success', steps: [] }, + ], + }; + stubGh(green, path.join(tmp, 'nothing.log')); + const r = run(githubDir, 'ci-run-log.sh', { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_LOG_TAIL: '5' }); + expect(r.status).toBe(2); + expect(r.json!.detail).toContain('set CODEV_CI_JOB_ID'); + expect(r.json!.jobs).toHaveLength(2); + }); +}); + +// --------------------------------------------------------------------------- +// Timeouts +// --------------------------------------------------------------------------- + +describe.skipIf(!hasJq())('#13 — a timeout reports as a timeout, not as an empty result', () => { + it('names the timeout on stdout and on stderr, and exits non-zero', () => { + // A stub that spawns a child and waits: killing the wrapper alone leaves the + // child holding the stdout pipe, which is the shape that made an earlier + // watchdog print its message and hang anyway (#12). + stub('gh', 'sleep 60 &\nwait'); + const started = Date.now(); + const r = run(githubDir, 'ci-runs.sh', { CODEV_FORGE_TIMEOUT: '2' }); + const elapsed = Date.now() - started; + + expect(r.status).not.toBe(0); + expect(r.json!.ok).toBe(false); + expect(r.json!.error).toBe('timeout'); + expect(r.json!.seconds).toBe(2); + expect(r.json!.detail).toContain('gh run list'); + expect(r.json!.remedy).toContain('CODEV_FORGE_TIMEOUT'); + expect(r.stderr).toContain('did not return within 2s'); + expect(elapsed, 'the timeout did not actually unblock the caller').toBeLessThan(30_000); + }, 40_000); + + it('does not leave the shell watchdog notice on a concept stderr', () => { + stubGh(ONE_FAILING_JOB); + const r = run(githubDir, 'ci-run-view.sh', { CODEV_CI_RUN_ID: '32515040122' }); + expect(r.status).toBe(0); + expect(r.stderr, 'the watchdog is reporting its own death onto a clean path').toBe(''); + }); + + it('executeForgeCommandDetailed distinguishes a timeout from a failure and from silence', async () => { + const slow = path.join(tmp, 'slow.sh'); + fs.writeFileSync(slow, '#!/bin/sh\nsleep 30\n', { mode: 0o755 }); + fs.chmodSync(slow, 0o755); + + const timedOut = await executeForgeCommandDetailed('ci-runs', {}, { + forgeConfig: { 'ci-runs': slow }, + timeoutMs: 500, + }); + expect(timedOut.ok).toBe(false); + expect(timedOut.timedOut).toBe(true); + expect(timedOut.unavailable).toBe(false); + + const failing = path.join(tmp, 'fail.sh'); + fs.writeFileSync(failing, '#!/bin/sh\necho \'{"ok":false,"error":"forge-error"}\'\nexit 1\n', { mode: 0o755 }); + fs.chmodSync(failing, 0o755); + + const failed = await executeForgeCommandDetailed('ci-runs', {}, { forgeConfig: { 'ci-runs': failing } }); + expect(failed.ok).toBe(false); + expect(failed.timedOut, 'a plain failure was reported as a timeout').toBe(false); + // stdout survives a non-zero exit — the whole reason the envelope is printed + // there rather than only on stderr. + expect((failed.data as Record).error).toBe('forge-error'); + + const disabled = await executeForgeCommandDetailed('ci-runs', {}, { forgeConfig: { 'ci-runs': null } }); + expect(disabled.unavailable).toBe(true); + expect(disabled.timedOut).toBe(false); + }, 20_000); +}); + +// --------------------------------------------------------------------------- +// Forgejo: the measured footguns, and the version gate +// --------------------------------------------------------------------------- + +/** + * A fake `tea` serving the `tea api` slice these scripts use. + * + * TEA_ROUTES is a `\t` table, first match wins, body emitted + * verbatim from a FILE — a log contains newlines, which a line-oriented table + * would silently truncate. Every requested endpoint is appended to TEA_LOG, so + * a test can assert what was called AND what was not. + */ +const TEA_STUB = [ + 'if [ "$1" != "api" ]; then echo "unexpected tea invocation: $*" >&2; exit 9; fi', + 'for a in "$@"; do ep=$a; done', + 'printf "%s\\n" "$ep" >> "$TEA_LOG"', + 'while IFS="\t" read -r pattern bodyfile; do', + ' [ -n "$pattern" ] || continue', + ' case "$ep" in', + ' $pattern) cat "$bodyfile"; exit 0 ;;', + ' esac', + 'done < "$TEA_ROUTES"', + 'printf "404 page not found\\n"', + 'exit 0', +].join('\n'); + +function stubTea(routes: Array<[string, unknown | { raw: string }]>): void { + const table: string[] = []; + routes.forEach(([pattern, body], i) => { + const file = path.join(tmp, `route-${i}.body`); + const content = typeof body === 'object' && body !== null && 'raw' in (body as any) + ? (body as { raw: string }).raw + : JSON.stringify(body); + fs.writeFileSync(file, content); + table.push(`${pattern}\t${file}`); + }); + fs.writeFileSync(path.join(tmp, 'routes.tsv'), `${table.join('\n')}\n`); + stub('tea', TEA_STUB); +} + +const FORGEJO_15_VERSION = { version: '15.0.2+gitea-1.22.0' }; + +/** One page of `actions/runs`, reduced to the fields the scripts read. */ +function forgejoRuns(runs: Array>): Record { + return { total_count: runs.length, workflow_runs: runs }; +} + +function forgejoRun(id: number, index: number, extra: Record = {}): Record { + return { + id, index_in_repo: index, title: `run ${index}`, workflow_id: 'ci.yml', + status: 'failure', prettyref: 'main', commit_sha: 'deadbeef', event: 'push', + html_url: `https://forge.example.com/o/r/actions/runs/${index}`, created: '2026-08-18T21:47:24Z', + ...extra, + }; +} + +describe.skipIf(!hasJq())('#13 — Forgejo, as it actually behaves', () => { + function teaEnv(extra: Record = {}): Record { + return { TEA_ROUTES: path.join(tmp, 'routes.tsv'), ...extra }; + } + + it('always sends page= with limit=, because Forgejo ignores limit without it', () => { + // Measured: `actions/runs?limit=3` returned all 6922 runs on the reference + // Forgejo. This is the kind of default that turns into a #12. + stubTea([['repos/o/r/actions/runs?*', forgejoRuns([forgejoRun(11130, 6881)])]]); + run(giteaDir, 'ci-runs.sh', teaEnv({ CODEV_CI_LIMIT: '3' })); + const calls = fileLines(path.join(tmp, 'tea.log')); + expect(calls.length).toBeGreaterThan(0); + for (const call of calls.filter((c) => c.includes('limit='))) { + expect(call, `a list request without page=: ${call}`).toContain('page='); + } + }); + + it('matches a branch by its PULL REQUEST ref, which is what Forgejo records', () => { + // A pull_request run records `#3855`, not `builder/air-364`. Filtering on + // the branch name alone matches nothing on a repo that runs CI on PRs. + stubTea([ + ['repos/o/r', { default_branch: 'main' }], + ['repos/o/r/pulls/main/builder/air-364', { number: 3855, state: 'open', head: { label: 'builder/air-364' } }], + ['repos/o/r/actions/runs?*', forgejoRuns([ + forgejoRun(11142, 6886, { prettyref: '#3855', event: 'pull_request', status: 'success' }), + forgejoRun(11100, 6870, { prettyref: '#9999', event: 'pull_request' }), + forgejoRun(11000, 6800, { prettyref: 'main' }), + ])], + ]); + + const r = run(giteaDir, 'ci-runs.sh', teaEnv({ CODEV_BRANCH_NAME: 'builder/air-364' })); + expect(r.status).toBe(0); + expect(r.json!.runs).toHaveLength(1); + expect(r.json!.runs[0].id).toBe(11142); + expect(r.json!.runs[0].branch).toBe('#3855'); + }); + + it('says so when a branch has no PR, instead of returning a bare empty list', () => { + stubTea([ + ['repos/o/r', { default_branch: 'main' }], + ['repos/o/r/actions/runs?*', forgejoRuns([forgejoRun(11000, 6800, { prettyref: 'main' })])], + ]); + const r = run(giteaDir, 'ci-runs.sh', teaEnv({ CODEV_BRANCH_NAME: 'builder/no-pr' })); + expect(r.json!.runs).toEqual([]); + expect(r.json!.note, 'an empty list with no explanation reads as "CI never ran"').toContain('no pull request'); + }); + + it('emits conclusion: null rather than inventing one Forgejo does not have', () => { + stubTea([['repos/o/r/actions/runs?*', forgejoRuns([forgejoRun(11130, 6881)])]]); + const r = run(giteaDir, 'ci-runs.sh', teaEnv()); + expect(r.json!.runs[0].conclusion).toBeNull(); + expect(r.json!.runs[0].status).toBe('failure'); + }); + + it('ci-run-view recovers jobs from the task list when the jobs API is absent', () => { + stubTea([ + ['repos/o/r/actions/runs/11130', forgejoRun(11130, 6881)], + // no repos/o/r/actions/runs/11130/jobs route -> the stub 404s, as 15.0.2 does + ['repos/o/r/actions/tasks?*', { + total_count: 2, + workflow_runs: [ + { id: 40084, name: 'E2E coverage-drift guard', run_number: 6881, status: 'failure', run_started_at: 'x', updated_at: 'y' }, + { id: 40082, name: 'Lint', run_number: 6881, status: 'success', run_started_at: 'x', updated_at: 'y' }, + ], + }], + ]); + + const r = run(giteaDir, 'ci-run-view.sh', teaEnv({ CODEV_CI_RUN_ID: '11130' })); + expect(r.status).toBe(0); + expect(r.json!.jobSource).toBe('tasks-scan'); + expect(r.json!.jobs).toHaveLength(2); + // A task id is not a job id and the log API does not accept one, so it is + // never reported as `id`. + expect(r.json!.jobs[0].id).toBeNull(); + expect(r.json!.jobs[0].taskId).toBe(40084); + expect(r.json!.jobs[0].status).toBe('failure'); + }); + + it('ci-failures on Forgejo 15 says unsupported-server, names both versions, and still names the failing job', () => { + stubTea([ + ['repos/o/r/actions/runs/11130', forgejoRun(11130, 6881)], + ['repos/o/r/actions/tasks?*', { + total_count: 1, + workflow_runs: [{ id: 40084, name: 'E2E coverage-drift guard', run_number: 6881, status: 'failure', run_started_at: 'x', updated_at: 'y' }], + }], + ['version', FORGEJO_15_VERSION], + ]); + + const r = run(giteaDir, 'ci-failures.sh', teaEnv({ CODEV_CI_RUN_ID: '11130' })); + expect(r.status).not.toBe(0); + expect(r.json!.ok).toBe(false); + expect(r.json!.error).toBe('unsupported-server'); + expect(r.json!.serverVersion).toBe('15.0.2+gitea-1.22.0'); + expect(r.json!.needs).toBe('>=16.0'); + expect(r.json!.detail).toContain('Forgejo 16.0'); + // The rule the architect held this to: an unsupported server must never + // look like a run with no failures. + expect(r.json!, 'an old server produced an empty failures array').not.toHaveProperty('failures'); + expect(r.json!.jobsFailed).toBe(1); + expect(r.json!.failingJobs[0].jobName).toBe('E2E coverage-drift guard'); + }); + + it('ci-run-log on Forgejo 15 says unsupported-server rather than an empty window', () => { + stubTea([ + ['repos/o/r/actions/runs/11130', forgejoRun(11130, 6881)], + ['repos/o/r/actions/tasks?*', { total_count: 0, workflow_runs: [] }], + ['version', FORGEJO_15_VERSION], + ]); + const r = run(giteaDir, 'ci-run-log.sh', teaEnv({ CODEV_CI_RUN_ID: '11130', CODEV_CI_LOG_TAIL: '20' })); + expect(r.json!.error).toBe('unsupported-server'); + expect(r.json!).not.toHaveProperty('lines'); + }); + + it('ci-failures on Forgejo 16 extracts from the job-log endpoint', () => { + stubTea([ + ['repos/o/r/actions/runs/6554924', forgejoRun(6554924, 189242)], + ['repos/o/r/actions/runs/6554924/jobs', [ + { id: 11952743, task_id: 8848592, name: 'backend-checks', status: 'success' }, + { id: 11952749, task_id: 8848703, name: 'test-unit', status: 'failure' }, + ]], + ['repos/o/r/actions/jobs/11952749/logs', { raw: fixtureLog('forgejo-go-failure') }], + ]); + + const r = run(giteaDir, 'ci-failures.sh', teaEnv({ CODEV_CI_RUN_ID: '6554924' })); + expect(r.status).toBe(0); + expect(r.json!.extracted).toBe(true); + const f = r.json!.failures[0]; + // The log route takes the JOB id (11952749), not the task id (8848703). + expect(f.jobId).toBe(11952749); + expect(f.jobName).toBe('test-unit'); + expect(f.matchedBy).toBe('go-test'); + expect(f.text).toContain('--- FAIL: TestReadPointerFromBuffer'); + expect(f.logLines).toBe(1599); + expect(fileLines(path.join(tmp, 'tea.log')).some((c) => c.includes('actions/jobs/11952749/logs'))).toBe(true); + }); + + it('ci-run-log on Forgejo 16 windows the same log', () => { + stubTea([ + ['repos/o/r/actions/runs/6554924', forgejoRun(6554924, 189242)], + ['repos/o/r/actions/runs/6554924/jobs', [{ id: 11952749, task_id: 8848703, name: 'test-unit', status: 'failure' }]], + ['repos/o/r/actions/jobs/11952749/logs', { raw: fixtureLog('forgejo-go-failure') }], + ]); + const r = run(giteaDir, 'ci-run-log.sh', teaEnv({ CODEV_CI_RUN_ID: '6554924', CODEV_CI_LOG_GREP: '--- FAIL', CODEV_CI_LOG_CONTEXT: '0' })); + expect(r.json!.matches).toBe(1); + expect(r.json!.lines[0]).toContain('--- FAIL: TestReadPointerFromBuffer'); + }); + + it('rejects a status outside the shared vocabulary instead of passing it to the forge', () => { + stubTea([['repos/o/r/actions/runs?*', forgejoRuns([])]]); + const r = run(giteaDir, 'ci-runs.sh', teaEnv({ CODEV_CI_STATUS: 'broken' })); + expect(r.status).toBe(2); + expect(r.json!.error).toBe('bad-input'); + expect(r.json!.detail).toContain('in_progress'); + expect(fileLines(path.join(tmp, 'tea.log'))).toEqual([]); + }); +}); From 5cd31ef16e6392c6431dc97a89af361627c84697 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:30:58 -0600 Subject: [PATCH 10/30] [PIR #13] fix(forge): send status=cancelled, the spelling both forges actually accept The shared vocabulary spells it canceled because that is what `tea actions runs list --help` documents. Measured against the live Forgejo: status=canceled returns {"message":"unknown status: canceled"} and status=cancelled returns 2240 runs. GitHub has always wanted cancelled too. So the translation applies to both providers, not just to GitHub. Co-Authored-By: Claude Opus 5 --- packages/codev/scripts/forge/_ci-lib.sh | 13 +++++++++---- .../src/__tests__/pir-13-ci-concepts.test.ts | 19 +++++++++++++++++++ 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/packages/codev/scripts/forge/_ci-lib.sh b/packages/codev/scripts/forge/_ci-lib.sh index 0f0c44e6d..991bab49d 100644 --- a/packages/codev/scripts/forge/_ci-lib.sh +++ b/packages/codev/scripts/forge/_ci-lib.sh @@ -99,12 +99,17 @@ ci_check_status() { exit 2 } -# Translate the shared vocabulary into what a provider CLI expects. -# GitHub spells it "cancelled"; Forgejo spells it "canceled". Everything else is -# shared, which is why the vocabulary is worth having. +# Translate the shared vocabulary into what a provider expects on the wire. +# +# The vocabulary spells it `canceled` because that is what `tea actions runs +# list --help` documents. Both forges want `cancelled`: GitHub has always spelt +# it that way, and Forgejo — despite its own CLI help — answers +# `status=canceled` with `{"message":"unknown status: canceled"}` while +# `status=cancelled` returns 2240 runs. Measured, because the two spellings are +# exactly the kind of difference that turns into an empty list nobody questions. ci_status_for() { _provider="$1"; _status="$2" - if [ "$_provider" = "github" ] && [ "$_status" = "canceled" ]; then + if [ "$_status" = "canceled" ]; then printf 'cancelled' else printf '%s' "$_status" diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts index 72db117ba..01d624577 100644 --- a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -430,6 +430,13 @@ describe.skipIf(!hasJq())('#13 — ci-failures returns a bounded extract with it expect(r.json!.ok).toBe(false); }); + it('translates the shared canceled into gh cancelled', () => { + stubGh([]); + run(githubDir, 'ci-runs.sh', { CODEV_CI_STATUS: 'canceled' }); + const calls = fileLines(path.join(tmp, 'gh.log')); + expect(calls.some((c) => c.includes('--status cancelled'))).toBe(true); + }); + it('fetches the per-job log endpoint, never --log-failed', () => { stubGh(ONE_FAILING_JOB, writeFixtureTo('github-vitest-failure')); run(githubDir, 'ci-failures.sh', { CODEV_CI_RUN_ID: '32515040122' }); @@ -832,6 +839,18 @@ describe.skipIf(!hasJq())('#13 — Forgejo, as it actually behaves', () => { expect(r.json!.lines[0]).toContain('--- FAIL: TestReadPointerFromBuffer'); }); + it('sends status=cancelled, because Forgejo rejects the spelling its own CLI documents', () => { + // `tea actions runs list --help` says `canceled`; the API answers that with + // {"message":"unknown status: canceled"} and answers `cancelled` with 2240 + // runs. Both measured. This is the kind of difference that becomes an empty + // list nobody questions. + stubTea([['repos/o/r/actions/runs?*', forgejoRuns([])]]); + run(giteaDir, 'ci-runs.sh', teaEnv({ CODEV_CI_STATUS: 'canceled' })); + const calls = fileLines(path.join(tmp, 'tea.log')); + expect(calls.some((c) => c.includes('status=cancelled'))).toBe(true); + expect(calls.some((c) => c.includes('status=canceled'))).toBe(false); + }); + it('rejects a status outside the shared vocabulary instead of passing it to the forge', () => { stubTea([['repos/o/r/actions/runs?*', forgejoRuns([])]]); const r = run(giteaDir, 'ci-runs.sh', teaEnv({ CODEV_CI_STATUS: 'broken' })); From 3e552014e13161b69105e11de2e906277d9b7d56 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:32:07 -0600 Subject: [PATCH 11/30] [PIR #13] fix(forge): a CLI that exits 0 with non-JSON still gets an envelope MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit gh can exit 0 and print an auth prompt, an empty body, or an HTML error page. That reached jq, died under `set -e`, and left jq's diagnostic on stderr with NOTHING on stdout — the one shape these concepts promised never to produce. Co-Authored-By: Claude Opus 5 --- packages/codev/scripts/forge/_ci-lib.sh | 19 +++++++++++++++++++ .../codev/scripts/forge/github/ci-failures.sh | 2 ++ .../codev/scripts/forge/github/ci-run-log.sh | 2 ++ .../codev/scripts/forge/github/ci-run-view.sh | 2 ++ .../codev/scripts/forge/github/ci-runs.sh | 2 ++ .../src/__tests__/pir-13-ci-concepts.test.ts | 12 ++++++++++++ 6 files changed, 39 insertions(+) diff --git a/packages/codev/scripts/forge/_ci-lib.sh b/packages/codev/scripts/forge/_ci-lib.sh index 991bab49d..f5b113123 100644 --- a/packages/codev/scripts/forge/_ci-lib.sh +++ b/packages/codev/scripts/forge/_ci-lib.sh @@ -125,6 +125,25 @@ ci_status_is_terminal() { esac } +# Assert that a captured payload really is JSON, and emit an envelope if not. +# +# ci_require_json "" "" +# +# A forge CLI that exits 0 and prints something other than JSON — an auth +# prompt, an empty body, an HTML error page — would otherwise reach `jq` and +# kill the script under `set -e`, leaving jq's own diagnostic on stderr and +# NOTHING on stdout. That is the one shape these concepts promised never to +# produce: a caller with no structured answer at all. +ci_require_json() { + _concept="$1"; _payload="$2"; _what="$3" + if printf '%s' "$_payload" | jq -e . >/dev/null 2>&1; then + return 0 + fi + ci_fail "$_concept" forge-error \ + "${_what} did not return JSON: $(printf '%s' "$_payload" | head -c 200 | tr '\n' ' ')" + exit 1 +} + # --------------------------------------------------------------------------- # Log cache # --------------------------------------------------------------------------- diff --git a/packages/codev/scripts/forge/github/ci-failures.sh b/packages/codev/scripts/forge/github/ci-failures.sh index d203e2d7e..050556bb6 100755 --- a/packages/codev/scripts/forge/github/ci-failures.sh +++ b/packages/codev/scripts/forge/github/ci-failures.sh @@ -48,6 +48,8 @@ if [ "$rc" -ne 0 ]; then exit 1 fi +ci_require_json "$CONCEPT" "$RUN" "gh run view ${CODEV_CI_RUN_ID}" + FAILED=$(printf '%s' "$RUN" | jq -c "$GH_FAILED_JOBS_JQ") RUN_STATUS=$(printf '%s' "$RUN" | jq -r '.status // "unknown"') RUN_CONCLUSION=$(printf '%s' "$RUN" | jq -r '.conclusion // "null"') diff --git a/packages/codev/scripts/forge/github/ci-run-log.sh b/packages/codev/scripts/forge/github/ci-run-log.sh index b037c3967..109ea4b11 100755 --- a/packages/codev/scripts/forge/github/ci-run-log.sh +++ b/packages/codev/scripts/forge/github/ci-run-log.sh @@ -49,6 +49,8 @@ if [ "$rc" -ne 0 ]; then exit 1 fi +ci_require_json "$CONCEPT" "$RUN" "gh run view ${CODEV_CI_RUN_ID}" + if [ -n "$CODEV_CI_JOB_ID" ]; then TARGET=$(printf '%s' "$RUN" | jq -c --argjson j "$CODEV_CI_JOB_ID" 'first(.jobs[] | select(.databaseId == $j)) // empty') if [ -z "$TARGET" ]; then diff --git a/packages/codev/scripts/forge/github/ci-run-view.sh b/packages/codev/scripts/forge/github/ci-run-view.sh index 73f9704d9..13eb06e1c 100755 --- a/packages/codev/scripts/forge/github/ci-run-view.sh +++ b/packages/codev/scripts/forge/github/ci-run-view.sh @@ -37,6 +37,8 @@ if [ "$rc" -ne 0 ]; then exit 1 fi +ci_require_json "$CONCEPT" "$OUT" "gh run view ${CODEV_CI_RUN_ID}" + printf '%s' "$OUT" | jq -c '{ ok: true, provider: "github", diff --git a/packages/codev/scripts/forge/github/ci-runs.sh b/packages/codev/scripts/forge/github/ci-runs.sh index 7a3c8bb89..2de456c65 100755 --- a/packages/codev/scripts/forge/github/ci-runs.sh +++ b/packages/codev/scripts/forge/github/ci-runs.sh @@ -52,6 +52,8 @@ if [ "$rc" -ne 0 ]; then exit 1 fi +ci_require_json "$CONCEPT" "$OUT" "gh run list" + printf '%s' "$OUT" | jq -c --argjson limit "$LIMIT" '{ ok: true, provider: "github", diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts index 01d624577..0565d4f41 100644 --- a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -430,6 +430,18 @@ describe.skipIf(!hasJq())('#13 — ci-failures returns a bounded extract with it expect(r.json!.ok).toBe(false); }); + it('emits an envelope when the forge CLI exits 0 with something that is not JSON', () => { + // An auth prompt, an empty body, an HTML error page. Without a guard this + // reaches jq, dies under `set -e`, and leaves jq's diagnostic on stderr + // with NOTHING on stdout — the one shape these concepts promised never to + // produce. + stub('gh', 'echo "gh: not logged into any hosts"'); + const r = run(githubDir, 'ci-run-view.sh', { CODEV_CI_RUN_ID: '32515040122' }); + expect(r.status).not.toBe(0); + expect(r.json!.error).toBe('forge-error'); + expect(r.json!.detail).toContain('did not return JSON'); + }); + it('translates the shared canceled into gh cancelled', () => { stubGh([]); run(githubDir, 'ci-runs.sh', { CODEV_CI_STATUS: 'canceled' }); From 494a173528c61ea47e7ad1ba9a454f8a4bb49c7e Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:34:10 -0600 Subject: [PATCH 12/30] [PIR #13] fix(forge): gitea ci-runs reports truncation when it hits the page ceiling Asking for 200 runs from a 6922-run repository collected exactly 200 and reported truncated=false, because the ceiling check only fired when a client-side branch or workflow filter was active. A capped answer that says it is complete is the failure this issue is about. Co-Authored-By: Claude Opus 5 --- packages/codev/scripts/forge/gitea/ci-runs.sh | 25 +++++++++++++------ 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/packages/codev/scripts/forge/gitea/ci-runs.sh b/packages/codev/scripts/forge/gitea/ci-runs.sh index 1b098cd9a..915e1b79b 100755 --- a/packages/codev/scripts/forge/gitea/ci-runs.sh +++ b/packages/codev/scripts/forge/gitea/ci-runs.sh @@ -85,18 +85,27 @@ while [ "$PAGE" -le "$CI_MAX_PAGES" ]; do } ]' "$TMP/page.json") ACC=$(printf '%s\n%s' "$ACC" "$HITS" | jq -s -c 'add') - [ "$(printf '%s' "$ACC" | jq 'length')" -ge "$LIMIT" ] && break + # -gt, not -ge: stopping at exactly LIMIT would leave `truncated` (computed + # below as length > limit) reading false when there is more to see. One extra + # item is what makes "there are more runs than this" a fact rather than a + # guess — the same reason the github script asks gh for LIMIT + 1. + [ "$(printf '%s' "$ACC" | jq 'length')" -gt "$LIMIT" ] && break if [ "$RAW" -lt "$GITEA_PAGE_LIMIT" ]; then break; fi PAGE=$((PAGE + 1)) - # Ran out of pages we are allowed to walk while a client-side filter was still - # discarding candidates. Say so: a short list and a truncated one look - # identical once printed. - if [ "$PAGE" -gt "$CI_MAX_PAGES" ] && { [ -n "$CODEV_BRANCH_NAME" ] || [ -n "$CODEV_CI_WORKFLOW" ]; }; then - TRUNCATED=true - echo "${CONCEPT}: stopped after ${CI_MAX_PAGES} pages of runs while filtering; raise CODEV_CI_MAX_PAGES for a deeper search" >&2 - fi done +# The loop condition — not a break — is what ends a walk that ran out of +# allowance, so PAGE past the ceiling means there was more to see. Checking it +# here rather than inside the loop also covers the UNFILTERED case: asking for +# 200 runs from a 6922-run repo collects exactly 200 and used to report +# truncated=false, because the old check only fired when a client-side filter +# was active. A capped answer that says it is complete is the failure this +# whole issue is about. +if [ "$PAGE" -gt "$CI_MAX_PAGES" ]; then + TRUNCATED=true + echo "${CONCEPT}: stopped after ${CI_MAX_PAGES} pages of runs; raise CODEV_CI_MAX_PAGES for a deeper search" >&2 +fi + printf '%s' "$ACC" | jq -c --argjson limit "$LIMIT" --argjson truncated "$TRUNCATED" --argjson note "$NOTE" '{ ok: true, provider: "gitea", runs: .[0:$limit], From 8e26a661aaaca07e146d40864cc91abe45ec1f36 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:36:16 -0600 Subject: [PATCH 13/30] [PIR #13] fix(forge): reject a non-numeric run or job id before it reaches a URL Both ids are interpolated into a gitea URL path and into jq --argjson on both sides. A pasted run URL built a path nobody meant; anything non-numeric killed jq under set -e with nothing on stdout. Now it is a named bad-input envelope that also says what to pass instead. Co-Authored-By: Claude Opus 5 --- codev/state/pir-13_thread.md | 49 +++++++++++++++++++ packages/codev/scripts/forge/_ci-lib.sh | 20 ++++++++ .../codev/scripts/forge/gitea/ci-failures.sh | 2 + .../codev/scripts/forge/gitea/ci-run-log.sh | 2 + .../codev/scripts/forge/gitea/ci-run-view.sh | 2 + .../codev/scripts/forge/github/ci-failures.sh | 2 + .../codev/scripts/forge/github/ci-run-log.sh | 2 + .../codev/scripts/forge/github/ci-run-view.sh | 2 + .../src/__tests__/pir-13-ci-concepts.test.ts | 13 +++++ 9 files changed, 94 insertions(+) diff --git a/codev/state/pir-13_thread.md b/codev/state/pir-13_thread.md index 818c34d54..86faaf076 100644 --- a/codev/state/pir-13_thread.md +++ b/codev/state/pir-13_thread.md @@ -33,3 +33,52 @@ Plan at `codev/plans/13-ci-forge-concepts.md`. Two open questions for the archit git.pseudoseed.com is due a Forgejo 16 upgrade. Awaiting `plan-approval`. + +## IMPLEMENT phase + +Plan approved 20:59Z. Six commits on `builder/pir-13`. + +Shape that emerged: one shared extraction ladder (`scripts/forge/_ci-extract.sh`) and one shared +envelope/cache/window lib (`_ci-lib.sh`), with thin per-provider scripts. Both providers now fetch +**the same thing** — one job's log from `actions/jobs/{id}/logs` — so they share a cache and an +extractor. `gh run view --log-failed` is not used at all: it returned 2528 lines / 293 KB with +every line tagged UNKNOWN STEP, and the failing step NAME is available from `--json jobs` anyway. + +Measured result on the live GitHub run: **293 KB of log becomes a 1.2 KB response** carrying the +assertion, the test name, the step name and the line range. Cold 2.5s, cached 1.3s. + +### Things found while building that changed the code + +- **`ci_fail` returning 1 aborted its own caller.** Every concept runs under `set -e`, so a + non-zero return killed the script at that line, before the intended `exit 2` for a bad input. + Every input error was arriving as exit 1. `ci_fail` now reports and returns 0; call sites decide + the status. +- **Two subshell-assignment bugs.** `ci_text_json` and `gitea_ci_jobs` set globals that their + callers read — inside `$( )`, so the values were discarded and jq got empty `--argjson`. Both now + return their data (or write files) instead. +- **Forgejo rejects the spelling its own CLI documents.** `tea actions runs list --help` says + `canceled`; the API answers `{"message":"unknown status: canceled"}` and accepts `cancelled` + (2240 runs). Both providers now get `cancelled`. +- **Truncation was under-reported twice.** Stopping at exactly the limit, and hitting the page + ceiling with no client-side filter, both reported `truncated: false`. Fixed and pinned. +- **A CLI that exits 0 with non-JSON** (an auth prompt) reached jq, died under `set -e`, and left + nothing on stdout. Guarded. +- **The Forgejo-15 task scan needed its own, higher page ceiling.** A page of 50 tasks spans ~6 + runs, so 4 pages reached 24 runs back and reported truncation for anything older. Now 20 pages, + with an early stop the moment the walk passes the run — a recent run costs one page. + +### Verification coverage, stated precisely + +- **GitHub — end to end through the real dispatcher** (config → preset → env → script → JSON parse), + all four concepts, live against `pseudoseed/codev`. Timings in the PR body. +- **Forgejo 15.0.2 — end to end through the real dispatcher**, live against `~/dev/entriq` on the + bare gitea preset. entriq was READ ONLY; its working tree carries an unrelated uncommitted config + edit that predates this session (13:34 MDT). +- **Forgejo 16 — HTTP level only.** The v16 routes were verified with unauthenticated curl against + codeberg.org, and the code path is covered by unit tests with a stubbed `tea` serving the REAL + captured codeberg job log. It has NOT been driven through the dispatcher against a live v16 + server; that needs a Codeberg token, which the architect deferred. + +Question for the gate: nothing in codev can invoke a forge concept from the command line, so a +builder reaches these four by script path. A `codev forge ` entry point would fix that; +it is not in the approved plan, so I am asking rather than adding it. diff --git a/packages/codev/scripts/forge/_ci-lib.sh b/packages/codev/scripts/forge/_ci-lib.sh index f5b113123..ceff8d5f3 100644 --- a/packages/codev/scripts/forge/_ci-lib.sh +++ b/packages/codev/scripts/forge/_ci-lib.sh @@ -125,6 +125,26 @@ ci_status_is_terminal() { esac } +# Require a run/job id to be a plain positive integer. +# +# ci_require_id "" +# +# Both ids are interpolated into a URL path on the gitea side and into a jq +# `--argjson` on both sides, so a non-numeric value either builds a URL nobody +# meant or kills jq under `set -e` with nothing on stdout. Neither is a useful +# answer, and "CODEV_CI_RUN_ID must be numeric" is. +ci_require_id() { + _concept="$1"; _name="$2"; _value="$3" + case "$_value" in + ''|*[!0-9]*) + _msg="${_name} must be a numeric id, got '${_value}' — pass the \`id\` field from ci-runs, not the run \`number\` or a URL" + jq -cn --arg d "$_msg" '{ok: false, error: "bad-input", detail: $d}' + echo "${_concept}: ${_msg}" >&2 + exit 2 + ;; + esac +} + # Assert that a captured payload really is JSON, and emit an envelope if not. # # ci_require_json "" "" diff --git a/packages/codev/scripts/forge/gitea/ci-failures.sh b/packages/codev/scripts/forge/gitea/ci-failures.sh index 93fa3aefe..6f8389aea 100755 --- a/packages/codev/scripts/forge/gitea/ci-failures.sh +++ b/packages/codev/scripts/forge/gitea/ci-failures.sh @@ -27,6 +27,8 @@ if [ -z "$CODEV_CI_RUN_ID" ]; then echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 exit 2 fi +ci_require_id "$CONCEPT" CODEV_CI_RUN_ID "$CODEV_CI_RUN_ID" +[ -z "$CODEV_CI_JOB_ID" ] || ci_require_id "$CONCEPT" CODEV_CI_JOB_ID "$CODEV_CI_JOB_ID" REPO="$(gitea_repo)" || exit 1 TMP=$(mktemp -d) diff --git a/packages/codev/scripts/forge/gitea/ci-run-log.sh b/packages/codev/scripts/forge/gitea/ci-run-log.sh index 6ad082f0c..3735e7af1 100755 --- a/packages/codev/scripts/forge/gitea/ci-run-log.sh +++ b/packages/codev/scripts/forge/gitea/ci-run-log.sh @@ -24,6 +24,8 @@ if [ -z "$CODEV_CI_RUN_ID" ]; then echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 exit 2 fi +ci_require_id "$CONCEPT" CODEV_CI_RUN_ID "$CODEV_CI_RUN_ID" +[ -z "$CODEV_CI_JOB_ID" ] || ci_require_id "$CONCEPT" CODEV_CI_JOB_ID "$CODEV_CI_JOB_ID" # Window first: a malformed request should not cost an API call. ci_window_parse "$CONCEPT" diff --git a/packages/codev/scripts/forge/gitea/ci-run-view.sh b/packages/codev/scripts/forge/gitea/ci-run-view.sh index 625bf7361..ee5739a00 100755 --- a/packages/codev/scripts/forge/gitea/ci-run-view.sh +++ b/packages/codev/scripts/forge/gitea/ci-run-view.sh @@ -29,6 +29,8 @@ if [ -z "$CODEV_CI_RUN_ID" ]; then echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 exit 2 fi +ci_require_id "$CONCEPT" CODEV_CI_RUN_ID "$CODEV_CI_RUN_ID" +[ -z "$CODEV_CI_JOB_ID" ] || ci_require_id "$CONCEPT" CODEV_CI_JOB_ID "$CODEV_CI_JOB_ID" REPO="$(gitea_repo)" || exit 1 TMP=$(mktemp -d) diff --git a/packages/codev/scripts/forge/github/ci-failures.sh b/packages/codev/scripts/forge/github/ci-failures.sh index 050556bb6..7f908099a 100755 --- a/packages/codev/scripts/forge/github/ci-failures.sh +++ b/packages/codev/scripts/forge/github/ci-failures.sh @@ -35,6 +35,8 @@ if [ -z "$CODEV_CI_RUN_ID" ]; then echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 exit 2 fi +ci_require_id "$CONCEPT" CODEV_CI_RUN_ID "$CODEV_CI_RUN_ID" +[ -z "$CODEV_CI_JOB_ID" ] || ci_require_id "$CONCEPT" CODEV_CI_JOB_ID "$CODEV_CI_JOB_ID" rc=0 RUN=$(gh_run_json "$CODEV_CI_RUN_ID") || rc=$? diff --git a/packages/codev/scripts/forge/github/ci-run-log.sh b/packages/codev/scripts/forge/github/ci-run-log.sh index 109ea4b11..d45b594cf 100755 --- a/packages/codev/scripts/forge/github/ci-run-log.sh +++ b/packages/codev/scripts/forge/github/ci-run-log.sh @@ -33,6 +33,8 @@ if [ -z "$CODEV_CI_RUN_ID" ]; then echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 exit 2 fi +ci_require_id "$CONCEPT" CODEV_CI_RUN_ID "$CODEV_CI_RUN_ID" +[ -z "$CODEV_CI_JOB_ID" ] || ci_require_id "$CONCEPT" CODEV_CI_JOB_ID "$CODEV_CI_JOB_ID" # Window first: a malformed request should not cost an API call. ci_window_parse "$CONCEPT" diff --git a/packages/codev/scripts/forge/github/ci-run-view.sh b/packages/codev/scripts/forge/github/ci-run-view.sh index 13eb06e1c..115161b3e 100755 --- a/packages/codev/scripts/forge/github/ci-run-view.sh +++ b/packages/codev/scripts/forge/github/ci-run-view.sh @@ -24,6 +24,8 @@ if [ -z "$CODEV_CI_RUN_ID" ]; then echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 exit 2 fi +ci_require_id "$CONCEPT" CODEV_CI_RUN_ID "$CODEV_CI_RUN_ID" +[ -z "$CODEV_CI_JOB_ID" ] || ci_require_id "$CONCEPT" CODEV_CI_JOB_ID "$CODEV_CI_JOB_ID" rc=0 OUT=$(gh_run_json "$CODEV_CI_RUN_ID") || rc=$? diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts index 0565d4f41..13eeba8cb 100644 --- a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -430,6 +430,19 @@ describe.skipIf(!hasJq())('#13 — ci-failures returns a bounded extract with it expect(r.json!.ok).toBe(false); }); + it.each([ + ['ci-failures.sh', { CODEV_CI_RUN_ID: '../../etc/passwd' }], + ['ci-run-log.sh', { CODEV_CI_RUN_ID: '32515040122', CODEV_CI_JOB_ID: 'x; echo pwned', CODEV_CI_LOG_TAIL: '5' }], + ['ci-run-view.sh', { CODEV_CI_RUN_ID: 'https://github.com/o/r/actions/runs/123' }], + ])('%s rejects a non-numeric id before it reaches a URL or jq', (script, env) => { + stubGh(ONE_FAILING_JOB); + const r = run(githubDir, script, env); + expect(r.status).toBe(2); + expect(r.json!.error).toBe('bad-input'); + expect(r.json!.detail).toContain('must be a numeric id'); + expect(fileLines(path.join(tmp, 'gh.log'))).toEqual([]); + }); + it('emits an envelope when the forge CLI exits 0 with something that is not JSON', () => { // An auth prompt, an empty body, an HTML error page. Without a guard this // reaches jq, dies under `set -e`, and leaves jq's diagnostic on stderr From 067f7b17922089c64648f89aaba69057c7cba120 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 15:58:39 -0600 Subject: [PATCH 14/30] [PIR #13] test(forge): the concept count is 22, not 18 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit forge.test.ts pins the size of KNOWN_CONCEPTS in two places. Four CI concepts were added, so both move — and the four names are asserted individually, so a future addition cannot satisfy the count while dropping one of these. Co-Authored-By: Claude Opus 5 --- packages/codev/src/__tests__/forge.test.ts | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/packages/codev/src/__tests__/forge.test.ts b/packages/codev/src/__tests__/forge.test.ts index 8ee0f2ca8..1250fe596 100644 --- a/packages/codev/src/__tests__/forge.test.ts +++ b/packages/codev/src/__tests__/forge.test.ts @@ -320,7 +320,7 @@ describe('executeForgeCommandSync', () => { // ============================================================================= describe('getKnownConcepts', () => { - it('returns all 18 known concept names', () => { + it('returns all 22 known concept names', () => { const concepts = getKnownConcepts(); expect(concepts).toContain('issue-view'); expect(concepts).toContain('pr-list'); @@ -340,7 +340,11 @@ describe('getKnownConcepts', () => { expect(concepts).toContain('pr-diff'); expect(concepts).toContain('auth-status'); expect(concepts).toContain('repo-archive'); - expect(concepts.length).toBe(18); + expect(concepts).toContain('ci-runs'); + expect(concepts).toContain('ci-run-view'); + expect(concepts).toContain('ci-failures'); + expect(concepts).toContain('ci-run-log'); + expect(concepts.length).toBe(22); }); }); @@ -574,9 +578,9 @@ describe('graceful degradation when command not found', () => { // ============================================================================= describe('resolveAllConcepts', () => { - it('returns all 18 concepts with default source when no config', () => { + it('returns all 22 concepts with default source when no config', () => { const resolutions = resolveAllConcepts(); - expect(resolutions).toHaveLength(18); + expect(resolutions).toHaveLength(22); expect(resolutions.every(r => r.source === 'default')).toBe(true); expect(resolutions.every(r => r.executable !== null)).toBe(true); }); From 9893db9cdf5d9260fc4791f1b334e1daa32a8b0a Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:31:23 -0600 Subject: [PATCH 15/30] [PIR #13] test: give the prompt-surface instrument a ceiling above its own cost MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two tests in this file failed the full-suite run, and a DIFFERENT pair failed the run before it — the signature of a ceiling set below the work, not of a defect. Measured: one measure-prompt-surface.sh invocation costs 25-30s on a quiet machine, and several of these tests invoke it two or three times (two locales, two runs for determinism, a fixture plus the live repo). The 60s inline ceiling sat under the honest cost of the slowest cases, so under load they were killed mid-run and whichever lost the race read as flaky. Raised to 240s via one named constant. The file passes 24/24 in isolation before and after; this is the same coverage, given room. Preferred over .skip, which would have bought a green run by deleting the check. Co-Authored-By: Claude Opus 5 --- .../spec-1280-measurement-instrument.test.ts | 43 ++++++++++++------- 1 file changed, 28 insertions(+), 15 deletions(-) diff --git a/packages/codev/src/__tests__/spec-1280-measurement-instrument.test.ts b/packages/codev/src/__tests__/spec-1280-measurement-instrument.test.ts index eefd3c10f..efdb2b57f 100644 --- a/packages/codev/src/__tests__/spec-1280-measurement-instrument.test.ts +++ b/packages/codev/src/__tests__/spec-1280-measurement-instrument.test.ts @@ -22,6 +22,19 @@ import * as path from 'node:path'; const repoRoot = path.resolve(import.meta.dirname, '../../../..'); const script = path.join(repoRoot, 'scripts/measure-prompt-surface.sh'); +/** + * Per-test ceiling for anything that shells out to the instrument. + * + * Measured on this repo: one `measure-prompt-surface.sh` invocation costs ~25-30s + * on a quiet machine, and several of these tests invoke it two or three times + * (two locales, two runs for determinism, a fixture plus the live repo). The old + * 60s ceiling therefore sat *below* the honest cost of the slowest cases, so + * under full-suite load they were killed mid-run — and a different one lost the + * race each time, which reads as flakiness rather than as a ceiling set too low. + * The whole file passes in isolation once given room. + */ +const INSTRUMENT_TIMEOUT_MS = 240_000; + function run(root: string = repoRoot, env: Record = {}): string { return execFileSync('bash', [script, root], { encoding: 'utf-8', @@ -130,7 +143,7 @@ describe('T1b — per-file four-tier resolution, not directory-level selection', const out = run(dir); // Directory-level selection would have missed the override entirely. expect(out).toMatch(/\| spir \| \d+ \| 8 \| \d+ \|/); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); it('prefers .codev/ over codev/ over codev-skeleton/', () => { const src = fs.readFileSync(script, 'utf-8'); @@ -154,7 +167,7 @@ describe('T2 — phantom-savings proof: includes are expanded', () => { fs.writeFileSync(path.join(dir, 'codev-skeleton/protocols/spir/templates/frag.md'), 'cc dd ee\n'); expect(num(run(dir), 'ALWAYS_ON_WORDS')).toBe(before); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); it('expands non-markdown includes too — protocol.json delivery depends on it (P6)', () => { const dir = makeFixture(); @@ -169,7 +182,7 @@ describe('T2 — phantom-savings proof: includes are expanded', () => { ); // The JSON's words must appear in the served count, not vanish. expect(num(run(dir), 'ALWAYS_ON_WORDS')).toBeGreaterThan(before); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); }); describe('T3 — per-surface reporting completeness (not a ceiling)', () => { @@ -189,11 +202,11 @@ describe('T3 — per-surface reporting completeness (not a ceiling)', () => { new RegExp(`^\\| ${name} \\|`, 'm'), ); } - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); it('includes codev-only protocols with no skeleton twin (release)', () => { expect(run()).toMatch(/^\| release \|/m); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); }); describe('T11 — buckets vs audience loads are reported on different bases', () => { @@ -202,7 +215,7 @@ describe('T11 — buckets vs audience loads are reported on different bases', () expect(out).toContain('ALWAYS_ON(builder,p,I) = SHARED + BUILDER_SPAWN[p]'); expect(out).toMatch(/OVERLAP by design/); expect(out).toMatch(/these SUM/); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); it('one bucket growing while another shrinks shows BOTH movements, not a netted zero', () => { const dir = makeFixture(); @@ -219,7 +232,7 @@ describe('T11 — buckets vs audience loads are reported on different bases', () expect(sharedAfter).toBeLessThan(sharedBefore); expect(archAfter).toBeGreaterThan(archBefore); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); }); describe('T15 — relocation is visible, never reported as deletion (M0c)', () => { @@ -236,7 +249,7 @@ describe('T15 — relocation is visible, never reported as deletion (M0c)', () = const after = run(dir); expect(num(after, 'ALWAYS_ON_WORDS')).toBeLessThan(num(before, 'ALWAYS_ON_WORDS')); expect(num(after, 'TOTAL_AUTHORED_WORDS')).toBe(num(before, 'TOTAL_AUTHORED_WORDS')); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); it('counts all four skill trees — one-tree counting would report relocation as deletion', () => { const src = fs.readFileSync(script, 'utf-8'); @@ -271,13 +284,13 @@ describe('portability — the count must not depend on the host', () => { const utf8 = num(run(repoRoot, { LC_ALL: 'en_US.UTF-8' }), 'ALWAYS_ON_WORDS'); const c = num(run(repoRoot, { LC_ALL: 'C' }), 'ALWAYS_ON_WORDS'); expect(c).toBe(utf8); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); }); describe('T12 — determinism', () => { it('emits byte-identical output twice at the same commit', () => { expect(run()).toBe(run()); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); }); describe('instrument correctness — asserted without pinning the live surface', () => { @@ -304,7 +317,7 @@ describe('instrument correctness — asserted without pinning the live surface', // human reads them as findings rather than maintaining them as expectations. let out: string; - beforeAll(() => { out = run(); }, 60_000); + beforeAll(() => { out = run(); }, INSTRUMENT_TIMEOUT_MS); describe('layer 1 — invariants over the live repo (hold at any surface size)', () => { it('ALWAYS_ON = SHARED + BUILDER_SPAWN[spir] + I x (HOT + PHASE mean[spir])', () => { @@ -330,7 +343,7 @@ describe('instrument correctness — asserted without pinning the live surface', const spir = out.match(/^\| spir \| \d+ \| (\d+) \| \d+ \|$/m)!; const hot = Number(out.match(/lessons-critical\(\d+\) = (\d+)/)![1]); expect(two - one).toBe(hot + Number(spir[1])); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); }); describe('layer 2 — absolute values on a fixture whose arithmetic a human can check', () => { @@ -343,13 +356,13 @@ describe('instrument correctness — asserted without pinning the live surface', it('reports the hand-computed total for a known surface', () => { const out2 = run(makeFixture()); expect(num(out2, 'ALWAYS_ON_WORDS')).toBe(107); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); it('reports the hand-computed architect and consultant loads', () => { const out2 = run(makeFixture()); expect(out2).toMatch(/\| Architect \(per session\) \| 10 \|/); expect(out2).toMatch(/\| Consultant \(per review, spir\) \| 4 \|/); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); it('a component silently omitted would fail here even though invariants still hold', () => { // The blind spot invariants cannot see: drop the hot tier from SHARED and every @@ -357,7 +370,7 @@ describe('instrument correctness — asserted without pinning the live surface', // Only an externally-known expected value catches it. const out2 = run(makeFixture()); expect(Number(out2.match(/\| SHARED [^|]*\| (\d+) \|/)![1])).toBe(7); - }, 60_000); + }, INSTRUMENT_TIMEOUT_MS); }); describe('layer 3 — the frozen baseline artifacts are historical records', () => { From 80c29b307e962ebb56b9d9239201737d5b051e4d Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:38:21 -0600 Subject: [PATCH 16/30] chore(porch): 13 dev-approval gate-requested --- codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 38ca610b9..7508d17e4 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -11,10 +11,11 @@ gates: approved_at: '2026-08-21T20:58:47.286Z' dev-approval: status: pending + requested_at: '2026-08-21T22:38:21.754Z' pr: status: pending iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T20:59:08.728Z' +updated_at: '2026-08-21T22:38:21.756Z' From 6b4f8a2ab0fd0c89932c406ead511b4d98bd49b7 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:39:29 -0600 Subject: [PATCH 17/30] chore(porch): 13 dev-approval gate-approved --- codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 7508d17e4..4ac1b1b9f 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -10,12 +10,13 @@ gates: requested_at: '2026-08-21T20:58:03.660Z' approved_at: '2026-08-21T20:58:47.286Z' dev-approval: - status: pending + status: approved requested_at: '2026-08-21T22:38:21.754Z' + approved_at: '2026-08-21T22:39:29.289Z' pr: status: pending iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T22:38:21.756Z' +updated_at: '2026-08-21T22:39:29.291Z' From 3334bf9e89fcf24a3db3948648bbf7bedd7d8c82 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:43:07 -0600 Subject: [PATCH 18/30] =?UTF-8?q?[PIR=20#13]=20feat(cli):=20codev=20forge?= =?UTF-8?q?=20=20=E2=80=94=20run=20a=20concept=20through=20the=20?= =?UTF-8?q?real=20resolver?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Naming a concept script by path BYPASSES resolution: it skips the .codev/config.json lookup, the provider preset, and any per-repo override. A project that overrides ci-failures would have its override silently ignored and would get GitHub's script against its own forge. The reference Forgejo repo carried three such overrides until #12 shipped, so this is not hypothetical — and the test instructions in this PR's own dev-approval summary told people to do exactly that. Deliberately thin: it delegates to executeForgeCommandDetailed, so there is one place where a forge command is actually run. stdout verbatim (envelope included on the failure path, which is the whole reason the ci-* scripts print it there), and the script's own exit code. Two additions, both about a caller being wrong rather than a forge being wrong: exit 2 for an unknown concept name, listing the valid ones; exit 3 for a concept disabled for this provider, NAMED — team-activity on gitea says 'not available for provider "gitea"' rather than printing nothing. Both SKILL.md twins now document it and say plainly not to call a script by path. Co-Authored-By: Claude Opus 5 --- .claude/skills/forge/SKILL.md | 18 ++++ .codex/skills/forge/SKILL.md | 18 ++++ .../src/__tests__/pir-13-ci-concepts.test.ts | 89 +++++++++++++++++ packages/codev/src/cli.ts | 19 ++++ packages/codev/src/commands/forge.ts | 97 +++++++++++++++++++ 5 files changed, 241 insertions(+) create mode 100644 packages/codev/src/commands/forge.ts diff --git a/.claude/skills/forge/SKILL.md b/.claude/skills/forge/SKILL.md index 603ff988f..43b3ec82d 100644 --- a/.claude/skills/forge/SKILL.md +++ b/.claude/skills/forge/SKILL.md @@ -27,6 +27,24 @@ Forge concept commands decouple codev from direct `gh` CLI calls. Each GitHub op | `ci-failures` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (optional) | The failing job's assertion, extracted and capped | | `ci-run-log` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (opt), and exactly one of `CODEV_CI_LOG_TAIL` / `CODEV_CI_LOG_HEAD` / `CODEV_CI_LOG_GREP` | A raw log window | +## Running a concept + +```bash +codev forge # CODEV_* environment is passed through +CODEV_CI_RUN_ID=32515040122 codev forge ci-failures | jq +``` + +**Never call a concept script by its path.** `packages/codev/scripts/forge/github/ci-failures.sh` +bypasses resolution — it skips the `.codev/config.json` lookup, the provider +preset, and any per-repo override — so a project that overrides that concept +gets GitHub's script against its own forge and never learns why. + +`codev forge` is a thin dispatcher over the same resolver every other caller +uses: it prints the script's stdout verbatim (envelope included on the failure +path) and exits with the script's own exit code. Its own exit codes are `2` for +an unknown concept name (it lists the valid ones) and `3` for a concept +disabled for this provider, which it names rather than printing nothing. + ## CI concepts Four concepts, **tiered so the cheap question stays cheap**. A builder asks about diff --git a/.codex/skills/forge/SKILL.md b/.codex/skills/forge/SKILL.md index 603ff988f..43b3ec82d 100644 --- a/.codex/skills/forge/SKILL.md +++ b/.codex/skills/forge/SKILL.md @@ -27,6 +27,24 @@ Forge concept commands decouple codev from direct `gh` CLI calls. Each GitHub op | `ci-failures` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (optional) | The failing job's assertion, extracted and capped | | `ci-run-log` | `CODEV_CI_RUN_ID`, `CODEV_CI_JOB_ID` (opt), and exactly one of `CODEV_CI_LOG_TAIL` / `CODEV_CI_LOG_HEAD` / `CODEV_CI_LOG_GREP` | A raw log window | +## Running a concept + +```bash +codev forge # CODEV_* environment is passed through +CODEV_CI_RUN_ID=32515040122 codev forge ci-failures | jq +``` + +**Never call a concept script by its path.** `packages/codev/scripts/forge/github/ci-failures.sh` +bypasses resolution — it skips the `.codev/config.json` lookup, the provider +preset, and any per-repo override — so a project that overrides that concept +gets GitHub's script against its own forge and never learns why. + +`codev forge` is a thin dispatcher over the same resolver every other caller +uses: it prints the script's stdout verbatim (envelope included on the failure +path) and exits with the script's own exit code. Its own exit codes are `2` for +an unknown concept name (it lists the valid ones) and `3` for a concept +disabled for this provider, which it names rather than printing nothing. + ## CI concepts Four concepts, **tiered so the cheap question stays cheap**. A builder asks about diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts index 13eeba8cb..6e0f8081d 100644 --- a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -48,6 +48,7 @@ import * as os from 'node:os'; import * as path from 'node:path'; import * as zlib from 'node:zlib'; import { getForgeCommand, resolveAllConcepts, executeForgeCommandDetailed } from '../lib/forge.js'; +import { runForgeConcept } from '../commands/forge.js'; const codevPkgRoot = path.resolve(import.meta.dirname, '..', '..'); const forgeScripts = path.join(codevPkgRoot, 'scripts', 'forge'); @@ -885,3 +886,91 @@ describe.skipIf(!hasJq())('#13 — Forgejo, as it actually behaves', () => { expect(fileLines(path.join(tmp, 'tea.log'))).toEqual([]); }); }); + +// --------------------------------------------------------------------------- +// codev forge +// --------------------------------------------------------------------------- + +describe.skipIf(!hasJq())('#13 — codev forge runs a concept through the real resolver', () => { + /** A workspace with a .codev/config.json, so resolution has something to resolve. */ + function workspace(forge: Record): string { + const root = path.join(tmp, 'ws'); + fs.mkdirSync(path.join(root, '.codev'), { recursive: true }); + fs.writeFileSync(path.join(root, '.codev', 'config.json'), JSON.stringify({ forge })); + return root; + } + + function capture() { + const out: string[] = []; + const err: string[] = []; + return { out, err, stdout: (t: string) => out.push(t), stderr: (t: string) => err.push(t) }; + } + + it('HONOURS a per-repo override — the reason this command exists', () => { + // Naming the script by path bypasses the config lookup, the provider preset + // and any override. A project that overrides ci-failures would have its + // override silently ignored, and would get GitHub's script against a + // Forgejo repo. The reference Forgejo repo carried three such overrides + // until #12 shipped, so this is not hypothetical. + const custom = path.join(tmp, 'my-ci-failures.sh'); + fs.writeFileSync(custom, '#!/bin/sh\necho \'{"ok":true,"mine":true}\'\n', { mode: 0o755 }); + fs.chmodSync(custom, 0o755); + + const io = capture(); + return runForgeConcept('ci-failures', { cwd: workspace({ 'ci-failures': custom }), ...io }).then((code) => { + expect(code).toBe(0); + expect(JSON.parse(io.out.join(''))).toEqual({ ok: true, mine: true }); + }); + }); + + it('names a concept disabled for the provider and exits non-zero', async () => { + const io = capture(); + const code = await runForgeConcept('team-activity', { cwd: workspace({ provider: 'gitea' }), ...io }); + expect(code).not.toBe(0); + expect(io.out.join('')).toBe(''); + expect(io.err.join('')).toContain('team-activity'); + expect(io.err.join(''), 'a disabled concept must name WHY it is unavailable').toContain('gitea'); + }); + + it('lists the valid concepts when given an unknown name', async () => { + const io = capture(); + const code = await runForgeConcept('ci-failure', { cwd: workspace({}), ...io }); + expect(code).toBe(2); + expect(io.err.join('')).toContain('not a known forge concept'); + expect(io.err.join('')).toContain('ci-failures'); + expect(io.err.join('')).toContain('pr-exists'); + }); + + it('prints stdout verbatim and propagates the exit code, envelope included', async () => { + // A concept that fails still has something to say — the ci-* scripts print + // their error envelope on stdout precisely so the class of failure survives + // a non-zero exit. Swallowing stdout on failure would throw it away at the + // last step. + const failing = path.join(tmp, 'failing.sh'); + fs.writeFileSync(failing, '#!/bin/sh\necho \'{"ok":false,"error":"timeout","seconds":60}\'\nexit 1\n', { mode: 0o755 }); + fs.chmodSync(failing, 0o755); + + const io = capture(); + const code = await runForgeConcept('ci-runs', { cwd: workspace({ 'ci-runs': failing }), ...io }); + expect(code).toBe(1); + expect(JSON.parse(io.out.join('')).error).toBe('timeout'); + }); + + it('passes the ambient CODEV_* environment through to the script', async () => { + const echoer = path.join(tmp, 'echo-env.sh'); + fs.writeFileSync(echoer, '#!/bin/sh\nprintf \'{"runId":"%s"}\' "$CODEV_CI_RUN_ID"\n', { mode: 0o755 }); + fs.chmodSync(echoer, 0o755); + + const previous = process.env.CODEV_CI_RUN_ID; + process.env.CODEV_CI_RUN_ID = '32515040122'; + try { + const io = capture(); + const code = await runForgeConcept('ci-failures', { cwd: workspace({ 'ci-failures': echoer }), ...io }); + expect(code).toBe(0); + expect(JSON.parse(io.out.join('')).runId).toBe('32515040122'); + } finally { + if (previous === undefined) delete process.env.CODEV_CI_RUN_ID; + else process.env.CODEV_CI_RUN_ID = previous; + } + }); +}); diff --git a/packages/codev/src/cli.ts b/packages/codev/src/cli.ts index 8d065a2b5..d22dbe7b3 100644 --- a/packages/codev/src/cli.ts +++ b/packages/codev/src/cli.ts @@ -8,6 +8,7 @@ import { existsSync } from 'node:fs'; import { join } from 'node:path'; import { Command } from 'commander'; import { doctor } from './commands/doctor.js'; +import { runForgeConcept } from './commands/forge.js'; import { init } from './commands/init.js'; import { adopt } from './commands/adopt.js'; import { update } from './commands/update.js'; @@ -61,6 +62,24 @@ program } }); +// Forge command +// +// Deliberately thin. Its whole reason for existing is that naming a concept +// script by path bypasses resolution — the config lookup, the provider preset +// and any per-repo override — so a project that overrides a concept would have +// its override silently ignored. See commands/forge.ts. +program + .command('forge ') + .description('Run one forge concept through the configured provider (CODEV_* env is passed through)') + .action(async (concept: string) => { + try { + process.exit(await runForgeConcept(concept)); + } catch (error) { + console.error(error instanceof Error ? error.message : String(error)); + process.exit(1); + } + }); + // Init command program .command('init [project-name]') diff --git a/packages/codev/src/commands/forge.ts b/packages/codev/src/commands/forge.ts new file mode 100644 index 000000000..1bb9df9de --- /dev/null +++ b/packages/codev/src/commands/forge.ts @@ -0,0 +1,97 @@ +/** + * `codev forge ` — run one forge concept through the real resolver. + * + * WHY THIS EXISTS (issue #13) + * + * Before this, the only way to invoke a concept from a shell was to name its + * script by path — `packages/codev/scripts/forge/github/ci-failures.sh`. That + * path **bypasses resolution**: it skips the `.codev/config.json` lookup, the + * provider preset, and any per-repo override. A project that overrides + * `ci-failures` would have its override silently ignored by anyone following + * those instructions, and would get GitHub's script against a Forgejo repo. + * That is not hypothetical — the reference Forgejo repo carried three concept + * overrides until #12 shipped. + * + * So this is deliberately a THIN dispatcher and nothing else: resolve the + * concept exactly as `executeForgeCommand` does, pass the ambient `CODEV_*` + * environment through, print stdout verbatim, and exit with the script's own + * exit code. No parsing, no reformatting, no second code path — it delegates to + * `executeForgeCommandDetailed` so there is exactly one place where a forge + * command is actually run. + * + * The two things it adds are the two ways a caller can be wrong: + * + * - a concept **disabled for this provider** says so by name and exits + * non-zero, rather than printing nothing and letting silence read as an + * empty answer; + * - an **unknown concept name** lists the valid ones. + */ + +import { + executeForgeCommandDetailed, + getKnownConcepts, + getForgeCommand, + loadForgeConfig, + describeUnavailableConcept, +} from '../lib/forge.js'; + +export interface ForgeCommandCliOptions { + /** Workspace root to resolve `.codev/config.json` from. Defaults to cwd. */ + cwd?: string; + /** Sinks, injectable for tests. Default to the real streams. */ + stdout?: (text: string) => void; + stderr?: (text: string) => void; +} + +/** + * Run one concept. Returns the process exit code; never calls process.exit, so + * the caller (cli.ts) decides and tests can assert. + */ +export async function runForgeConcept( + concept: string | undefined, + options: ForgeCommandCliOptions = {}, +): Promise { + const cwd = options.cwd ?? process.cwd(); + const out = options.stdout ?? ((t: string) => process.stdout.write(t)); + const err = options.stderr ?? ((t: string) => process.stderr.write(t)); + + const known = getKnownConcepts(); + + if (!concept) { + err(`codev forge: a concept name is required.\nKnown concepts: ${known.join(', ')}\n`); + return 2; + } + + if (!known.includes(concept)) { + err(`codev forge: '${concept}' is not a known forge concept.\nKnown concepts: ${known.join(', ')}\n`); + return 2; + } + + const forgeConfig = loadForgeConfig(cwd); + + // Disabled is not an error in the command; it is an answer the caller must be + // able to see. `describeUnavailableConcept` names the provider or the config, + // which is the difference between "why is this empty" and "right, Forgejo has + // no GraphQL". + if (getForgeCommand(concept, forgeConfig) === null) { + err(`codev forge: ${describeUnavailableConcept(concept, forgeConfig)}\n`); + return 3; + } + + const result = await executeForgeCommandDetailed(concept, undefined, { cwd, forgeConfig }); + + // stdout verbatim, including on the failure path: the ci-* concepts print + // their structured error envelope there precisely so the class of failure + // survives a non-zero exit. + if (result.stdout) out(result.stdout.endsWith('\n') ? result.stdout : `${result.stdout}\n`); + if (result.stderr) err(result.stderr.endsWith('\n') ? result.stderr : `${result.stderr}\n`); + + if (result.timedOut) { + err(`codev forge: '${concept}' was killed for exceeding the command timeout\n`); + // 124 is what timeout(1) uses, and what scripts/forge/_timeout.sh reports + // internally — so a caller sees the same number wherever the clock ran out. + return 124; + } + + return result.exitCode ?? (result.ok ? 0 : 1); +} From 8ba2f26b3f11ffb02827b934cd3b246c46f6e800 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:43:33 -0600 Subject: [PATCH 19/30] chore(porch): 13 review phase-transition --- codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 4ac1b1b9f..0b19f660c 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -1,7 +1,7 @@ id: '13' title: add-ci-concepts-to-the-forge-l protocol: pir -phase: implement +phase: review plan_phases: [] current_plan_phase: null gates: @@ -19,4 +19,4 @@ iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T22:39:29.291Z' +updated_at: '2026-08-21T22:43:32.878Z' From 5f57f107e7e05453c4e9b13a59434e4b4eab27e2 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:49:52 -0600 Subject: [PATCH 20/30] [PIR #13] Review + retrospective Corrects the record on gh run view --log-failed, states the three verification tiers separately (GitHub live end to end, Forgejo 15 live end to end, Forgejo 16 HTTP level only), and routes the measured forge facts to arch.md. The hot-tier lesson on truncation is rewritten rather than displaced: the same rule arrived through a wider door this time (an unsupported server, not just a trimmed result), so it now covers every way 'I could not tell' can be spelled the same way as 'no'. The file stays at ten entries. Co-Authored-By: Claude Opus 5 --- codev/resources/arch.md | 23 ++- codev/resources/lessons-critical.md | 2 +- codev/resources/lessons-learned.md | 7 + codev/reviews/13-ci-forge-concepts.md | 240 ++++++++++++++++++++++++++ 4 files changed, 270 insertions(+), 2 deletions(-) create mode 100644 codev/reviews/13-ci-forge-concepts.md diff --git a/codev/resources/arch.md b/codev/resources/arch.md index 5a54d570e..6291bbb8b 100644 --- a/codev/resources/arch.md +++ b/codev/resources/arch.md @@ -2086,7 +2086,7 @@ All interactions with the repository hosting platform (GitHub by default) are ro **Configuration**: `.codev/config.json` `forge` section maps concept names to shell commands. Set to `null` to disable a concept. Omit to use the default (`gh`-based) command. -**18 concepts**: `issue-view`, `pr-list`, `issue-list`, `issue-search`, `issue-comment`, `pr-exists`, `recently-closed`, `recently-merged`, `user-identity`, `team-activity`, `on-it-timestamps`, `pr-create`, `pr-merge`, `pr-search`, `pr-view`, `pr-diff`, `auth-status`, `repo-archive`. +**22 concepts**: `issue-view`, `pr-list`, `issue-list`, `issue-search`, `issue-comment`, `pr-exists`, `recently-closed`, `recently-merged`, `user-identity`, `team-activity`, `on-it-timestamps`, `pr-create`, `pr-merge`, `pr-search`, `pr-view`, `pr-diff`, `auth-status`, `repo-archive`, `ci-runs`, `ci-run-view`, `ci-failures`, `ci-run-log`. **Environment variables**: Each concept receives `CODEV_*` env vars (e.g., `CODEV_ISSUE_NUMBER`, `CODEV_PR_NUMBER`) that the command uses to parameterize its output. @@ -2101,6 +2101,27 @@ All interactions with the repository hosting platform (GitHub by default) are ro - Gitea's `/repos/{o}/{r}/pulls` list is priced **per returned PR object**, not per request: 0.78s at `limit=1`, 32.8s at `limit=50`. Paging it is linear in total PRs regardless of page size, so on a 1599-PR repo a full walk costs ~17 minutes. Anything answerable by a targeted endpoint must not use it. The cheap index for the same rows is `/issues?type=pulls` (~1.8s per 50), which carries `pull_request.merged_at` but no head/base refs. - `head.ref` is rewritten to `refs/pull/N/head` once a merged PR's source branch is deleted — the normal state of every merged PR — but **`head.label` retains the original branch name**, and `GET /pulls/{base}/{head}` matches on the stored head branch. Branch→PR lookup is therefore possible after branch deletion, which the earlier `head.ref` scan had documented as impossible. +**Invoking a concept**: `codev forge ` (`src/commands/forge.ts`), a thin wrapper over `executeForgeCommandDetailed` that passes the ambient `CODEV_*` through, prints stdout verbatim and exits with the script's own code. **Naming a concept script by path bypasses resolution** — the config lookup, the provider preset and any per-repo override — so a project that overrides a concept gets the github default against its own forge. Its own exit codes: `2` unknown concept (lists the valid ones), `3` disabled for this provider (named). + +**`executeForgeCommandDetailed(concept, env, options)`** exists because `executeForgeCommand` collapses every failure mode to `null`: a timeout, a non-zero exit, unparseable output and a disabled concept arrive identically. It returns `{ok, data, stdout, stderr, exitCode, timedOut, unavailable, durationMs}` and **keeps stdout on the failure path**. Use it whenever "could not answer" and "answered no" must not be the same value. + +**CI concepts (PIR #13)** — `ci-runs`, `ci-run-view`, `ci-failures`, `ci-run-log`, tiered so the cheap question stays cheap: only the last two ever read log bytes. Their contract adds two rules to the exit-status one above: + +- **Errors are values.** Every ci-* concept prints one JSON object on stdout on success *and* failure, so the class of failure (`timeout` | `not-found` | `unsupported-server` | `forge-error` | `bad-input`) survives a non-zero exit. +- **Anything carrying log text also carries `logLines`, `returnedLines`, `truncated`.** When extraction recognises nothing it returns `extracted: false` with the job identity and a ready-to-run `next`, and **no log lines at all** — never a fallback slice, which a reader treats as a diagnosis. + +Shared implementation, so the two providers cannot drift: `scripts/forge/_ci-extract.sh` (the extraction ladder — ANSI stripping, then vitest/jest → go test → tsc → the runner's `##[error]` marker → a line-**anchored** first error → refusal), `_ci-lib.sh` (envelope, caps, `$TMPDIR` log cache keyed by job id and written only for terminal jobs, window parsing), `_timeout.sh` (#12's watchdog, now used by both providers). + +**CI behavior verified live against GitHub (gh 2.87.0), Forgejo 15.0.2 and Forgejo 16.0.0-dev** (PIR #13). Each of these looks like something else and is not: + +- **`gh run view --log-failed` does not narrow to the failing step.** It selects the failing JOB and returns all of it — 2528 lines / 293 KB on the reference run, with every line tagged `UNKNOWN STEP` because gh's filename-to-step mapping had missed. Both providers therefore fetch `actions/jobs/{id}/logs` and codev extracts; the failing step NAME comes from `gh run view --json jobs`. +- **Forgejo has no Actions job-log API before 16.0** (released 2026-07-16). On 15.x there is no token-reachable log by any route: `tea actions runs view` / `runs logs` both 404, and the web UI's log route is session-only, rejecting an API token and basic auth alike. `ci-failures` / `ci-run-log` return `unsupported-server` there naming both versions; `ci-runs` / `ci-run-view` keep working. +- **Forgejo ignores `limit` unless `page` is also sent** — `actions/runs?limit=3` returned all 6922 runs. `status=` filters server-side; **`branch=` and `event=` are silently ignored**. +- **A `pull_request` run records `head_branch` as `#`**, not a branch name, so branch filtering resolves the branch to its PR first (the #12 base/head lookup) and filters client-side. +- **Run `id` and `index_in_repo` are two id spaces and both resolve on `/actions/runs/{x}`**, to different real runs. Concepts take `id` only and refuse a non-numeric value rather than guess. +- **Forgejo rejects `status=canceled`** — the spelling its own `tea` CLI documents — and accepts `cancelled`; GitHub wants `cancelled` too. +- `actions/runs` costs ~17.8 KB **per run** (an embedded repository object) at ~0.3s per page; `actions/tasks` is 482 B per job and is the cheaper index, but its ids are TASK ids, which the log API does not accept. + ### Two remote-command paths into an editor surface (Spec 1401) Tower has **two** ways for an external controller to drive an editor, and picking the wrong one diff --git a/codev/resources/lessons-critical.md b/codev/resources/lessons-critical.md index cdad4cac1..0f4e3a8c8 100644 --- a/codev/resources/lessons-critical.md +++ b/codev/resources/lessons-critical.md @@ -9,7 +9,7 @@ MAINTAIN polices the cap and keeps the map in sync with lessons-learned.md's sec - Trust the protocol — never skip CMAP/consultation; it catches security, design, and protocol issues solo review misses. - Check for existing work (PRs, git history) before building from scratch. - "It compiled" / "tests pass" is not "it works" — verify the real user path end-to-end before calling it done. -- A truncated result is indistinguishable from a complete one once emitted — give "I stopped early" its own signal and emit nothing, never a partial answer that reads as whole. +- "I could not tell" must never be spelled the same way as "no". A truncation, an unreachable API, and a server too old to answer each need their own signal and must emit nothing else — a partial or empty answer reads as a complete, negative one. - Single source of truth beats distributed state — consolidate duplicates rather than syncing them. - After any rename or framework change, grep the whole repo across BOTH codev/ and codev-skeleton/ before claiming "all fixed." - When stuck (2 failed hypotheses or ~30 min), get an outside model's perspective instead of guessing. diff --git a/codev/resources/lessons-learned.md b/codev/resources/lessons-learned.md index 263584b1e..ee8e83785 100644 --- a/codev/resources/lessons-learned.md +++ b/codev/resources/lessons-learned.md @@ -304,6 +304,10 @@ just its inputs. - [From 0755] Vestigial production code can survive for unknown durations. `setArchitect` was orphaned (only called from tests) for an unknown period; the local `architect` table it wrote to was effectively dead state. When a feature touches a long-lived API, run a "who calls this in production?" grep during planning, not after the implementation has diverged. Reviewers caught it in iter-1; planning would have caught it earlier. - [From 0755] When a plan references specific migration version numbers, verify against the current schema before commit -- or reference migrations by purpose ("the next available after issue_number widening") rather than fixed numbers. The plan said v5 local + v5 global; the actual code needed v9 + v13 because the project had already advanced past those. - [From 1313] Trace a contract change end-to-end before calling it specified. A send-outcome change (`delivered` vs `held`+reason) was specified server-side but not client-side (`packages/core/src/tower-client.ts` + `commands/send.ts`, on BOTH the single-send and `--all` paths), and drew repeat REQUEST_CHANGES across the plan and Phase 4. Name every layer the contract crosses (wire → client → each CLI path) in the plan deliverable so the client surfacing isn't discovered at review time. +- [From #13] **When an issue tells you a tool already does the work, measure it before building on that.** #13 asserted that `gh run view --log-failed` extracts the failing step and told the implementer not to duplicate it. It does not: it selects the failing JOB and returned 2528 lines / 293 KB with every line tagged `UNKNOWN STEP`. Two people had read that output that day without registering what it meant. The whole design rests on that premise, so a two-minute check was worth more than the paragraph asserting it. +- [From #13] **Extraction must recognise, not truncate — and it must strip ANSI before it can recognise anything.** The payload line in a real runner log is `ESC[41mESC[1m FAIL ESC[22m…`, so a matcher that skips cleaning reports "no recognized failure" on a log that plainly contains one. And the naive alternatives both pick the wrong line on the same log: the first "Error:" is a *passing* test's fixture string 1214 lines above the real failure, and the last 25 lines of the Forgejo log are git credential cleanup. Anchor error patterns at the START of the line — that alone kills the observed false positive, and it holds in logs with no test summary to measure against. +- [From #13] **A CLI's `--help` is not the API's vocabulary.** `tea actions runs list --help` documents `canceled`; the Forgejo API answers that with `{"message":"unknown status: canceled"}` and accepts `cancelled` (2240 runs). A filter spelled from the help text returns an empty list that nobody questions. +- [From #13] **Ask which id space an endpoint means.** Forgejo run `id` and `index_in_repo` BOTH resolve on `/actions/runs/{x}`, to different real runs, and the web URL shows the second. Its jobs list carries `id` and `task_id` and the log API accepts only the first. Where two plausible ids resolve on the same route, a concept must take one by name and refuse the other rather than guess. ## Testing @@ -457,6 +461,9 @@ Measure accumulated CPU time, or just let it finish, before concluding a process a fresh database. Forget the bump and a fresh install records the older version and re-runs the new block on its next open. Nothing breaks loudly — the columns arrive from the schema anyway — so it survives review. Pin the constant to the highest migration block in a test. +- [From #13] **Under `set -e`, a helper that returns non-zero decides its caller's exit status.** `ci_fail` printed an error envelope and returned 1; every call site then had `exit 2` for a bad input on the next line, and none of them ever ran — `set -e` killed the script at the helper. Reporting and deciding the exit status are separate jobs; a reporting helper should return 0. +- [From #13] **A shell helper that "sets a global" sets nothing when the caller captures it.** Two helpers assigned `CI_TRUNCATED` / `GITEA_JOB_SOURCE` for their callers to read, and every caller ran them inside `$( )` — a subshell, so the values were discarded and jq got an empty `--argjson`. Return the data, or write it to a file the caller reads. +- [From #13] **A test whose ceiling sits below its own cost reads as flaky.** `spec-1280-measurement-instrument.test.ts` capped tests at 60s that cost 80-100s (the instrument is ~25-30s per invocation and several tests invoke it two or three times). A *different* pair failed each full-suite run, which looks like flakiness and invites `.skip`. Measure the work, then set the ceiling above it — that is fixing the bug rather than hiding it. ## UI/UX diff --git a/codev/reviews/13-ci-forge-concepts.md b/codev/reviews/13-ci-forge-concepts.md new file mode 100644 index 000000000..2d16469f5 --- /dev/null +++ b/codev/reviews/13-ci-forge-concepts.md @@ -0,0 +1,240 @@ +# PIR Review: CI concepts for the forge layer + +Fixes #13 + +## Summary + +Adds four CI concepts to the forge layer — `ci-runs`, `ci-run-view`, `ci-failures`, `ci-run-log` — for both `github` and `gitea`, tiered so that only the last two ever read log bytes. A builder asking why CI failed now gets the failing job, the failing step and the assertion instead of a log: on the reference run, **293 KB becomes a 1.2 KB response**. Also adds `codev forge `, because naming a concept script by path bypasses resolution and would silently ignore a repo's own overrides. + +## The record this PR corrects + +**Issue #13 says `gh run view --log-failed` "already returns only failed steps" and instructs the implementer not to re-derive that. It does not, and the instruction is wrong.** Measured on run `32515040122` of this repository: + +``` +gh run view 32515040122 --log-failed → 2528 lines, 293 KB, 2.0 s +Unit Tests UNKNOWN STEP 2026-08-21T18:47:09.5820646Z Current runner version: '2.336.0' +``` + +Every one of those 2528 lines is tagged `UNKNOWN STEP`. `--log-failed` selects the failing **job** and returns all of it; `gh` maps log files to steps by name and falls back to `UNKNOWN STEP` when that mapping misses. The architect independently reproduced this on run `32448538074`: 919 lines, all 919 tagged `UNKNOWN STEP` — and had read that same output earlier the same day while diagnosing #6 without registering what it meant. + +So codev extracts on **both** providers, and neither uses `--log-failed`. Both fetch `actions/jobs/{id}/logs` — one job, no invented step column, the same shape Forgejo 16 serves — which is also why they share one cache and one extractor. The failing step *name* comes from `gh run view --json jobs`, which is structured and reliable. + +Anyone reading #13 later should read this section instead of its "Provider notes". + +## What the two forges actually do + +Everything below was measured against live instances on 2026-08-21, not reasoned about. + +**Forgejo has no Actions job-log API before 16.0** (released 2026-07-16). `git.pseudoseed.com` reports `15.0.2+gitea-1.22.0`, and there `tea actions runs view` and `tea actions runs logs` both 404 — they call `/actions/runs/{id}/jobs` and `/actions/jobs/{id}/logs`, neither of which exists. Every alternative route was probed. The web UI's own log route exists but is session-only: it rejects `Authorization: token` and HTTP basic auth alike, while the API accepts the same token (verified as a control). There is no token-reachable log on 15.x by any path. + +`tea actions runs list --output json` is also lossy where it does work — `workflow`, `branch`, `started` and `duration` all come back as empty strings — so these scripts go through `tea api`, as #12 established. + +Four query-parameter facts, all footguns: + +| | | +|---|---| +| `limit` is ignored unless `page` is also sent | `actions/runs?limit=3` returned **all 6922 runs** | +| `status=` filters server-side | works, and is used | +| `branch=` and `event=` are silently ignored | branch filtering is client-side | +| `status=canceled` is rejected | `{"message":"unknown status: canceled"}`; `cancelled` returns 2240 runs — the opposite of what `tea`'s own `--help` documents | + +And two shape facts: + +- **A `pull_request` run records `head_branch` as `#3847`** — the PR number, not a branch. In the first 100 tasks on the reference repo: `#3869` ×32, `#3865` ×10, `main` ×7, `v1.0.230` ×1. So `CODEV_BRANCH_NAME=builder/x` matches *nothing* on a repo that runs CI on pull requests unless the branch is resolved to its PR first, which `ci-runs` does with #12's base/head lookup. +- **Run `id` and `index_in_repo` are two id spaces, and both resolve on `/actions/runs/{x}`** to different real runs. The web URL shows the second. `ci-runs` emits both, the log concepts take `id` only, and a non-numeric value is refused rather than guessed. + +On Forgejo 16 (verified on codeberg.org, `16.0.0-dev-694`): `actions/runs/{id}/jobs` returns jobs carrying both `id` and `task_id`, and **the log endpoint accepts the job `id`, not the `task_id`** — passing the task id returns `{"message":"resource does not exist"}`, which is exactly the 404 that reads as "no logs" if it is not distinguished. `actions/jobs/{id}/logs` served 142 KB / 1599 lines in 1.02 s as `text/plain` with `accept-ranges: bytes`; `?step=N` is accepted and ignored. + +## Design + +### Tiering + +| Question | Concept | Reads a log? | +|---|---|---| +| Did my push pass? | `ci-runs` | No | +| Is it still running, which job is pending? | `ci-runs`, `ci-run-view` | No | +| It failed — why? | `ci-failures` | Yes, one job | +| Is this mine or pre-existing? | `ci-runs` + `CODEV_CI_WORKFLOW` | No | +| Extraction gave up — show me | `ci-run-log` | Yes, one window | + +`ci-run-log` is a separate concept rather than a flag, per the issue's second comment: a window parameter on the main call gets passed by habit, and then every status question drags a log again. + +### The extraction ladder, and the three traps it was built against + +All three are from the real captured logs, and each is a test: + +1. **ANSI.** The payload line is `ESC[41mESC[1m FAIL ESC[22mESC[49m src/…`, so a matcher that does not clean first matches **nothing** and reports "no recognized failure" on a log that plainly contains one. Cleaning is not cosmetic. +2. **"First line matching an error pattern" returns line 1257 of 2528**: `[artifact-canvas] Error: host blew up` — a fixture string printed by a *passing* test. The real failure is at 2471. The ladder's generic rung therefore anchors patterns at the **start** of the line; that decoy's `Error:` is mid-line and cannot match, and anchoring holds even in logs with no test summary to measure against. +3. **`Test Files` appears four times before the failing summary**, three of them saying `passed` and one a shell line echoing `grep -q "Test Files.*passed"`. Any rule taking the first match reports a passing suite as the failure. + +Rungs, in order, with the one that fired named in `matchedBy`: `vitest`/jest → `go-test` → `tsc` → the runner's `##[error]` marker → line-anchored `first-error` → refusal. Runner recognition sits above the `##[error]` marker deliberately (and as issue #13's own priority order asks): the marker returns one sentence, the vitest rung returns the whole Failed Tests block — test name, assertion, expected/received, file:line. + +**Deviation from the approved plan, stated plainly.** The plan said the generic rung would fire only *after* a passing-suite boundary and otherwise fall through to refusal. Implemented, it fires with anchoring always and the boundary as a preference. Anchoring is what actually kills the observed false positive; refusing whenever no test summary exists would have returned `extracted: false` for the whole class of install/setup/compile failures and bought no safety. + +### Refusal is a handoff + +```json +{ "extracted": false, "reason": "no recognized failure pattern", + "failures": [{ "jobId": 11952749, "jobName": "test-unit", "logLines": 1599 }], + "next": "ci-run-log CODEV_CI_RUN_ID=6554924 CODEV_CI_JOB_ID=11952749 CODEV_CI_LOG_TAIL=80" } +``` + +No log lines at all. A builder handed 50 arbitrary lines treats them as the diagnosis and reasons from noise; one told extraction failed reads the log with the call the response already handed it. + +### Errors are values + +Every ci-* concept prints one JSON object on stdout on success **and** failure, so `timeout` / `not-found` / `unsupported-server` / `forge-error` / `bad-input` stay distinguishable after `executeForgeCommand` has flattened everything else to `null`. `executeForgeCommandDetailed` is added for callers that need the distinction in TypeScript: it returns `{ok, data, stdout, stderr, exitCode, timedOut, unavailable, durationMs}` and keeps stdout on the failure path. + +On a Forgejo below 16 the response is `unsupported-server`, naming the version found and the version needed **and still listing the failing job names it could determine** — never an empty `failures` array. "Your CI is fine" and "I cannot see your CI at all" are opposite facts and must not be the same observation. + +### `codev forge ` + +Added at the architect's direction at the dev-approval gate, and the reasoning is correctness rather than convenience: calling `packages/codev/scripts/forge/github/ci-failures.sh` by path **bypasses resolution** — the config lookup, the provider preset, and any per-repo override — so a repo that overrides a concept gets the github default against its own forge. The reference Forgejo repo carried three such overrides until #12 shipped. It delegates to `executeForgeCommandDetailed`, prints stdout verbatim and exits with the script's code; its own additions are exit 2 for an unknown concept (listing the valid ones) and exit 3 for a concept disabled for the provider, named. + +## Files Changed + +- `packages/codev/scripts/forge/_ci-extract.sh` (+202 / -0) — the extraction ladder +- `packages/codev/scripts/forge/_ci-lib.sh` (+399 / -0) — envelope, caps, cache, windows, id validation +- `packages/codev/scripts/forge/_timeout.sh` (+100 / -0) — #12's watchdog, now shared by both providers +- `packages/codev/scripts/forge/gitea/_lib.sh` (+8 / -79) — sources the shared watchdog +- `packages/codev/scripts/forge/gitea/_ci.sh` (+229 / -0) +- `packages/codev/scripts/forge/gitea/{ci-runs,ci-run-view,ci-failures,ci-run-log}.sh` (+468 / -0) +- `packages/codev/scripts/forge/github/_lib.sh` (+55 / -0) +- `packages/codev/scripts/forge/github/{ci-runs,ci-run-view,ci-failures,ci-run-log}.sh` (+407 / -0) +- `packages/codev/src/lib/forge.ts` (+119 / -11) — registration, gitlab/linear disabled, `executeForgeCommandDetailed` +- `packages/codev/src/lib/forge-contracts.ts` (+181 / -0) +- `packages/codev/src/commands/forge.ts` (+97 / -0) — `codev forge ` +- `packages/codev/src/cli.ts` (+19 / -0) +- `packages/codev/src/__tests__/pir-13-ci-concepts.test.ts` (+976 / -0) +- `packages/codev/src/__tests__/fixtures/pir-13/{github-vitest-failure,forgejo-go-failure}.log.gz` (2 files, 57 KB) +- `packages/codev/src/__tests__/forge.test.ts` (+10 / -2) — concept count 18 → 22 +- `packages/codev/src/__tests__/spec-1280-measurement-instrument.test.ts` (+18 / -25) — timeout ceiling, see Flaky Tests +- `.claude/skills/forge/SKILL.md`, `.codex/skills/forge/SKILL.md` (+133 / -0 each, byte-identical twins) +- `codev/resources/arch.md`, `codev/resources/lessons-critical.md`, `codev/resources/lessons-learned.md` +- `codev/plans/13-ci-forge-concepts.md`, `codev/reviews/13-ci-forge-concepts.md`, `codev/state/pir-13_thread.md` + +## Commits + +- `700aefc63` feat(forge): CI concept plumbing — shared timeout, extraction ladder, envelope +- `d4605f6b2` feat(forge): the four CI concepts for GitHub +- `1fd18d491` feat(forge): the four CI concepts for Gitea/Forgejo +- `346243ef5` test(forge): pin the CI concepts against two real captured logs +- `5cd31ef16` fix(forge): send status=cancelled, the spelling both forges actually accept +- `3e552014e` fix(forge): a CLI that exits 0 with non-JSON still gets an envelope +- `494a17352` fix(forge): gitea ci-runs reports truncation when it hits the page ceiling +- `8e26a661a` fix(forge): reject a non-numeric run or job id before it reaches a URL +- `067f7b179` test(forge): the concept count is 22, not 18 +- `9893db9cd` test: give the prompt-surface instrument a ceiling above its own cost +- `3334bf9e8` feat(cli): codev forge \ — run a concept through the real resolver + +## Test Results + +- `npm run build`: ✓ pass +- `npm test`: ✓ pass — 5567 passed, 0 failed, 48 skipped (5615). **62 new tests** in `pir-13-ci-concepts.test.ts`, plus the two concept-count assertions updated in `forge.test.ts`. + +### Verification coverage — three tiers, and they are not the same + +**Do not read the third as if it were the first two.** + +**1. GitHub — live, end to end through the real dispatcher** (config load → preset → env → script → JSON parse), against `pseudoseed/codev`: + +| Call | Time | Result | +|---|---|---| +| `ci-runs` (limit 3) | 987 ms | 3 runs, `truncated: true` | +| `ci-runs --status failure` | 695 ms | 3 runs | +| `ci-runs --branch builder/pir-12` | 875 ms | 3 runs | +| `ci-run-view` run 32515040122 | 1464 ms | 5 jobs, 1 failing, failing step named | +| `ci-failures` (cold) | 2503 ms | `vitest`, **23 of 2528 lines**, 1.2 KB response | +| `ci-failures` (cached) | 1323 ms | identical answer, no download | +| `ci-run-log` tail 10 | 1215 ms | lines 2519–2528 | +| `ci-run-log` head 5 | 1232 ms | lines 1–5 | +| `ci-run-log` grep AssertionError | 1190 ms | 14 lines, matched at 2472 and 2497 | +| `ci-run-log` with no window | 29 ms | `bad-input` — refused before spending an API call | + +Also verified against a second real run (`32448538074`, the architect's): 919 lines → 20, `matchedBy: vitest`. + +**2. Forgejo 15.0.2 — live, end to end through the real dispatcher**, against `~/dev/entriq` on the bare `gitea` preset with no overrides: + +| Call | Time | Result | +|---|---|---| +| `ci-runs` (limit 3) | 726 ms | 3 runs | +| `ci-runs --status failure` | 407 ms | 3 runs | +| `ci-runs --branch builder/air-364` | 2255 ms | 2 runs, matched via PR ref `#3855` | +| `ci-run-view` run 11130 | 3269 ms | **10 jobs via `tasks-scan`**, 1 failing | +| `ci-failures` run 11130 | 3737 ms | `unsupported-server`, both versions named, failing job still listed | +| `ci-run-log` run 11130 | 3903 ms | `unsupported-server` | + +`codev doctor` in that repo resolves all four concepts to `tea`. **entriq was read-only** — nothing was written to it. Its working tree carries an unrelated uncommitted `.codev/config.json` edit that predates this session (13:34 MDT, deleting the three overrides #12 made redundant). + +**3. Forgejo 16 — HTTP level only. NOT driven through the dispatcher against a live v16 server.** The v16 routes were verified with unauthenticated `curl` against `codeberg.org/forgejo/forgejo` (jobs list, job log 200/`text/plain`/142 KB/1.02 s, run-log zip), and the code path is covered by unit tests driving a stubbed `tea` that serves the **real captured codeberg job log**. Driving it through the dispatcher needs `tea` to hold a Codeberg login, which the architect deferred. `git.pseudoseed.com` was still on 15.0.2 at the time of writing; the owner has filed the Forgejo 16 upgrade, and if it lands the same code lights up with no change. + +**4. Provider degradation**: `resolveAllConcepts` reports all four concepts as `disabled` for `gitlab` and `linear` — not merely absent, which would fall through to the github default and run `gh` against whatever remote it resolved. + +## Architecture Updates + +**COLD — `codev/resources/arch.md`**, § Integration Points → Forge Concept Commands. Concept count 18 → 22, plus four additions, all current-state reference rather than changelog: + +1. **How to invoke a concept**, and that naming a script by path bypasses resolution — the defect `codev forge` exists to close. +2. **`executeForgeCommandDetailed`** and when the `null`-flattening of `executeForgeCommand` is not good enough. +3. **The ci-* contract**: errors as values on stdout, `logLines`/`returnedLines`/`truncated` on anything carrying log text, refusal carrying no log lines, and where the shared implementation lives. +4. **The measured CI behaviour of both forges** — the `--log-failed` correction, the Forgejo 16 log-API floor, the four query-parameter footguns, the `#` branch labelling, the two id spaces, the `cancelled` spelling, and the per-endpoint costs. + +**Nothing promoted to `arch-critical.md`.** It is at its cap of ten, and all of this matters only when writing or calling a forge concept — the SKILL.md and the cold map carry it to whoever needs it. + +## Lessons Learned Updates + +**HOT — `codev/resources/lessons-critical.md`**, one entry rewritten in place rather than a displacement, because the new instance is the same lesson arriving through a wider door. The existing entry covered truncation only: + +> ~~A truncated result is indistinguishable from a complete one once emitted — give "I stopped early" its own signal and emit nothing, never a partial answer that reads as whole.~~ + +is now + +> **"I could not tell" must never be spelled the same way as "no". A truncation, an unreachable API, and a server too old to answer each need their own signal and must emit nothing else — a partial or empty answer reads as a complete, negative one.** + +The architect counted this as the seventh arrival of the same rule in one day. This PR met it three times: an old Forgejo that would have looked like a green run, a page ceiling that reported `truncated: false`, and a CLI exiting 0 with non-JSON that produced no stdout at all. The file stays at ten entries. + +**COLD — `codev/resources/lessons-learned.md`**, seven entries across Process and Testing: + +- Measure a tool before building on the claim that it already does the work (`--log-failed`). +- Extraction must recognise, not truncate — and must strip ANSI before it can recognise anything; anchor generic error patterns at the start of the line. +- A CLI's `--help` is not the API's vocabulary (`canceled` vs `cancelled`). +- Ask which id space an endpoint means when two plausible ids both resolve. +- Under `set -e`, a helper returning non-zero decides its caller's exit status. +- A shell helper that "sets a global" sets nothing when the caller captures it in `$( )`. +- A test whose ceiling sits below its own cost reads as flaky. + +## Things to Look At During PR Review + +- **`_ci-extract.sh`, the awk program.** It is the piece most likely to be subtly wrong on a runner nobody here uses. The ladder is ordered and each rung names itself in `matchedBy`, so a wrong answer is at least attributable — but a new runner format falls to `first-error` or to refusal, and the refusal path is the safe one by design. +- **`ci_clean_log`'s sed pipeline.** ANSI/OSC stripping in POSIX `sed` with a literal ESC. It is line-count preserving, which is what makes `from`/`to` usable as line numbers into the raw log; a change that drops or adds a line silently breaks the `ci-run-log` handoff. +- **The Forgejo-15 `tasks-scan` fallback** (`gitea/_ci.sh`). It pages `actions/tasks` filtering on `run_number`, stops the moment it walks past the run, and reports truncation only when it ran out of allowance *before* reaching it. The early stop is what keeps a recent run at one page; the ceiling is a separate, higher knob (`CODEV_CI_TASKS_MAX_PAGES`, default 20) because a page of 50 tasks spans only ~6 runs. +- **`executeForgeCommandDetailed`'s `timedOut`** is derived from `err.killed && err.signal`, not from an exit code — a killed process can still exit with a status, which is the exact confusion #12 documented in `gitea_timeout`. +- **The gzipped fixtures.** They are verbatim captures; the tests assert against traps that only exist because the bytes are real. Regenerating or normalising them would quietly delete the coverage. + +## How to Test Locally + +- **View diff**: VSCode sidebar → right-click builder `pir-13` → **Review Diff** +- **The headline path** (needs `gh` authenticated against this repo): + ```bash + CODEV_CI_RUN_ID=32515040122 codev forge ci-failures | jq + # 23 lines out of 2528: the AssertionError, the test file and line, the step name + ``` +- **The windows**, and that the second call is free (cached): + ```bash + CODEV_CI_RUN_ID=32515040122 CODEV_CI_LOG_GREP=AssertionError codev forge ci-run-log | jq '{from,to,matches,matchLines}' + CODEV_CI_RUN_ID=32515040122 codev forge ci-run-log # refuses: no window + ``` +- **Loud degradation**, from a Forgejo repo (`~/dev/entriq`): + ```bash + CODEV_CI_RUN_ID=11130 codev forge ci-run-view | jq '{jobSource, jobs: (.jobs|length)}' # works + CODEV_CI_RUN_ID=11130 codev forge ci-failures | jq '{error, serverVersion, needs}' # unsupported-server + codev forge team-activity # named, exit 3 + codev forge ci-failure # unknown, lists valid, exit 2 + ``` +- **`codev doctor`** in both repos: four new concepts, `gh` under github, `tea` under gitea. + +## Flaky Tests + +None skipped. One pre-existing flake was **fixed rather than skipped**: + +`packages/codev/src/__tests__/spec-1280-measurement-instrument.test.ts` capped every test at 60 s inline, while `scripts/measure-prompt-surface.sh` costs 25–30 s per invocation and several of those tests invoke it two or three times (two locales, two runs for determinism, a fixture plus the live repo). Under full-suite load a *different* pair failed each run — the signature of a ceiling set below the work, not of a defect. The file passes 24/24 in isolation before and after. Raised to 240 s via one named constant (`INSTRUMENT_TIMEOUT_MS`), which is the same coverage given room; `.skip` would have bought a green run by deleting the check. From 1f71145bb84e7d6815efc59ffe862a6c4d37e86e Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:50:04 -0600 Subject: [PATCH 21/30] chore(porch): 13 record PR #23 --- .../projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 0b19f660c..b2f4d6ae0 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -19,4 +19,9 @@ iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T22:43:32.878Z' +updated_at: '2026-08-21T22:50:04.111Z' +pr_history: + - phase: review + pr_number: 23 + branch: builder/pir-13 + created_at: '2026-08-21T22:50:04.111Z' From f45136a3b0649438bcb74f50340fc66713cb492a Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:50:10 -0600 Subject: [PATCH 22/30] chore(porch): 13 review build-complete --- codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index b2f4d6ae0..826b90413 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -16,10 +16,10 @@ gates: pr: status: pending iteration: 1 -build_complete: false +build_complete: true history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T22:50:04.111Z' +updated_at: '2026-08-21T22:50:10.139Z' pr_history: - phase: review pr_number: 23 From 79fb7b6646b0fae2fcd62072ce72f3d42f5e9822 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 16:54:10 -0600 Subject: [PATCH 23/30] [PIR #13] docs: record the review-lane coverage gap and porch's wrong remedy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two of three lanes could not run — codex and gemini, both provider quota. The lane files are gitignored, so the evidence lives here, where it survives the merge: the verbatim agy probe showing 1.1.17 installed and authenticated and failing on 'Individual quota reached', against porch's notice advising a reinstall and sign-in that could not have helped. Porch will print 'All reviewers approved' over two SKIPPED files (#20). The table is the truth, not the summary line. Co-Authored-By: Claude Opus 5 --- codev/reviews/13-ci-forge-concepts.md | 38 +++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/codev/reviews/13-ci-forge-concepts.md b/codev/reviews/13-ci-forge-concepts.md index 2d16469f5..7ce9c9624 100644 --- a/codev/reviews/13-ci-forge-concepts.md +++ b/codev/reviews/13-ci-forge-concepts.md @@ -170,6 +170,44 @@ Also verified against a second real run (`32448538074`, the architect's): 919 li **4. Provider degradation**: `resolveAllConcepts` reports all four concepts as `disabled` for `gitlab` and `linear` — not merely absent, which would fall through to the github default and run `gh` against whatever remote it resolved. +## ⚠ Review lane coverage — read this before trusting the review depth + +**As of 2026-08-21 ~17:00 MDT, two of the three lanes could not run, and the PR is being HELD rather than merged on the remainder.** + +| Lane | Verdict | Why | +|---|---|---| +| **codex** (gpt-5.6-sol) | **NEVER RAN** | Provider quota. Refused in seconds, before any model work: *"You've hit your usage limit… try again at Aug 27th, 2026 4:01 PM."* The same quota blocked codex on #2, #4, #11 and #12 the same day. | +| **gemini** (agy) | **NEVER RAN** | Provider quota, **not** the reason porch reported — see below. Resets ~2026-08-28. | +| **claude** (opus-5) | in progress at time of writing | — | + +**Porch's own gate summary will say otherwise, and it is wrong.** Both lane files carry `VERDICT: SKIPPED`, which `parseVerdict` (`porch/verdict.ts`) does not recognise — it knows only `APPROVE`, `REQUEST_CHANGES`, `COMMENT` — so it falls through to the "treat as COMMENT" default, and `allApprove` counts `COMMENT` as approval. Porch will print **"All reviewers approved!"** over two reviewers that read nothing. That is **#20**, filed by PIR #12; it is porch behaviour, not anything in this diff, and it is not fixed here. Read this table, not the summary line. + +**The lane files themselves are gitignored** (`.gitignore:65`, `codev/projects/*/*.txt`), which is why the evidence is restated here, where it survives the merge. + +### Porch reported a remedy that cannot work + +The gemini skip notice read: + +> The Gemini (Antigravity `agy`) reviewer was skipped: agy exited with code 1. This is a non-blocking skip; the remaining reviewers still apply. To enable the Gemini lane, install the CLI (https://antigravity.google/cli/install.sh) and run `agy` once to sign in. + +agy **is** installed and authenticated. Probed directly, verbatim: + +``` +$ agy --version +1.1.17 +$ which agy +/Users/chris/.local/bin/agy +$ echo hello | agy -p "reply with the single word OK" +Error: Individual quota reached. Please upgrade your subscription to increase your limits. Resets in 157h50m8s. +rc=1 +``` + +Reinstalling and signing in again would have changed nothing and cost whoever followed the advice their afternoon. A confidently printed remedy that cannot work is the same defect class as #21, where the stuck-mailbox alert names a command that cannot clear a composer. The architect is filing it separately. + +### Why this PR is held rather than merged on one lane + +One of three would be the thinnest coverage of the day, on the diff least suited to it: five POSIX `sh` scripts, a hand-rolled awk extractor and a pile of jq, where the two absent lanes are the ones that most often catch quoting and portability defects — and where two of the three bugs found during implementation were exactly that class. PR #24 (issue #22) adds an **opencode** consult lane on an account unrelated to either exhausted quota; the architect is merging it and re-running this review phase with that lane available, rather than waiting for the Aug 27/28 quota resets or merging thin. **This section is rewritten with the real verdicts once that re-run completes.** + ## Architecture Updates **COLD — `codev/resources/arch.md`**, § Integration Points → Forge Concept Commands. Concept count 18 → 22, plus four additions, all current-state reference rather than changelog: From 8a57b262c660662027e2878dd2174a647ba89b9d Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 17:00:04 -0600 Subject: [PATCH 24/30] [PIR #13] fix: the two defects the claude review lane found MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. THE TIMEOUT LAYERING WAS INVERTED. executeForgeCommandDetailed defaults to a 30s ceiling; the scripts default to a 60s CODEV_FORGE_TIMEOUT. At the defaults the outer kill fired first, so a stalled forge arrived as a generic Node kill and the script's NAMED timeout envelope never printed — the whole point of the inner watchdog. The comment in forge.ts asserted the opposite. It survived because the timeout test forced CODEV_FORGE_TIMEOUT=2, and a test that overrides the defaults cannot detect the defaults being wrong. The new test pins the ORDERING at defaults instead. The inversion predates this PR — #12 gave gitea a 60s watchdog under the same 30s ceiling — so the fix here is narrow: codev forge sets its ceiling above the watchdog, and forge.ts documents the real ordering rather than the intended one. 2. sed -n "1,0p" ON AN EMPTY JOB LOG. head/tail built a reversed range, tolerated by BSD sed and REJECTED by GNU sed — invisible on the macOS box this was written on, and on Linux, where CI runs, an abort under set -e with nothing on stdout. An empty log is now an answer, pinned for all three window modes. Plus a misindented exit and a trailing space in gitea/ci-runs.sh. NOT done, deliberately: a conformance test tying the Ci* contracts to script output. Doing it for four contracts would leave the other eighteen untested while implying they were covered; it is filed as its own issue. Co-Authored-By: Claude Opus 5 --- codev/resources/lessons-learned.md | 2 + codev/reviews/13-ci-forge-concepts.md | 16 ++++ packages/codev/scripts/forge/_ci-lib.sh | 17 ++++ packages/codev/scripts/forge/gitea/ci-runs.sh | 4 +- .../src/__tests__/pir-13-ci-concepts.test.ts | 85 +++++++++++++++++++ packages/codev/src/commands/forge.ts | 17 +++- packages/codev/src/lib/forge.ts | 20 +++-- 7 files changed, 153 insertions(+), 8 deletions(-) diff --git a/codev/resources/lessons-learned.md b/codev/resources/lessons-learned.md index ee8e83785..9d41ccc78 100644 --- a/codev/resources/lessons-learned.md +++ b/codev/resources/lessons-learned.md @@ -464,6 +464,8 @@ so it survives review. Pin the constant to the highest migration block in a test - [From #13] **Under `set -e`, a helper that returns non-zero decides its caller's exit status.** `ci_fail` printed an error envelope and returned 1; every call site then had `exit 2` for a bad input on the next line, and none of them ever ran — `set -e` killed the script at the helper. Reporting and deciding the exit status are separate jobs; a reporting helper should return 0. - [From #13] **A shell helper that "sets a global" sets nothing when the caller captures it.** Two helpers assigned `CI_TRUNCATED` / `GITEA_JOB_SOURCE` for their callers to read, and every caller ran them inside `$( )` — a subshell, so the values were discarded and jq got an empty `--argjson`. Return the data, or write it to a file the caller reads. - [From #13] **A test whose ceiling sits below its own cost reads as flaky.** `spec-1280-measurement-instrument.test.ts` capped tests at 60s that cost 80-100s (the instrument is ~25-30s per invocation and several tests invoke it two or three times). A *different* pair failed each full-suite run, which looks like flakiness and invites `.skip`. Measure the work, then set the ceiling above it — that is fixing the bug rather than hiding it. +- [From #13] **A test that overrides the defaults cannot detect the defaults being wrong.** The timeout test forced `CODEV_FORGE_TIMEOUT=2` and passed, proving the watchdog fires when told to — while at the real defaults the outer 30s ceiling killed the command before the inner 60s watchdog could name the endpoint, so the whole point of the inner watchdog never fired. Where two ceilings, retries or precedence rules compose, pin the ORDERING at the values that actually ship. +- [From #13] **`sed -n "1,0p"` is tolerated by BSD sed and rejected by GNU sed.** An empty input built that reversed range, which was invisible on the macOS box it was written on and would have aborted the script under `set -e` on Linux, where CI runs. Guard the empty case explicitly rather than trusting a range expression to degrade. ## UI/UX diff --git a/codev/reviews/13-ci-forge-concepts.md b/codev/reviews/13-ci-forge-concepts.md index 7ce9c9624..9e8184332 100644 --- a/codev/reviews/13-ci-forge-concepts.md +++ b/codev/reviews/13-ci-forge-concepts.md @@ -204,6 +204,22 @@ rc=1 Reinstalling and signing in again would have changed nothing and cost whoever followed the advice their afternoon. A confidently printed remedy that cannot work is the same defect class as #21, where the stuck-mailbox alert names a command that cannot clear a composer. The architect is filing it separately. +### What the one lane that did run found + +**claude (opus-5) — VERDICT: COMMENT, CONFIDENCE: HIGH.** Two real defects, both fixed before this was written, and both in precisely the class the two absent lanes exist to catch. + +**1. The timeout layering was inverted, and my own test hid it.** `executeForgeCommandDetailed` defaults to a 30s ceiling; the scripts default to a 60s `CODEV_FORGE_TIMEOUT`. **At the defaults the outer kill fires first**, so a stalled forge arrived as a generic Node kill and the script's *named* timeout envelope — the entire point of the inner watchdog, and the thing #17 and #8 are about — never printed. The comment in `forge.ts` asserted the opposite ordering. + +The reason it survived to review is worth more than the fix: **the timeout test forced `CODEV_FORGE_TIMEOUT=2`, and a test that overrides the defaults cannot detect the defaults being wrong.** It proved the watchdog works when you tell it to; it could not prove that the watchdog is the ceiling that fires. The correction is a test that pins the **ordering at defaults** — it lets the real 60s-vs-30s relationship decide, and fails if the outer ceiling ever eats the inner one again. + +**The inversion predates this PR.** #12 gave the gitea scripts a 60s watchdog under this same 30s ceiling; it was inherited here, not introduced. Correcting it globally would change the timeout behaviour of every concept and every caller, so what this PR does is narrower: `codev forge` sets its ceiling to the script watchdog plus 30s, and `forge.ts` now documents the real ordering instead of the intended one. The general case is left to a caller passing `timeoutMs`. + +**2. `sed -n "1,0p"` on an empty job log.** head/tail built a reversed range whenever a log came back empty. **BSD sed tolerates it and GNU sed rejects it** — so this was invisible on the macOS box it was written on, and on Linux, *which is where CI runs*, the script would have aborted under `set -e` with nothing at all on stdout: the one shape these concepts promised never to produce. It is the textbook case for why the absent lanes matter, found by the lane that ran on the platform where it cannot bite. An empty log is now an answer (`logLines: 0`, empty window, `truncated: false`), pinned for all three window modes plus an assertion against the reversed range itself. + +A third finding was cosmetic (a misindented `exit` and a trailing space in `gitea/ci-runs.sh`), fixed. + +A fourth was noted rather than requested: the `Ci*` contracts in `forge-contracts.ts` are documentation-only, with no conformance test tying them to actual script output. **Deliberately not done here.** Adding it for the four CI contracts alone would leave the other eighteen forge contracts untested while implying they were covered — worse than uniformly untested. The architect is filing it as its own issue across all forge contracts. + ### Why this PR is held rather than merged on one lane One of three would be the thinnest coverage of the day, on the diff least suited to it: five POSIX `sh` scripts, a hand-rolled awk extractor and a pile of jq, where the two absent lanes are the ones that most often catch quoting and portability defects — and where two of the three bugs found during implementation were exactly that class. PR #24 (issue #22) adds an **opencode** consult lane on an account unrelated to either exhausted quota; the architect is merging it and re-running this review phase with that lane available, rather than waiting for the Aug 27/28 quota resets or merging thin. **This section is rewritten with the real verdicts once that re-run completes.** diff --git a/packages/codev/scripts/forge/_ci-lib.sh b/packages/codev/scripts/forge/_ci-lib.sh index ceff8d5f3..af39830b0 100644 --- a/packages/codev/scripts/forge/_ci-lib.sh +++ b/packages/codev/scripts/forge/_ci-lib.sh @@ -328,6 +328,23 @@ ci_window_emit() { _concept="$1"; _tmp="$2"; _provider="$3"; _run="$4"; _job="$5"; _jobname="$6"; _cached="$7" _log="${_tmp}/clean.log" _total=$(awk 'END {print NR}' "$_log") + # An empty log is an answer, not a crash. Without this, head/tail build + # `sed -n "1,0p"`, which BSD sed tolerates and GNU sed rejects — so the script + # would abort under `set -e` on Linux with NOTHING on stdout, which is the one + # shape these concepts promised never to produce. Found by the claude review + # lane and reproduced with `sed --posix` before fixing. + if [ "$_total" -eq 0 ]; then + jq -cn --arg provider "$_provider" --arg run "$_run" --argjson job "$_job" --arg jobName "$_jobname" \ + --arg kind "$CI_WIN_KIND" --arg arg "$CI_WIN_ARG" --argjson cached "$_cached" \ + '{ok: true, provider: $provider, runId: ($run | tonumber? // $run), jobId: $job, jobName: $jobName, + window: {kind: $kind, arg: $arg}, + logLines: 0, returnedLines: 0, from: 0, to: 0, + contiguous: true, truncated: false, + matches: (if $kind == "grep" then 0 else null end), + matchLines: (if $kind == "grep" then [] else null end), + cached: $cached, lines: []}' + return 0 + fi _matchlines=null _matches=null _contiguous=true diff --git a/packages/codev/scripts/forge/gitea/ci-runs.sh b/packages/codev/scripts/forge/gitea/ci-runs.sh index 915e1b79b..d82b90b4f 100755 --- a/packages/codev/scripts/forge/gitea/ci-runs.sh +++ b/packages/codev/scripts/forge/gitea/ci-runs.sh @@ -57,8 +57,8 @@ while [ "$PAGE" -le "$CI_MAX_PAGES" ]; do exit 1 fi if [ "$rc" -eq 44 ]; then - gitea_ci_unsupported "$CONCEPT" "workflow-runs" "$(jq -cn --arg r "$REPO" '{repo: $r}')" - exit 1 + gitea_ci_unsupported "$CONCEPT" "workflow-runs" "$(jq -cn --arg r "$REPO" '{repo: $r}')" + exit 1 fi if [ "$rc" -ne 0 ]; then ci_fail "$CONCEPT" forge-error "Forgejo could not list workflow runs for ${REPO}: $(head -c 200 "$TMP/page.json")" diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts index 6e0f8081d..24713597c 100644 --- a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -572,6 +572,34 @@ describe.skipIf(!hasJq())('#13 — ci-run-log takes exactly one window, and says expect(narrow.json!.returnedLines).toBe(2); }); + it.each(['tail', 'head', 'grep'])('survives an EMPTY log in %s mode', (kind) => { + // Without a guard, head/tail build `sed -n "1,0p"` — tolerated by BSD sed, + // REJECTED by GNU sed, so on Linux the script aborted under `set -e` with + // nothing on stdout. Found by the claude review lane on a macOS box, where + // the defect is invisible. + fs.writeFileSync(path.join(tmp, 'empty.log'), ''); + stubGh(ONE_FAILING_JOB, path.join(tmp, 'empty.log')); + const env: Record = { CODEV_CI_RUN_ID: '32515040122' }; + if (kind === 'tail') env.CODEV_CI_LOG_TAIL = '10'; + if (kind === 'head') env.CODEV_CI_LOG_HEAD = '10'; + if (kind === 'grep') env.CODEV_CI_LOG_GREP = 'anything'; + + const r = run(githubDir, 'ci-run-log.sh', env); + expect(r.status, r.stderr).toBe(0); + expect(r.json!.ok).toBe(true); + expect(r.json!.logLines).toBe(0); + expect(r.json!.returnedLines).toBe(0); + expect(r.json!.lines).toEqual([]); + expect(r.json!.truncated).toBe(false); + }); + + it('builds no reversed sed range for an empty log', () => { + // The portable-behaviour assertion, since the box this runs on may be the + // one that tolerates it: reject the range itself, not just its effect. + const lib = fs.readFileSync(path.join(forgeScripts, '_ci-lib.sh'), 'utf-8'); + expect(lib).toMatch(/if \[ "\$_total" -eq 0 \]; then/); + }); + it('reports no match as an empty window rather than as a failure', () => { stubWithFixture(); const r = run(githubDir, 'ci-run-log.sh', { @@ -632,6 +660,63 @@ describe.skipIf(!hasJq())('#13 — a timeout reports as a timeout, not as an emp expect(r.stderr, 'the watchdog is reporting its own death onto a clean path').toBe(''); }); + it('codev forge puts its own ceiling ABOVE the script watchdog, so the named envelope wins', async () => { + // The inversion the claude lane caught: executeForgeCommandDetailed defaults + // to 30s and the scripts default to a 60s CODEV_FORGE_TIMEOUT, so at the + // DEFAULTS Node killed the command first and the script's named timeout + // envelope never printed. The earlier timeout test forced a 2s watchdog, + // which hid it. This one pins the ordering itself. + const watchdog = 4; + // The stub lives one directory DOWN from a copy of _timeout.sh, because + // _ci-lib.sh resolves its sibling as `$(dirname "$0")/../_timeout.sh` — + // the same layout every real provider directory has. + const providerDir = path.join(tmp, 'fake-provider'); + fs.mkdirSync(providerDir, { recursive: true }); + fs.copyFileSync(path.join(forgeScripts, '_timeout.sh'), path.join(tmp, '_timeout.sh')); + const slow = path.join(providerDir, 'slow-forge.sh'); + fs.writeFileSync( + slow, + [ + '#!/bin/sh', + `. ${JSON.stringify(path.join(forgeScripts, '_ci-lib.sh'))}`, + 'rc=0', + 'ci_tool sleep 30 || rc=$?', + '[ "$rc" -eq 124 ] && ci_fail_timeout ci-runs "the slow forge" "$CI_TIMEOUT"', + 'exit 1', + ].join('\n'), + { mode: 0o755 }, + ); + fs.chmodSync(slow, 0o755); + + const root = path.join(tmp, 'ws-timeout'); + fs.mkdirSync(path.join(root, '.codev'), { recursive: true }); + fs.writeFileSync(path.join(root, '.codev', 'config.json'), JSON.stringify({ forge: { 'ci-runs': slow } })); + + const previous = process.env.CODEV_FORGE_TIMEOUT; + process.env.CODEV_FORGE_TIMEOUT = String(watchdog); + try { + const out: string[] = []; + const err: string[] = []; + const code = await runForgeConcept('ci-runs', { + cwd: root, + stdout: (t) => out.push(t), + stderr: (t) => err.push(t), + }); + // The SCRIPT reported, not Node: a named envelope on stdout, and an exit + // status from the script rather than a signal kill. + expect(JSON.parse(out.join('')), 'the outer ceiling fired first and ate the named envelope').toMatchObject({ + ok: false, + error: 'timeout', + seconds: watchdog, + }); + expect(code).toBe(1); + expect(code, 'Node killed it — 124 is the outer backstop, not the inner watchdog').not.toBe(124); + } finally { + if (previous === undefined) delete process.env.CODEV_FORGE_TIMEOUT; + else process.env.CODEV_FORGE_TIMEOUT = previous; + } + }, 60_000); + it('executeForgeCommandDetailed distinguishes a timeout from a failure and from silence', async () => { const slow = path.join(tmp, 'slow.sh'); fs.writeFileSync(slow, '#!/bin/sh\nsleep 30\n', { mode: 0o755 }); diff --git a/packages/codev/src/commands/forge.ts b/packages/codev/src/commands/forge.ts index 1bb9df9de..ee6ca1b6d 100644 --- a/packages/codev/src/commands/forge.ts +++ b/packages/codev/src/commands/forge.ts @@ -78,7 +78,22 @@ export async function runForgeConcept( return 3; } - const result = await executeForgeCommandDetailed(concept, undefined, { cwd, forgeConfig }); + // The outer ceiling MUST sit above the script watchdog, or the wrong one wins. + // executeForgeCommandDetailed defaults to 30s while the scripts default to a + // 60s CODEV_FORGE_TIMEOUT, so with the defaults a stalled forge was killed by + // Node first and the script's named timeout envelope — the whole point of the + // inner watchdog — never printed. Found by the claude review lane, which also + // noted that the test hid it by forcing a 2s watchdog. Give the script its + // full allowance plus a margin, and keep this as the backstop it is meant to + // be: it fires only if the watchdog itself fails. + const watchdogSeconds = Number.parseInt(process.env.CODEV_FORGE_TIMEOUT ?? '', 10); + const innerMs = (Number.isFinite(watchdogSeconds) && watchdogSeconds > 0 ? watchdogSeconds : 60) * 1000; + + const result = await executeForgeCommandDetailed(concept, undefined, { + cwd, + forgeConfig, + timeoutMs: innerMs + 30_000, + }); // stdout verbatim, including on the failure path: the ci-* concepts print // their structured error envelope there precisely so the class of failure diff --git a/packages/codev/src/lib/forge.ts b/packages/codev/src/lib/forge.ts index f8a34dce3..527bd0abb 100644 --- a/packages/codev/src/lib/forge.ts +++ b/packages/codev/src/lib/forge.ts @@ -37,12 +37,22 @@ function resolveScriptPath(provider: string, concept: string): string { const DEFAULT_MAX_BUFFER = 10 * 1024 * 1024; /** - * Default wall-clock ceiling for a forge command (30s). + * Default wall-clock ceiling for a forge command (30s), unchanged since before + * the scripts had watchdogs of their own. * - * The scripts carry their own, shorter watchdog (CODEV_FORGE_TIMEOUT, default - * 60s in scripts/forge/_timeout.sh) so that a stalled CLI surfaces as a NAMED - * timeout rather than as this outer kill, which can only report that something - * died. Both exist: the inner one explains, the outer one guarantees. + * **It is SHORTER than the script watchdog, not longer** (CODEV_FORGE_TIMEOUT, + * default 60s in scripts/forge/_timeout.sh), and the ordering matters: whichever + * ceiling fires first decides what the caller learns. This one can only report + * that something died; the inner one reports WHICH endpoint stopped answering + * and how long it was given. With the defaults as they stand, a stalled forge + * hits this 30s kill first and the named message never gets to fire. + * + * That inversion predates the CI concepts — #12 gave the gitea scripts a 60s + * watchdog under this same 30s ceiling — and correcting it globally would change + * the timeout behaviour of every concept and every caller, so it is not done + * here. What IS done: any caller that needs the named envelope passes + * `timeoutMs` above the script watchdog. `codev forge` does exactly that (see + * commands/forge.ts), which is why its timeout path reports a timeout by name. */ const DEFAULT_TIMEOUT_MS = 30_000; From 9a5a19bacdd2e044487557d134ab5e157d186ebd Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 17:04:25 -0600 Subject: [PATCH 25/30] [PIR #13] docs: test counts after the review-lane fixes (5572 passed, 67 new) Co-Authored-By: Claude Opus 5 --- codev/reviews/13-ci-forge-concepts.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/codev/reviews/13-ci-forge-concepts.md b/codev/reviews/13-ci-forge-concepts.md index 9e8184332..705a28ff3 100644 --- a/codev/reviews/13-ci-forge-concepts.md +++ b/codev/reviews/13-ci-forge-concepts.md @@ -130,7 +130,7 @@ Added at the architect's direction at the dev-approval gate, and the reasoning i ## Test Results - `npm run build`: ✓ pass -- `npm test`: ✓ pass — 5567 passed, 0 failed, 48 skipped (5615). **62 new tests** in `pir-13-ci-concepts.test.ts`, plus the two concept-count assertions updated in `forge.test.ts`. +- `npm test`: ✓ pass — 5572 passed, 0 failed, 48 skipped (5620). **67 new tests** in `pir-13-ci-concepts.test.ts`, plus the two concept-count assertions updated in `forge.test.ts`. ### Verification coverage — three tiers, and they are not the same From 5a61226d60d709a8e68913c80528554cabc60145 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 17:15:56 -0600 Subject: [PATCH 26/30] [PIR #13] fix: an unusable TMPDIR must not be reported as a missing run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The branch CI went red while the local suite was green. The trigger was a test harness that pointed TMPDIR at a directory it never created: macOS mktemp -d ignores an unusable TMPDIR and falls back, GNU mktemp honours it and fails, so 31 tests passed here and failed on the Linux runner. What that exposed matters more than the harness bug. Every concept needs a temp dir and the watchdog needs one per call, so mktemp -d failed inside the watchdog, gh was never run, the wrapper returned 1 — indistinguishable from the wrapped command failing — and the concept answered: {"ok":false,"error":"not-found","detail":"run 32515040122 could not be read (gh exit 1); pass the `id` from ci-runs, not the run `number`"} A temp-directory problem wearing the face of a missing run, with confident advice about which id to pass. Same rule as this PR's hot-tier lesson, arriving again in the code written to enforce it. - forge_timeout returns 125, not 1, when it cannot create a temp dir: the command never ran and must not share a status with the command failing. - Every CI concept runs ci_require_tmpdir before touching a forge — named forge-error, and no forge call at all. - The check tests [ -d "$TMPDIR" ] explicitly rather than leaning on mktemp, so macOS and Linux behave identically. mktemp alone would have kept the split. Diagnosed in an ubuntu:24.04 container rather than by round-tripping CI. Both paths verified there; the misdiagnosis is pinned by a test green on both platforms. Co-Authored-By: Claude Opus 5 --- codev/resources/lessons-learned.md | 1 + codev/reviews/13-ci-forge-concepts.md | 24 ++++++++++++- packages/codev/scripts/forge/_ci-lib.sh | 34 +++++++++++++++++++ packages/codev/scripts/forge/_timeout.sh | 11 +++++- .../codev/scripts/forge/gitea/ci-failures.sh | 2 ++ .../codev/scripts/forge/gitea/ci-run-log.sh | 2 ++ .../codev/scripts/forge/gitea/ci-run-view.sh | 2 ++ packages/codev/scripts/forge/gitea/ci-runs.sh | 2 ++ .../codev/scripts/forge/github/ci-failures.sh | 2 ++ .../codev/scripts/forge/github/ci-run-log.sh | 2 ++ .../codev/scripts/forge/github/ci-run-view.sh | 2 ++ .../codev/scripts/forge/github/ci-runs.sh | 2 ++ .../src/__tests__/pir-13-ci-concepts.test.ts | 23 +++++++++++++ 13 files changed, 107 insertions(+), 2 deletions(-) diff --git a/codev/resources/lessons-learned.md b/codev/resources/lessons-learned.md index 9d41ccc78..146ab6b3a 100644 --- a/codev/resources/lessons-learned.md +++ b/codev/resources/lessons-learned.md @@ -466,6 +466,7 @@ so it survives review. Pin the constant to the highest migration block in a test - [From #13] **A test whose ceiling sits below its own cost reads as flaky.** `spec-1280-measurement-instrument.test.ts` capped tests at 60s that cost 80-100s (the instrument is ~25-30s per invocation and several tests invoke it two or three times). A *different* pair failed each full-suite run, which looks like flakiness and invites `.skip`. Measure the work, then set the ceiling above it — that is fixing the bug rather than hiding it. - [From #13] **A test that overrides the defaults cannot detect the defaults being wrong.** The timeout test forced `CODEV_FORGE_TIMEOUT=2` and passed, proving the watchdog fires when told to — while at the real defaults the outer 30s ceiling killed the command before the inner 60s watchdog could name the endpoint, so the whole point of the inner watchdog never fired. Where two ceilings, retries or precedence rules compose, pin the ORDERING at the values that actually ship. - [From #13] **`sed -n "1,0p"` is tolerated by BSD sed and rejected by GNU sed.** An empty input built that reversed range, which was invisible on the macOS box it was written on and would have aborted the script under `set -e` on Linux, where CI runs. Guard the empty case explicitly rather than trusting a range expression to degrade. +- [From #13] **macOS `mktemp -d` falls back when `TMPDIR` is unusable; GNU `mktemp` fails.** A test harness pointing `TMPDIR` at a directory it never created passed locally and failed 31 tests on the Linux runner — and worse, the failure surfaced as a *wrong answer*: the temp dir failed inside the timeout watchdog, the forge CLI never ran, the wrapper returned 1, and the concept reported "run could not be read". Check `[ -d "$TMPDIR" ]` explicitly so both platforms agree, and give "could not set up" a different status from "the command failed". ## UI/UX diff --git a/codev/reviews/13-ci-forge-concepts.md b/codev/reviews/13-ci-forge-concepts.md index 705a28ff3..bd991db54 100644 --- a/codev/reviews/13-ci-forge-concepts.md +++ b/codev/reviews/13-ci-forge-concepts.md @@ -130,7 +130,7 @@ Added at the architect's direction at the dev-approval gate, and the reasoning i ## Test Results - `npm run build`: ✓ pass -- `npm test`: ✓ pass — 5572 passed, 0 failed, 48 skipped (5620). **67 new tests** in `pir-13-ci-concepts.test.ts`, plus the two concept-count assertions updated in `forge.test.ts`. +- `npm test`: ✓ pass — 5573 passed, 0 failed, 48 skipped (5621). **68 new tests** in `pir-13-ci-concepts.test.ts`, plus the two concept-count assertions updated in `forge.test.ts`. ### Verification coverage — three tiers, and they are not the same @@ -220,6 +220,28 @@ A third finding was cosmetic (a misindented `exit` and a trailing space in `gite A fourth was noted rather than requested: the `Ci*` contracts in `forge-contracts.ts` are documentation-only, with no conformance test tying them to actual script output. **Deliberately not done here.** Adding it for the four CI contracts alone would leave the other eighteen forge contracts untested while implying they were covered — worse than uniformly untested. The architect is filing it as its own issue across all forge contracts. +### What CI on this PR found that the local suite could not + +The branch's own CI went red while `npm test` was green locally, and the cause is the same platform split the review lane had just warned about — this time with a **wrong answer** at the end of it rather than a crash. + +**The trigger was a test-harness bug**: the harness pointed `TMPDIR` at a directory it never created. macOS `mktemp -d` ignores an unusable `TMPDIR` and falls back to the system temp dir; GNU `mktemp` honours it strictly and fails. So 31 tests passed on the Mac and failed on the Linux runner. + +**What the failure exposed is the part worth reading.** Every CI concept needs a temp dir, and the watchdog in `_timeout.sh` needs one for *every single call*. With `TMPDIR` unusable, `mktemp -d` failed inside the watchdog, `gh` was never run, and the wrapper returned 1 — indistinguishable from the wrapped command failing. The concept then answered: + +```json +{"ok":false,"error":"not-found","detail":"run 32515040122 could not be read (gh exit 1); pass the `id` from ci-runs, not the run `number`"} +``` + +A temp-directory problem wearing the face of a missing run, complete with confident advice about which id to pass. That is the same rule this PR's hot-tier lesson is about, arriving one more time: **"I could not tell" must never be spelled the same way as "no."** + +Three fixes, and the diagnosis was made in an `ubuntu:24.04` container rather than by round-tripping CI: + +1. `forge_timeout` returns **125** when it cannot create a temp dir, not 1 — the command never ran, so it must not share a status with the command failing. +2. Every CI concept runs `ci_require_tmpdir` before it touches a forge, which fails by name with `forge-error` and makes **no forge call at all**. +3. The check tests `[ -d "$TMPDIR" ]` explicitly rather than leaning on `mktemp`, so macOS and Linux behave **identically**. Relying on `mktemp` alone would have preserved the split — and the log cache, which reads `${TMPDIR:-/tmp}` directly, silently does nothing on macOS in that state anyway. + +The harness bug is fixed, and the misdiagnosis is pinned by a test that passes on both platforms. + ### Why this PR is held rather than merged on one lane One of three would be the thinnest coverage of the day, on the diff least suited to it: five POSIX `sh` scripts, a hand-rolled awk extractor and a pile of jq, where the two absent lanes are the ones that most often catch quoting and portability defects — and where two of the three bugs found during implementation were exactly that class. PR #24 (issue #22) adds an **opencode** consult lane on an account unrelated to either exhausted quota; the architect is merging it and re-running this review phase with that lane available, rather than waiting for the Aug 27/28 quota resets or merging thin. **This section is rewritten with the real verdicts once that re-run completes.** diff --git a/packages/codev/scripts/forge/_ci-lib.sh b/packages/codev/scripts/forge/_ci-lib.sh index af39830b0..38f3d3962 100644 --- a/packages/codev/scripts/forge/_ci-lib.sh +++ b/packages/codev/scripts/forge/_ci-lib.sh @@ -145,6 +145,40 @@ ci_require_id() { esac } +# Fail loudly and early if TMPDIR is unusable. +# +# Every CI concept writes logs and windows to a temp dir, and the watchdog in +# _timeout.sh needs one for every single call. macOS mktemp quietly falls back +# to the system temp dir when TMPDIR points nowhere; GNU mktemp does not, so an +# unusable TMPDIR fails on Linux and passes on a Mac. Checked ONCE up front, by +# name, because the alternative is what it did before: the first forge call +# failed for a reason nobody could see, and the concept reported "run could +# not be read" — a temp-dir problem wearing the face of a missing run. +ci_require_tmpdir() { + _concept="$1" + # The explicit directory test comes FIRST, and is the reason this is uniform: + # relying on mktemp alone would keep the platform split, since macOS would + # quietly succeed against a TMPDIR that does not exist while Linux failed. A + # TMPDIR naming somewhere that is not a directory is an error on both, and the + # log cache — which reads ${TMPDIR:-/tmp} directly — silently does nothing on + # macOS in that state anyway. + if [ -n "$TMPDIR" ] && [ ! -d "$TMPDIR" ]; then + _msg="TMPDIR=${TMPDIR} is not a directory; no forge call was made" + jq -cn --arg d "$_msg" '{ok: false, error: "forge-error", detail: $d}' 2>/dev/null \ + || printf '{"ok":false,"error":"forge-error","detail":"%s"}\n' "$_msg" + echo "${_concept}: ${_msg}" >&2 + exit 1 + fi + _probe=$(mktemp -d 2>/dev/null) || { + _msg="TMPDIR=${TMPDIR:-/tmp} is not usable (mktemp -d failed); no forge call was made" + jq -cn --arg d "$_msg" '{ok: false, error: "forge-error", detail: $d}' 2>/dev/null \ + || printf '{"ok":false,"error":"forge-error","detail":"%s"}\n' "$_msg" + echo "${_concept}: ${_msg}" >&2 + exit 1 + } + rmdir "$_probe" 2>/dev/null || : +} + # Assert that a captured payload really is JSON, and emit an envelope if not. # # ci_require_json "" "" diff --git a/packages/codev/scripts/forge/_timeout.sh b/packages/codev/scripts/forge/_timeout.sh index 4ee5f1d82..4dba75463 100644 --- a/packages/codev/scripts/forge/_timeout.sh +++ b/packages/codev/scripts/forge/_timeout.sh @@ -45,7 +45,16 @@ forge_timeout() { # derived from the output file's ("$_tf.fired"), which mktemp does not reserve # — a predictable name in a world-writable tmpdir that anyone could pre-create # to make every call report a timeout. - _dir=$(mktemp -d) || return 1 + # 125, not 1: a temp dir we could not create is NOT the wrapped command + # failing — the command never ran. Returning 1 made an unusable TMPDIR + # indistinguishable from "gh says no such run", and on Linux (where mktemp + # honours TMPDIR strictly, unlike macOS which falls back) that turned into a + # confident "run 32515040122 could not be read". Reproduced in an ubuntu:24.04 + # container before fixing. + _dir=$(mktemp -d 2>/dev/null) || { + echo "forge: could not create a temporary directory under TMPDIR=${TMPDIR:-/tmp}; the command was never run" >&2 + return 125 + } _tf="${_dir}/out" _fired="${_dir}/fired" "$@" >"$_tf" & diff --git a/packages/codev/scripts/forge/gitea/ci-failures.sh b/packages/codev/scripts/forge/gitea/ci-failures.sh index 6f8389aea..8e4ea089c 100755 --- a/packages/codev/scripts/forge/gitea/ci-failures.sh +++ b/packages/codev/scripts/forge/gitea/ci-failures.sh @@ -22,6 +22,8 @@ set -e CONCEPT=ci-failures +ci_require_tmpdir "$CONCEPT" + if [ -z "$CODEV_CI_RUN_ID" ]; then jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 diff --git a/packages/codev/scripts/forge/gitea/ci-run-log.sh b/packages/codev/scripts/forge/gitea/ci-run-log.sh index 3735e7af1..a8c8c6dc8 100755 --- a/packages/codev/scripts/forge/gitea/ci-run-log.sh +++ b/packages/codev/scripts/forge/gitea/ci-run-log.sh @@ -19,6 +19,8 @@ set -e CONCEPT=ci-run-log +ci_require_tmpdir "$CONCEPT" + if [ -z "$CODEV_CI_RUN_ID" ]; then jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 diff --git a/packages/codev/scripts/forge/gitea/ci-run-view.sh b/packages/codev/scripts/forge/gitea/ci-run-view.sh index ee5739a00..c78febbc5 100755 --- a/packages/codev/scripts/forge/gitea/ci-run-view.sh +++ b/packages/codev/scripts/forge/gitea/ci-run-view.sh @@ -24,6 +24,8 @@ set -e CONCEPT=ci-run-view +ci_require_tmpdir "$CONCEPT" + if [ -z "$CODEV_CI_RUN_ID" ]; then jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 diff --git a/packages/codev/scripts/forge/gitea/ci-runs.sh b/packages/codev/scripts/forge/gitea/ci-runs.sh index d82b90b4f..ddbdff885 100755 --- a/packages/codev/scripts/forge/gitea/ci-runs.sh +++ b/packages/codev/scripts/forge/gitea/ci-runs.sh @@ -21,6 +21,8 @@ set -e CONCEPT=ci-runs +ci_require_tmpdir "$CONCEPT" + ci_check_status "$CONCEPT" "$CODEV_CI_STATUS" LIMIT=${CODEV_CI_LIMIT:-$CI_LIMIT_DEFAULT} diff --git a/packages/codev/scripts/forge/github/ci-failures.sh b/packages/codev/scripts/forge/github/ci-failures.sh index 7f908099a..e52244472 100755 --- a/packages/codev/scripts/forge/github/ci-failures.sh +++ b/packages/codev/scripts/forge/github/ci-failures.sh @@ -30,6 +30,8 @@ set -e CONCEPT=ci-failures +ci_require_tmpdir "$CONCEPT" + if [ -z "$CODEV_CI_RUN_ID" ]; then jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 diff --git a/packages/codev/scripts/forge/github/ci-run-log.sh b/packages/codev/scripts/forge/github/ci-run-log.sh index d45b594cf..2d5592bd3 100755 --- a/packages/codev/scripts/forge/github/ci-run-log.sh +++ b/packages/codev/scripts/forge/github/ci-run-log.sh @@ -28,6 +28,8 @@ set -e CONCEPT=ci-run-log +ci_require_tmpdir "$CONCEPT" + if [ -z "$CODEV_CI_RUN_ID" ]; then jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 diff --git a/packages/codev/scripts/forge/github/ci-run-view.sh b/packages/codev/scripts/forge/github/ci-run-view.sh index 115161b3e..d778c4138 100755 --- a/packages/codev/scripts/forge/github/ci-run-view.sh +++ b/packages/codev/scripts/forge/github/ci-run-view.sh @@ -19,6 +19,8 @@ set -e CONCEPT=ci-run-view +ci_require_tmpdir "$CONCEPT" + if [ -z "$CODEV_CI_RUN_ID" ]; then jq -cn '{ok: false, error: "bad-input", detail: "CODEV_CI_RUN_ID is required"}' echo "${CONCEPT}: CODEV_CI_RUN_ID is required" >&2 diff --git a/packages/codev/scripts/forge/github/ci-runs.sh b/packages/codev/scripts/forge/github/ci-runs.sh index 2de456c65..457172aaa 100755 --- a/packages/codev/scripts/forge/github/ci-runs.sh +++ b/packages/codev/scripts/forge/github/ci-runs.sh @@ -26,6 +26,8 @@ set -e CONCEPT=ci-runs +ci_require_tmpdir "$CONCEPT" + ci_check_status "$CONCEPT" "$CODEV_CI_STATUS" LIMIT=${CODEV_CI_LIMIT:-$CI_LIMIT_DEFAULT} diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts index 24713597c..8f28620c0 100644 --- a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -77,6 +77,12 @@ let tmp: string; beforeEach(() => { tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'pir13-')); + // The scripts run with TMPDIR pointed here so the log cache is per-test. It + // has to EXIST: macOS mktemp falls back to the system temp dir when TMPDIR + // points nowhere, GNU mktemp does not — so a missing directory passed locally + // and failed 31 tests on the Linux runner, with the concept reporting + // "run could not be read" for what was really an unusable TMPDIR. + fs.mkdirSync(path.join(tmp, 'cache'), { recursive: true }); }); afterEach(() => { @@ -444,6 +450,23 @@ describe.skipIf(!hasJq())('#13 — ci-failures returns a bounded extract with it expect(fileLines(path.join(tmp, 'gh.log'))).toEqual([]); }); + it('names an unusable TMPDIR instead of blaming the run', () => { + // The failure this cost a red CI run to find. mktemp -d fails on Linux when + // TMPDIR points nowhere; the watchdog needs one for EVERY call, so the + // first gh invocation died for an invisible reason and the concept answered + // "run 32515040122 could not be read" — a temp-dir problem wearing the face + // of a missing run. macOS hides it by falling back to the system temp dir. + stubGh(ONE_FAILING_JOB); + const r = run(githubDir, 'ci-failures.sh', { + CODEV_CI_RUN_ID: '32515040122', + TMPDIR: path.join(tmp, 'no-such-directory'), + }); + expect(r.status).not.toBe(0); + expect(r.json!.error, 'an unusable TMPDIR was reported as a missing run').toBe('forge-error'); + expect(r.json!.detail).toContain('TMPDIR'); + expect(fileLines(path.join(tmp, 'gh.log')), 'the forge was called despite a broken TMPDIR').toEqual([]); + }); + it('emits an envelope when the forge CLI exits 0 with something that is not JSON', () => { // An auth prompt, an empty body, an HTML error page. Without a guard this // reaches jq, dies under `set -e`, and leaves jq's diagnostic on stderr From b1f8c7fcef1c68c4c13545b9522d23315847ad7b Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 17:34:00 -0600 Subject: [PATCH 27/30] [PIR #13] fix(forge): the extractor pointed at a passing test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by running ci-failures against this repository's own red CI run while this PR was open. It returned Error: Refusing to POST /api/tunnel/disconnect to the default Tower port ... from inside a `stderr |` capture block belonging to a test that PASSES — the test asserts that error is thrown, so the text is the suite working as designed — while the real failure, a vitest worker crash, sat 355 lines further down. Same decoy class as the artifact-canvas one, arriving through a different door: this decoy IS anchored at the start of its line, so the anchoring rule that killed the first one does nothing here. Two fixes: - rung 6 skips lines inside vitest stdout|/stderr| capture blocks (header to the blank line that ends the block); - a new rung 4 recognises the Unhandled Errors banner — a worker death, an unhandled rejection, a teardown crash. No test reports as failed, the rungs above find nothing, and the run is still red. That CI log is now a third gzipped fixture. Both original fixtures extract identically, and the review's CI claim is corrected: the branch was red, no test failed, and test.yml's tolerance guard cannot fire because `grep -q "failed" ` matches the guard's own echoed script and ordinary test names. Co-Authored-By: Claude Opus 5 --- codev/resources/lessons-learned.md | 1 + codev/reviews/13-ci-forge-concepts.md | 55 +++++++++++++++--- packages/codev/scripts/forge/_ci-extract.sh | 39 ++++++++++++- .../pir-13/github-vitest-worker-crash.log.gz | Bin 0 -> 42477 bytes .../src/__tests__/pir-13-ci-concepts.test.ts | 38 ++++++++++++ 5 files changed, 121 insertions(+), 12 deletions(-) create mode 100644 packages/codev/src/__tests__/fixtures/pir-13/github-vitest-worker-crash.log.gz diff --git a/codev/resources/lessons-learned.md b/codev/resources/lessons-learned.md index 146ab6b3a..be0c84b78 100644 --- a/codev/resources/lessons-learned.md +++ b/codev/resources/lessons-learned.md @@ -467,6 +467,7 @@ so it survives review. Pin the constant to the highest migration block in a test - [From #13] **A test that overrides the defaults cannot detect the defaults being wrong.** The timeout test forced `CODEV_FORGE_TIMEOUT=2` and passed, proving the watchdog fires when told to — while at the real defaults the outer 30s ceiling killed the command before the inner 60s watchdog could name the endpoint, so the whole point of the inner watchdog never fired. Where two ceilings, retries or precedence rules compose, pin the ORDERING at the values that actually ship. - [From #13] **`sed -n "1,0p"` is tolerated by BSD sed and rejected by GNU sed.** An empty input built that reversed range, which was invisible on the macOS box it was written on and would have aborted the script under `set -e` on Linux, where CI runs. Guard the empty case explicitly rather than trusting a range expression to degrade. - [From #13] **macOS `mktemp -d` falls back when `TMPDIR` is unusable; GNU `mktemp` fails.** A test harness pointing `TMPDIR` at a directory it never created passed locally and failed 31 tests on the Linux runner — and worse, the failure surfaced as a *wrong answer*: the temp dir failed inside the timeout watchdog, the forge CLI never ran, the wrapper returned 1, and the concept reported "run could not be read". Check `[ -d "$TMPDIR" ]` explicitly so both platforms agree, and give "could not set up" a different status from "the command failed". +- [From #13] **A CI tolerance guard that greps the whole log for a word can never fire.** `test.yml` tolerates a known vitest worker-teardown crash only when the output contains no "failed" — but the runner echoes the guard's own script into the log, and ordinary test names (`clear-failed`, "reports a failed Tower send") match too. The escape hatch had never once been reachable, so every worker crash was a hard failure. Scope such a check to the summary line it means, not to the whole transcript. ## UI/UX diff --git a/codev/reviews/13-ci-forge-concepts.md b/codev/reviews/13-ci-forge-concepts.md index bd991db54..67f71f04f 100644 --- a/codev/reviews/13-ci-forge-concepts.md +++ b/codev/reviews/13-ci-forge-concepts.md @@ -130,7 +130,39 @@ Added at the architect's direction at the dev-approval gate, and the reasoning i ## Test Results - `npm run build`: ✓ pass -- `npm test`: ✓ pass — 5573 passed, 0 failed, 48 skipped (5621). **68 new tests** in `pir-13-ci-concepts.test.ts`, plus the two concept-count assertions updated in `forge.test.ts`. +- `npm test`: ✓ pass — 5622 passed, 0 failed, 48 skipped (5670), after merging `origin/main`. **70 new tests** in `pir-13-ci-concepts.test.ts`, plus the two concept-count assertions updated in `forge.test.ts`. + +### Branch CI: red, and NOT because of a failing test + +**Read this before reading `npm test: ✓ pass` above.** That line is the local suite and it is true; the branch's own CI is a separate claim, and at the time of writing it was **red**. The claude review lane caught the review presenting it as resolved when it was not. Here is what it actually is. + +The last red run (`32536232930`) reports: + +``` +Test Files 280 passed | 3 skipped (284) +⎯⎯ Unhandled Errors ⎯⎯ +Error: [vitest-pool]: Worker forks emitted error. +Caused by: Error: Worker exited unexpectedly +``` + +**No test failed.** A vitest worker fork died during teardown, so one file went unreported. `.github/workflows/test.yml` already knows about this and tries to tolerate it — its own comment says *"Vitest forks pool has a known issue where the worker process crashes during cleanup after all tests pass"* — with: + +```sh +if grep -q "Test Files.*passed" /tmp/vitest-output.txt && ! grep -q "failed" /tmp/vitest-output.txt; then + echo "::warning::Vitest worker crashed during cleanup but all tests passed" +``` + +**That guard cannot fire.** `grep -q "failed"` runs over the whole captured output, and "failed" appears in it six times on this run — none of them a failing test: + +| line | what it is | +|---|---| +| 1475 | **the guard's own script**, echoed into the log by the Actions runner | +| 1657, 1663, 1664 | `spec-1470` test names and stdout — `reentry-failed`, `clear-failed`, "reports a **failed** Tower send" | +| 2202, 2205 | `git fetch … failed` warnings printed by consult tests | + +So *any* worker crash in this repository is a hard CI failure regardless of the test results, and the tolerance the workflow author wrote has never been reachable. That is a defect in `test.yml`, not in this diff, and it is **not fixed here** — fixing another team's CI gate to turn an unrelated red green is the scope creep the review phase warns against. It is reported to the architect with this evidence. + +What this PR *did* cause, and has fixed, is the earlier red: the `TMPDIR` harness bug above, which failed 31 tests on Linux while passing locally. ### Verification coverage — three tiers, and they are not the same @@ -290,22 +322,27 @@ The architect counted this as the seventh arrival of the same rule in one day. T ## How to Test Locally - **View diff**: VSCode sidebar → right-click builder `pir-13` → **Review Diff** -- **The headline path** (needs `gh` authenticated against this repo): +- **Build the branch first, and invoke ITS cli** — the globally installed `codev` predates this + PR and has no `forge` subcommand (`error: unknown command 'forge'`): ```bash - CODEV_CI_RUN_ID=32515040122 codev forge ci-failures | jq + pnpm --filter @cluesmith/codev build + CODEV_CI_RUN_ID=32515040122 node packages/codev/dist/cli.js forge ci-failures | jq # 23 lines out of 2528: the AssertionError, the test file and line, the step name ``` + Substitute `node packages/codev/dist/cli.js` for `codev` in every command below. After this + merges and you reinstall globally, plain `codev forge …` works. - **The windows**, and that the second call is free (cached): ```bash - CODEV_CI_RUN_ID=32515040122 CODEV_CI_LOG_GREP=AssertionError codev forge ci-run-log | jq '{from,to,matches,matchLines}' - CODEV_CI_RUN_ID=32515040122 codev forge ci-run-log # refuses: no window + CODEV_CI_RUN_ID=32515040122 CODEV_CI_LOG_GREP=AssertionError node packages/codev/dist/cli.js forge ci-run-log | jq '{from,to,matches,matchLines}' + CODEV_CI_RUN_ID=32515040122 node packages/codev/dist/cli.js forge ci-run-log # refuses: no window ``` - **Loud degradation**, from a Forgejo repo (`~/dev/entriq`): ```bash - CODEV_CI_RUN_ID=11130 codev forge ci-run-view | jq '{jobSource, jobs: (.jobs|length)}' # works - CODEV_CI_RUN_ID=11130 codev forge ci-failures | jq '{error, serverVersion, needs}' # unsupported-server - codev forge team-activity # named, exit 3 - codev forge ci-failure # unknown, lists valid, exit 2 + CLI=/path/to/this/worktree/packages/codev/dist/cli.js + CODEV_CI_RUN_ID=11130 node $CLI forge ci-run-view | jq '{jobSource, jobs: (.jobs|length)}' # works + CODEV_CI_RUN_ID=11130 node $CLI forge ci-failures | jq '{error, serverVersion, needs}' # unsupported-server + node $CLI forge team-activity # named, exit 3 + node $CLI forge ci-failure # unknown, lists valid, exit 2 ``` - **`codev doctor`** in both repos: four new concepts, `gh` under github, `tea` under gitea. diff --git a/packages/codev/scripts/forge/_ci-extract.sh b/packages/codev/scripts/forge/_ci-extract.sh index e7a403658..baeb75329 100644 --- a/packages/codev/scripts/forge/_ci-extract.sh +++ b/packages/codev/scripts/forge/_ci-extract.sh @@ -93,6 +93,8 @@ ci_extract() { last_failed_banner = 0 # vitest "⎯⎯ Failed Tests N ⎯⎯" first_ts_error = 0 first_go_fail = 0 + first_unhandled = 0 + in_capture = 0 for (i = 1; i <= n; i++) { s = line[i] if (!first_marker && s ~ /^##\[error\]/ && !is_noise_marker(s)) first_marker = i @@ -104,6 +106,20 @@ ci_extract() { if (s ~ /^[0-9]+ (passing|tests? passed)/) last_pass_boundary = i if (!first_ts_error && s ~ /error TS[0-9]+:/) first_ts_error = i if (!first_go_fail && s ~ /^ *--- FAIL: /) first_go_fail = i + if (!first_unhandled && s ~ /Unhandled Error/) first_unhandled = i + # Vitest prints a PASSING tests captured output as a block: + # stderr | path.test.ts > suite > name + # + # + # Those lines are a test working as designed, and rung 6 must not anchor + # on one. Marking the region is the only way to tell them apart: the + # captured text is often a perfectly formed "Error: ..." at the start of + # its line, which is exactly what rung 6 looks for. + if (s ~ /^(stdout|stderr) \| /) { in_capture = 1; captured[i] = 1 } + else if (in_capture) { + if (s ~ /^[ \t]*$/) in_capture = 0 + else captured[i] = 1 + } } # ---- rung 1: a recognised test runner -------------------------------- @@ -151,7 +167,18 @@ ci_extract() { exit 0 } - # ---- rung 4: the runner error marker --------------------------------- + # ---- rung 4: vitest unhandled errors --------------------------------- + # A worker that dies, an unhandled rejection, a native teardown crash. No + # test reports as failed, so the rungs above find nothing, and the run is + # still red. Below the Failed Tests rung because a real assertion beats a + # teardown crash when both are present. + if (first_unhandled) { + to = first_unhandled + 12; if (to > n) to = n + emit("vitest-unhandled", first_unhandled, to) + exit 0 + } + + # ---- rung 5: the runner error marker --------------------------------- # GitHub puts the message INSIDE the marker (##[error]AssertionError: # expected null to be unauth), so the marker line is itself the answer and # what FOLLOWS it is the useful context. What precedes it, on the log this @@ -166,7 +193,7 @@ ci_extract() { exit 0 } - # ---- rung 5: the first ANCHORED error, preferring after the last pass - + # ---- rung 6: the first ANCHORED error, preferring after the last pass - # Anchoring at the start of the line is what makes this safe: the false # positive this ladder was built against ("[artifact-canvas] Error: host # blew up", printed by a passing test) has its "Error:" mid-line and @@ -183,6 +210,12 @@ ci_extract() { s ~ /^[A-Za-z_][A-Za-z0-9_.]*(Error|Exception): / || s ~ /^error(\[[A-Z0-9]+\])?: /) { if (is_noise_marker(s)) continue + # A passing tests captured stdout/stderr is not a diagnosis. Found by + # running ci-failures against this repos own red CI run: it returned + # `Error: Refusing to POST /api/tunnel/disconnect ...` from inside a + # `stderr |` block belonging to a test that PASSED, while the real + # failure sat 350 lines further down. + if (i in captured) continue from = i - 3; if (from < 1) from = 1 to = i + 3; if (to > n) to = n emit("first-error", from, to) @@ -191,7 +224,7 @@ ci_extract() { } } - # ---- rung 6: give up honestly ---------------------------------------- + # ---- rung 7: give up honestly ---------------------------------------- exit 1 } function emit(rung, from, to, i) { diff --git a/packages/codev/src/__tests__/fixtures/pir-13/github-vitest-worker-crash.log.gz b/packages/codev/src/__tests__/fixtures/pir-13/github-vitest-worker-crash.log.gz new file mode 100644 index 0000000000000000000000000000000000000000..bee9a6801c7174f1090e37d649a5a6879ea0b154 GIT binary patch literal 42477 zcmV)OK(@ahiwFo{-iT@f19D+^Ep};iWpi{bcW-iQWpXWJa$$35E^KdS0PMZ%mK#Ts zCir{j73xn6)mc9`51eJM?!-ThDX&AnmXeV=^t z$qy$*zMTK~Jhp!_O;57qS&^TOv7fA3DIN3`Uk~Im%2X^szB2bH>X7^sLnJNE|aJwH|Wj zsouCU{`=(f<#;ko9;b^e($HhJC;Ut954k;L;(MyKa2&}0E-&8TGCe*_{>hKn=y<3- z)Etg_YBciWo$V77pgnO6%h0A=Vg&4&$E-TBw1Q6mZhMYXkLA>Y&MJ^ zjj9PBM_>82Pv=>YF2=KyOoZ_79PB+>M+%No~6TS)}IauvIOT!R+9%@O#o8ckW|DL<-sC7S(Em+;tyAeqehk- z(bM9a3a$9nZ~DcPmtVg6{PoMHe}D8%@+>Q+;}SnaNj3KjMbD#rwg72U9wr5>2~k%v z&fwF2oh{N~x=5Sra;1Q4o~45~xE9TILF)Kcla&Sp_4mn(Y*1v2q|BGa06xgWxcXEZ z5Nts8vm%=VqLOovIipE_o(#uDHdy4vMQuCDEYnI{wo4}yn8QCVq17-+8%Bhgdud@! zzsMFrOvK8`Fh8G7@^qMtXQRBfrE(y38LZmlRn@8~kdT*Sq&2y_g4*CT8@$Pvi{GEA zyUD#5Up_qK9K@$#sT9U)Hq=f!W?d%DU?knK)9J`qD~z~bJ2)WH2o&i}gU@I471(=a zwph+DM?VH2rK8bcka65i(;-jO(NLwwEb}NseC9{m9rOCpomI-AY-%(0Fmi8}53|eR z0YOq`sTR%+(rl1Aqr{MBPNlG`$dMfyCm0i~Zg|^=-J3CG!tOYqWU%@EIX_Nj=@iiQ zYBpXZFSBw{);8r{8i5k;8=wcIT{13{;|#z1Fq;FyU~3%qlb5IP^BQ{a-zX`t(m4o` zs;NqjFQCyVT}~GL!>_)2`rYeSFOFWn`sT&aqgT(5UVroS@zLvVkDkAP3ZE^CWtL>KvvHBn zrl|9rLFei5BD?mF;=4G}V z<}l;o!2l>fBid?Z49B*M{7nX1-EV&L8>%Mka1v{l)tEf+lyjzjpO<8)Fsk7BGd*l+;*)U3Ix1+f+g0%2HYz17PO^;+tkS<{V8|#i5^xv2m<><_ZH;{B!8vhF>8Q54FF;gQMiaHoVRNuWusN##K~S!i z2Lk3ks^7{&PvDs?x>N4o(|S>>d^!U0b}O8|j8{&_XT<#G}|cpt<>{c4O+>DXbm z8kAiJbe)c8ATC~&NX^|TSi)Ix7rzuzhH6a$xCwuZBi+eykuCSSy092DuBE1TAomy5wEN5$Mp%sEnWry!yhZ zsvaYE-~@-s**J|Kq~d}@YAAjQYpRM2&Xg_-l-hrUU-?{R2Q6Eo-XX8P3{nLIExDWU7JJevHk#cl~R1yW23B>Q>BCydFDGlgv zln@7fAm@M>gma0jRWE&tY?^}&gaQ~C=>!I<@p-gU6+lDgmbKX~jSTT#}&?lT@U{?hwUqizr=5p^)4-K_&qoa zKI5yM=(k|!X55XqKlWBxZ&JrbeyBMcxlA2f#)rpgHjwVH$VTXc$4^!s%$JkNfshB& ztf)QfaDD5U5xC3$p~$B@Zg|)W7g|9t$#0S$qCz_QF0}(5Hy5EXY;YTCQA*G+>`> z7EV6;s2DAZEPE7c*N)>Dhp%?kpnHVm;p!lXksC%-Q{W*+lDL{?Z|#_Z{{*=-2wk>E zsPA(Dx~eefYR2={5_HarS@YyB-Phed*_)SuRH_t}neUT7=F4Cf0PBH@zW`wh`skNO z4WE4HO4mJwSarM_)23n9yUi7J!E1iz`ce zoDSaL`lc%jnHoTb^Ki1%qSjUU^v~1T0{e`n5p<1mqB2(C=K@w4W^;9hvF3wnwinbW zV2i^j!#pYT`VuX2zyRgwjU$DHrb?|= zFeBtpbf+Q;(#2}yNxnvbkmwYcTbQ9?L(6>XGUGg#1IM&*TB|{NJOGPZX<#a##efQO ztQQFg;W;@jz&pm5amV zzg9=aI0)Ioi*RTIw7*8z7&`ru8SJA}0~Po^f$s#2d)UOWP*-cVuKOC+5m+Ic4K7~K zQ#=Smx=@9A)4xck^nl*QW4;(u@%qnYK6^bHgS=k-)W@Hw=32@io?&XUVLHiYn~2s% zwc%Uz>+AQzijWP;Y%=OqYnc`Oe*NBQEq6`;Y4;FNhk|>3Kl$PD(K7}#{Fpqx1l@Ph zX?vLb1g2L%Xp#5s{{)PAnx2gT-a*_gstFdu;5kG60raj}@JMGxkr(Iz28jr5U;{*h zW4yRPRbVoXqm=#RRT;h0-+lS;<#%5subzLEJbdbsOmD?9+d z4D$Icr)nCG==<&a#hmp)r$txtRRmT|GB=AQ^54*KlZ&3z3;`fwT?L=nB`hb6V={v^iL2QUsXk_ zu+s!0&n|;7$@(rhD^|N${SFzcdKRNH(#d&xQTEWGP3|C73#o!~9c@&3v2Q0n`d9Y#^d(G>jx&6O2Gjx*XzpXbkGN#lk8I)+k*RG zNAnJAK3pK!7nPO1jsX;`c00tD-dTs=u-Xm?>yeP)=1kIajr9PYGz2)o;m~XS*o9qIl z?GOqveB>My7Fhd72^BhG2Wsf`(Dm#qSR?pd#f+6+=<^l{P~k?nC0IJQII zv*`%hE5>^sn)Mt87)fOM33z#R;Y1;h%d{B;*YU?5Q0l}3kt`~E}M2WOnsc4)wMhY$39@+f@7 zQ!vC~7d?#IOZc$8uU7mLM(BYxz%nLA#|H0gz+Xe|Q=c7+p=HiF&n?*J!xV%`%3L~< z`uJGef#pLOO1q)2Cqv`eu&fb^fAuy3H+^G4O5wTk^K6ifQ7abk4=A==VDyI2pcsHj z?2~)W_Ql`ce~<_iJIU9dqh2wn?b8A`hszC@eXhH?-X{Q{&tj)w7H;a)a@-^$T3*2F zg7GbsgjUJD%RWE9Pc#RP;+ztcy|=W?8h{$ZJ+r-IP&;~RWECi7Ao#3gnaPaS12sAb zca33yBa8Z+7?(R-gxljRILs5xH8)zQp@p|zSnG(+YZCgpN4 zfG$SM$>gHGp^-93&UYGy%?`wW#|-B&Ko+K=Yn(JmWzbX2xWwK2S3gMeDW6_#@T(u# zw6+mwY0;Nbo@VHDS%3876pzNUdb*a2`U!*WQqwW-gA(oq8XmiL94gg&EI`%ZPVjJ>q_$K6LkAam0r(iP0&)X}9Z+P;LW-*vLuO9tzmRg4H3mg+gp7nJ1wTYk&pu0ih84^qVh8 zLQQW`U#FHAO0_{7s0Ioca<>Z>upKp+(Xs&yv5J-T)EW>Nj0at+Js3hG&?J0&Fqm)w zL}8%Ac3=n%C3mFuQPnW31gQV`2)@d5*$y3~MSaWT)zUCC4`gJl-j;QHCFLDI7IJa82fNhe@`=Z0Q!`^4vqJk5-@@0VEu5sfUq4* zVnfxieZ(5pen3xU=u@3|$ zFul_SMMB(g5O1it1LPvidO?RTlISO=8Vg}8F+0#1w6_r(YWkP|>;Hy-`)T~E8taUq zr_sfNd48Uu`G0(Yb=;@oQ=#>zT$oRVoEG_H5}WWo@9WT_epF+e#nYYZqgD|}F>v5l zKd9+$Yj_X|?^m1%rp2#*5Nb*+faOe8`uY`8ve^PJorWLFVsJ2?4YRj>l)~hkESF5X zcRN}Mgu~UER#c%-XzENjA>ZpP=ClE;r>JqVO5PVzDv9yk?|n@<{o&F7@#4qT-(or( zmx}}V4>A4K4}2P@HwY^wrZFNIYb9EN36{M1JpPV9{pH{Pdvfygcz&3mUyQHHD0&PR zY&&|^wDhOxJd(k|MpJ?yL-D@yo5-;2eYm|#h6u`IuQvwdfw=lV{aRv|QC z`?(GKgTC$`^xGh72(DLpkV0VoI?r_U09HQq~H;AC6vpy@6e(*Vtvn zFz9ul8HR5_Dr`hr+wCY?QxhSzVi^IdRs+khO@yFJXBR?-{pmGyH^I9FzClH`Tx|P| z-of{@9^Xk5z*`_3l)#W0rkOVNF2cR-3tFyt1-5c~hzr&PaSNg^MBP%Q&TG4^ZW9&7 z52oQnLcB3S!wF8pm5VS8RPP+DS~+w&)|NVejYH!l(A=?q7;(P_^)v=@lVePmTiss7>T7&_?MECS%~!O6X8D z$p)H)YDE)AziS94YfBSB40lZnBy}rl0C9!|k4+ObV+?X!^p5dDR9ajR1gS=P+L0rx zn|T}X14hv-gDqm7PsRg^9?+fk{#r_DX4t0n!6Ynb%)%!tmj&zz89|cSe~B>TkwN3W zJ*Hs;8WIHACW1^mf?yL1-p)pX)HgmB zoHIjp4@J!bcyHu-fEVamz*|{yfYmNRv=x}+bKUmwvbysY+k)0F%20G~QKYjHL!1-k z_!Jc3phF}K|1*lB#KO*p8giJO&=UW+8Z`~g?o*0*AVr1qquNp{xE73Wp_Rop+WOX7 z8Mc7E3H7+`w6fUHq1HoaWub-;`~$k-t3I{gn8;Ov&H*CRnfk>IrJy3PO-FazsAcsh zcX2GtN4-%m!j8sn3r6%MxzG=M-=Si_ZqkL3-$0Q(s)0T=c`8((tZAmba;{Q-;!~K>GB3e)~RNQYI8Zfz8L~E*Z-PpNh z%8L!+Sy64JiH^ZK@hqcj?7Dd7T-J+c9wl-V&z!$2omqo)MoOHg%sMD_C|~pz(mA%# znzxqD*ur~hwwBJZp|NCOMODLl^8uwZ^yxu-*^T)#qy`Ne2IsXgHdv|{=Xm?SwS#9H zLA}|t6DSZNuhRyZ9cwD#h?yA6to58M`Hll^JS)dTywejtJKnFsfb;Vr z#X!)IA#t#{*YEf5_leQaJ*mkRgnb9)?suozER29j6BGPgF+zTj!eq+ic?P3Y*)<}i zCEfs9PKKE5?~)iyq*0M`NDat^qCcqPK{dfn?%hv5`(5%+R7IiVZRvyGB%^VWC9wFI z@H0?dEoP+mHwRP~z$-BIf4ToDRRQ=O&;=vlrUL>zpM@&{`EoH|E*d&kL4~DCwL_sB zE?H(Xu#B^c>Wh4N7q3&ruMkFBUu(cW@KRTD9u~dKs)Q&|rO3~rpr3q|rNt~>g~Y23 z)eS04kA=x6&kiQz*_#9WR575Uht~}8W+O$WW%aRJPzr{vbjozic<1TlO_?0yMfwu& zTAasZF~g9=<-t1bOMqj@`VudhX2aSPxi}9aSWw?PVMRPR&bcdWlL6scJaCbRVoZ-9lhPG2xKaS65}SJ z>F~BBqZi7Ai*TJm69#m)fgCG%8)!mB7q3h20ZkVSlLlydsoXBmgbhfOQ&gPe)yLZo zO;9y49{EP0sW1?iUMLk3-F62};6-6*a$>sD&Cpba#a9G&hbD}Kcb{lzDjQ;Qw}z&& zjyJ4$ydZPip@~NnikO690y6@3t-$h2Dj4s}?X9MH5?D`3Il1p-Q39-+pM~eO7V9(DW);_*AJ5w|2UNrZ-A5nw6}r!K+{V}S+ahrBVG&vJNgxn-FIg`wIdZDB8Qhyf&CZ_~rUUsE$E9IP~{8dH)Kd>&h4zL5gA-4|AV#0@w z{He89a=)zuH&xN3)fmAo^4YfhX~3FwlsRt<9*MbI&!5IgAxYKDa+m}EWo%va%%hur zZS~GE%r#$@wdQYS`x4Ut5HQ~rCxU_JXpx6jn(xxSaKc-mNpd6$+sH;vbTa1R*n<-R zYNF}b3@doEy{{hHNo5^9MP=pf=R_w#WZ0Wk;+(XKH(_5?Yd*1W11C+SnIf znJPMOXJ~rk1)2{eG|^1h;TXxH0b!^Whf*6JM#xl2lU3#0D zwA@PJc3@gCqXjLOIM_wFkO)4(8AR`onN z$%zRF=40G;Cp;*)LQR&-p*-OvELk;oX&KOh1y>p}IV4lSD3Vc3iJcG%>UV^6+ z;ei@ho6GSYROc1fidVK({ro`N^t!X%15{r|o84ackg~Qj7WEonPf85$VtUDqEQ-O> zcCV8Q=t2nuA<3dp!Kxj6aNcRb8>VQ2{(Q?>)Pp|eDHe4M3|4LaYFX5A+2Jw2xC6)?fh3x!JUXjhS6u)ti;-hToWYwUIuSE{>IO%4yL;6_fwSDM^jVZ_*l<4v3>tOz`_Ehj=dVFW!z5nkH3 z<(w#Ne9q}cp@}yzCB0C}VvK!<(A3OXnj9UHQ?!7lR*vlnO&zSv251_F!D8D(6YOV3 zPti&#KLThv&(uvr(}!%|M5$w}%VQZF&`8%nj$^H$2?9@;{h;Z%<&DsEf-|!NH1R%> zr>7``8|1b_6YMMFZy1_ji+OrO$D2u_Q)n9HX_d{H@NjGkXxi{J^j^@k;X&I5Xxfl< zcYA0$K?YW!c^`!JZHK1k9K(Qb6q@){BP}((2%ArL(DcGwLbQRVm)h)S^cD{b?DB90hQK_CP$f8 z(3DJbvsd~D=mM|gHbT>A&US&Oam>r;P3xN4hu9*b6qC?)YN2CtN!onH>}0K+M~3Ttp$Whfjebod6yL=RGQv0-nxNsT z+kb{G4eYk$W~(_5cc;4zPaU7ka$KzthKe6!md|=Mv8qK13xjKx&XOYgb2bR;IbKk6 z0cl~oic!_ldD?lZ4Qy3i`f|0$j6W1P26i>u#`H7V8@`{z4DS=_rf4gC?#OI^0TOFDfK`IeH@Uce*8`Som{C)--07~k)Zk-+nqF>aM289E zU_dYE!$5=QD7+1q;tj3+HcjI&^HdX;;_H7qY?{!nNf5c1C*ZboDaPKAdbfr3_wKKn zODza-+trH(V8q1RB}6IS)a>y7GQJA$X%9cXw9)2rqyxgkI}LkqsdJ3GCO0Z9jojtF z2TXy}EAGMU{0JNZ9&BUs+c-XE@muDE>I!0y(#8U3sc;meF;fcyUm=C(JvXS|KJN*u!T$pg5zBb?; zH~RJ~vOdIC+^EaLU7zWvbD%5!zY(A22D(F=tSA9V^%blfzMMZh;6X4RMi0_5a-X)+Z+&va{RH zVw{DOnj3{CXNHkFt>9cR-X%0K&vX+xAuMz=G`W!bMcrc;PBcQ3OYV1pCYOqQJPS&d zzWrI@fJ_~3Ue}jO&pG#9OaFWh>$Q49d7k5PR9|GUEDFr%8`8hkldKr*S6$_=_ewfX zzNGCHSR|&y;|3VeF|KHGFet4B*C~ye4EEwW=py8CBe_oD6Wu%LT+q&t(Z5j2oxA-y zmogZddb1mXGK$j@7)qV*Fsj6B-th9F*O*78&4!@y_Sk+K0(9Y*RjW#I^U0&~z46 z_3feQxg^zSt(auu_Cr&F#ziVz&T_?N*Sbo%Jf4oL2#Li7B5qiuqeXNih4*=qd0M=I z-{A`duDPn$5HrDJy{;lAJSZ@XydR6DPIkFZI4=zCHo|BF*+zwNcv;);#f;9!4U}X? zs9<);_>RfYgH}zIehlfPF-9~v)ov1+UN}$Nv;YOwWiXG2&E1m^n0hD*nn^P>U7s-y z5Z)u60J|_gqynyoCRD)7x?8^vh3$;9M6?=~S{6d_Z!n22?w_N{l ze>Gb+pI94hVX(A|WWf3^_11iVCP`_X!*iU8s;zhZGf1!EOB* zKxn6hYnY-@_V({$(&M`$H@k}|W*G4>L@e!4_YVVFO+-S)QoA`-PpF8Q!uEou0zHS0 zmW~y)@!LZa+Iiy04of*+JiGPK#7C!Z^gJt!A;YUARH+K-e7l1thQ-mG0*2ydA5p9r z?mq7WO>QJoLPHajcqMj#Cbxpq+{ayTzTA3fqESvw0)ytX?OMSaq_aUbDU)=T;L+CQ zsTQ_5FUE`baO^3p9zixpVxzm#rqFjn z`*0q1Vd#Xcp@In4O1I=ZY$uJSr+E95{|7SF&88L%W9aZi2U+U7%dj2>G+H)%H)A%3 zxP^+A?{jbvyKqc2LepvMc7dknj-Dd?s`TxLrr^QGx>0Bfp}eLSN=om$gr)#hL($|Y zxj9rCD{3AhBlm%(=GLnwXlk*2sB~8PT_Dh|vRSHvb9zY9g5D(dK zJ*V@<#e-y;E(WKoHGGp@9Or28Ex z@ox8GMgvxzYB=h~3fsT97urchZ-oZmAQHEp84YTaH>|>d=FiEn2Q2OI?Wi!IXPVLE zcaKoHkg~@hE*6$Mk+YHP_R^;N=**R zB@In(!$sv5(6r$))n3qa#;`_c`tVHe_R#dkk+!W7QA@k+&;j*26!0dC`lS&wXkpR>ynAU8pb~Fh@%*&taSBFj^1r&7>q{L%1C& zS}-G23{HYQm=QURHSH5#$PnkaEi+<#DtQhIrOL}&f0ztpf}(Y^fut}7=<+nf3uip* z6q;V~hVSOBa&4d~QH9+Pni3E6nxH9B!`lIx5-+2ZQJFNi zW>(YWSlz}y2o)W3eott6W@IBYz0q<9XgbEopd1vw!kOC-O(&eSHw?}2)D68*N($K} zG#y}%mYPn>Hld7I(W^bBri+nlG&G%MJH(4&J35{PN?ov6Z#y(SW1bcv9-jjKE6`vh zpvbGvG#IairT0^BLay%4cu|2uCCG0Kr6Kj~MqX3`FI4Tti%MW*aRVVZrjw2PaBrjkr)dZA?W$8`ryQ1I=+CWnZzn`HCUC3a^($EA_ z+z#6WwA17XCzRq{-`fvOFao5zX=pmu1WinA-zhYGxW7b{D#n#+15L&3K3SYX7hE=_ zK)?#&WuvX331ZXH&;;Zfefy!Qlws7txX_?K%FZ>IDuHr3ixDsHXfacK<0Olzo736g z6c%FmAVeGEYObPZwmW4K+JGeK>KZy`K2V-i?=@)LTzAN;+8FvM5lgjbampkx=lXtE z0I&P1n$%S3urj7ZOV>v z1%}ZCZ&-A&qj{G;92n4%aWqh>xNDL13M+;@CwuU!Q97iIC3)2-!}$)p3hfLn9gVWi z-});fK&BU-l&P3!(fm@@$RM9Ar?b`LrfE{9qYRh=m=D+F3_jF&KtFE^)O+DVZnbV0 z5MK%-Z=yh{w6?=&AsA2@A!u@7e`*m`Dlk#H*ej4tU`lP#v{^z0&v)=td1WPS)M0`u zR8MX>1A5KED=D?5QcBKL_i5)A`F#DjX*OI}&u0Tz4#X#xPt!NJeDfkdfu${}3CM6> z(+Lm4U^YDUO_e$$J2dlQK=S#(SZYlRZ?Tk$h3l8Q^B{H+9-N_g5GwHCj@gn2p`Bn2 z@I^=Wtv^%*GIh9BdZW-(U^~;$R7Q81&kY8&QVE(I8y-_?0Zk`7-xHdiGu;SHZ+6J% zCM8H#QU!(5k0GBMkSW7i=#4^CgKk1&0rPU%E}^Nl;tk{gANe-W)F$M0*$0~12H{3S zQ#-aBG@T`bNuktx^$|!-yeW0l&@@7l=j2d|g0o9#f=$mAO^#AZw}Gb7A^L0|Xc`?5 zL_-r_Kib}^1wBA0LvkOwz0~cOno6snTHPcxmDZY(Jd9?O>CkVF>IEZH17c}-S9c>c z@kMdA*XAmQKxN4D z7A*Cmjnsrrl-Ly)NIZzti4$ySmXj*DIU@Cq4q3NV2z0jbhYtj8P ztwlD^iv_xH7pE}mSpvg`wJGxD$tirDqwqIy%pyyNNj^%la*)nfi+P%t2obJ#z2M8EQfz6JU4&m}NI{hKDtm_uPH za;y~Zt!A0Q3Uv*VF;28rrKPS^sLC=Ui71Ye~CMOncN$V zCmG1jW7vx{lIEPwX_=>7c}=IdVk zHhVh<W#arHtr2=L4+PT129rG|u7g zbhN-k=CBL&;CG8q<4$<(bSigO;AZ+J7siJz##qED}5?wmp z-f-vvzFkhGyL7d!vv}H-@~czq%b?0?fizpFpqDqAkea6kj)p#%k-F{(1-iqov0>Cgh$D# z)e?*cIocJI!GMC3j%_Rr&!B8#GR*G4_+Ag4U>AlrTvW!`#*klNOD4m1hI4v~0MA>0 z<^p6I&nd$JbO!X^*FNpDnZw?F&qp`B=fj0Ctse9N;Vn4weIrT3Y?Ll1iykc8G+Pn$ z4n3Sv^Z?6Cv6&jh`uNsONVD4Z?W5JZ0=-#!MlBTQ4GU$wZ$+8H*Di2lZ%f%|IhphT zW4&}V%8MaBK}8bC8VIz~v?9=GZZ+{C^Ruj&q!;BikVUWIDiEEb%{6>fVRZ}AcwY;{ zL#mwmy`^^fu6i{a_Lxv9+-6Dcm0;>898{49$^0}evxMJ&FGfGhhGjA;@+pXc@nW1# zuDEzVyL0(c`knfTNW3lCNrs^?n#R`(>tqR*Dtx%KD-$FPD1_oPISLerO(qCd)J!|K zyLBA80A_TQo);=$#NC$n??XH5T)hc`rG_z$>FsXu78J-7lF0rON-Z;-{T1+{W!yR4 z6))mps+ZKFTBlWK>R=oY-(sQ3@!_$|jlAeECHQ{4=-@+G)xe9+;)TfVc@fu}S!#+N zpAj~cEG>Ajp4*&siejeiCx&a*G>z>BnV2;5oh9P#4{POl*`VtdRGH_#%XdyNJF`aGqIxHBa?9KLNe)S&cq5< zx~(}A6C1Mnd<(HuGtX{2XX2|{j5B%?R(t+e$eG@DbGbW=#+l?aXL1%U>~zPOoNIE= z#8Ts1aV8gBnf*AE3q!ilV+F~C-GMWCeC&ZLRZ_{@&zW3HQi570xV#m%WEE;N#(=>| zdYnxFhR4gr0>0F}wOb+wK247&*@GnI{Ng?_CC9=|+D^FAV*VjXai!&8o`uk@mTr#P zwcW#g5MUW9;pA!&Jm6~fucmh!?1 z(6E&xrzi!x<;fr-nUZ*!E|%(Wr|hptUWgC^*p2;!0Wh5pbo_-g$ zc!8@k%m>RU23zIDFvi*yz(I@jIOKAPfmX1<VmZGnt~9d zXbHXYB#({#S|_r&x5R#6;4m>2OWtWB|Gux`CY-h+RUd&4^-_OeBC z?^)O#6o|mtqClZHFY&n{41uX4gVc8 z(}w>n@+A&cj6+VpS$3Yp^-ErU`TXgtFTPA(9{s~hYW7;2PEy#$(TuK7TMl z?X=%A#elA#Q@o-`(A1=78rG61P&Vo_1!JkjL$zqTPm`A@oc}PMWMvfV@V`8c#%(f3 z@i7boRYStAm)zs_e#79fOBm&M|Z&9`NedWqfbPx93SvkrJ0BN78@&QP$#d4m&rvD&`z)v)Y6Zk89uEg{; z{*lej#zj60eiqL9o|WDf2WzJC4X)33n*xpw6_EOQUZy?%GO4-`;+I*#riSSDb&xKgz>jZoaw zUqOha!s6EOyX5zS$ucWJVLFZ1u75B4vd^zCU()i*e)y9bPlLG9-S@A5 zoebviQ_}Ar!2g9lE=O+s@CF~QGZgO|&6&j9an~^Q322RYwwBDt^K6ukCrJtFP!ey7 zC-?*G^>{vq3fwe>@j-jK0N`hLK@)fjiX~Fmr$*$#hg|{j1C2$Z7soS>T^aA*7vsg3 z%j4u>ycm8rsqSnfUyWzWw-Mzy;7?cZ{ZoQU2B=GGG9NUJC`7yg6;SZch87fsXLyZqL`2Y6rNjf=)oj+mtN8_N@UY#c1B|}1k1{Oq82~vHwm|O%g zn>>2bzu(jysIPwA9dKdN%h!`{Y^Sf0@on`Xfg#Fdp68PUY#I2#Z)TV{YFw5}P`gjF zA}$E>4d{ko-V8FJaZtb~$jYYS6pxPfYPjLDzzGi~SvmvpTwVHy4;q&s%>3RAWc*o{ zL{S1IX)y$1?l*!+8FzJ@M=@K#RV_MR-pdAWlJRIgWHJg{Y24Tv2LSEQT^G;U+ib96 z1duC94|F}y13`==;&;F1U}FMSl7q!`eh_Q*A|?IB+r_8Jw@+Rky?FWh=pUZE{OoW3 z_r{T(x!f3l_8*eJxzaf40X|vs+SyMkI#E(|}Bp^i|5vu{Z`VxM+sFnp7u$+$< zaWU5{-87PASGKYQM>r`m5Rm_v+_}_W`oD?Y-yxO=08&7$zoqJbzHErNrYmory|vt5 zp(#nS!D*h{IXpZEYX&fRcv!JQh4F~(;k$@EMkHOsD-hK*t`kgN88@4hjZ5d8Vb{sT z1;6^in_7FWuYTguxUsVImr}5?D(HtOqPJBrpdn01H7FQZBe?ZGpn`#YI&^z%P%uni zf+C`#G(;s=fid{_=-Kn5M-N{fJ%%4urRFBVQ2Z(A|0p7#&cn|1Q*e^tCC9VqAgKVP z`pH8;$h^qFqJ~Z4r)zD}0-rHDPZs&SH_6VjNs`B+e)0@3R%Q>N%Mk!@kq3sGj!!C` z_hdO9W`}EoJMuuVxI@Xoy5=W1V6Vs!$&_9W`Y%3(SO0=lC^gK+9VQq63>U~uljpC# zS*?t`vOH#b782Mb)z9s7B!xels5CJ}P5cLy{YNV)n+45e829B%i~h_tb|(a(Q@TD^ zn3})dj)nojKpmyU^oo6ml$?#TbLw!!HeDqZu8no?`XapbeI>>O#NHJrW-Vch{5&gq z0DbsjMcn&`&zE)HEyM3s!Z*s2`tsozN*A!5mWb2K+7F*Ssk~vK9W@_o)wQs%AI7s$ z{$ui}$Wn|1M=$5eBtK3k{oyg1ir28zzcS;<0>{)i&eiAUgi^4%Wyt5y_FKaQ&u2ZH z>Y~V~<1pXkqA;S>_x9c?>0Eot>OS^+k*iI2hrv8=K^a)v%30>OpbXQw;koM=0~83- zRFemVw>qgqKY&);@@>})Q=sC7W@_0?qjyfVST!yCm@ShjzXffG^GR?W z7-2e|rITwwDofbeded|UOIA?a_b$jmL|!fn7hsBy@c?19cU{jkz?sr*X?Mo)&8yY} zVql9Jm#0;1PqkXOo1dSKCsCyiY8B?dgUVn?rpPVTa>Cx%jv~jH^BXA|W`jvuRJ2q} z3O}YpH1THFi2*wp@9Ct2(_sEr^f1$w9${BW6KQ%*#`x0X+78U#o~FxqCb32wNdE=C z*$V4&k|Mi;ZF*efZ-Cq~AH2yH6=l-53BT7qyq%(j+TbgnkBc6cy}=l4x*2A3Ati)s z;6txXYoaia_xr}xnx)g~(_z*-^j4sADX0TUe4j6gC!7z$pV4PA1E%>oEAkb8GToLU z_e8555sZ+eGD1et+6dMa8ZhV&J2}Po9`VdIEudY7 z*YqCXPv>Kx>=;vp$E$l0Z()V;pmr3_24@$Ql?;UUldnNn#-^27b2&2n00z$lc^%;s zoex!SD4#<3V?Q2aR^tv}Il^PkRUUKvNXy{wIl~~TeAXwvfbt5jy1d95?I`#n@O=&{ zOY@V{{Jb|_1kJCk?bqRB&qUZ)-CkCS*k1Lese`4a%SND3xK^kG+DtfK-LzND@%%Cg z9;HzMOo`tmO`;A`9noLc0q`q6d-+G$uQ2TOBFG!~`u9Fxo{YwClTSR;_gBK1@wUWT znjLB!&ip+Hqu5>g)Jmo-Rv63gU&0%UJ!b>AJ@1^CU$S?4H)HwL~(iae< z1ng4jaFCXZOL^9EeAKb+rtvX3{~QzEAnfNcdIFy;&&sQb{VOVQovQ6}{e2{ttM8-q z#%ANf50BHZ5_<>VsLsz?98&alrbTj-4wbNI5aaN z+(i11kJI@_r}q18GOan&`a4_h^(OgLMQs%s?8~!+MGFHJnqfA_K#SR6jLH@s9-;KVJgcpp7@A3go%4^JK+ee>w( zF!}1~cT{Dt%X#Cxlyx<+ZL3mkct|jtt)_7A;cS84VbFChlAnP7pYB&wW29GFuBLa` zJhwhIpk;b>y&zl*MXMuRM+qAb^c~J;aDN0FF4}i-a6Cyxoep%&$pt8RdF9l_>eOH^ zya#;c<2+bFC=BxGt0&35m2#I}&bxn-=O>fw0JiIcs(!yby&rT}&`|3~6DmH#6xJ-h zfDU?5IX;=en0@jCfgm)PI;0)&1jhgIn*ko)ki#gzR=|VGjyC~MVD2Ua*ecG=Zd>p&sGeZ3ptdCVTEw+5cq@CjOPwcapk zZ%D`UhD*H1)e3m{=7MT6n($>&qbcA)sW+@c;CZ9M!gT^XZ?t7y>3A5|A5PQjf!Ao} z5As>LoKXH8-E`Qmg>7YCW8j)MW}5(sY|z6O{d#3K8TE>6RAl98Z;%&cw1jH#J8izt z9Le;UmP&6+rmzwyEHk{IGh5KEF~b{3RH6lmyf2L7T5k~maGbu$Fm7TnE<=33St^wBt@SEva^`=#);8$qDFyCklJq?@y6pGU{&IVK0WqP%C8{ z=<@BpsAFDLH?CR-+>ydBK$Qx?8xNr?*goD1ugi)M8WHYYF2@s00220b2-dJ#05pk;OJJ zSbhI^twvWf$zU^2*8gCjcR@7bKz6f ztP@63EIzDz1FRGKS25JAE$hTa#%cnt`05|SJ1!ltP8?0uz29DyTHtco0Z@;#lkp57 zH<{$;z2*Gsl;6vpjkAaCptruf4bUw@d`=ZkGcWMnv2_s6>a{Xriq8A%@z+E>&omx0 zy?FTTQS$ug|9o}y;$>~RiB)wFN}gO-&t+YPs#U4t@s`u=S@nF1@km$JJfBT2){&A$ zR^`Z;1*;7|(FHX(6V_$(a^v%RTP(L_T+$0_nV=X>f!!G%_qy@rzPuSw4)Qq|=kHdS zQF@2_GZ^It{M_uiIa~2E0;#h?#M&ch_-32SUax&o?ZnVI&zYFyD}%R+uH- z`0WHWHZr7egiZZgJ~bcDl-LWo-iFIdH;-JwjFvZuT){&k!0nJLxK=b;gs>R0-w|>}{Y$c|QZ1bA z0J(#77Nlmd3Bg2Qegg%!3}K*sR_$<)j+arHhJ7LMzB z3$ttmOdnp8dLNjug=eGjEF%fW{c>Bxfk~h}`ISN8*Sa&rNp6j!X~E-%+lIJHfIWHn zB6<4VH-M{- z5Ospu9hb&eX5q?6C7^mRENrRadU^%Dhj?`&-p~5x>B|K32c+lJ zoOG4qWmk+z?N*MOtuZEz7Q2WruxYs9TVPCDEe!o#oxnKCt}q7tAxT;k=ZgQ*7=u3I zRS(=Dqp}vgnU#&PMja+98IB{&R=`Zg3uBjBy;pb1EV^)lNV0mNzy_k>XexK~n zt!&%0Y=awxxfibD<{r9M?vu5fB$LttABOrSkJ?*s)KvFKvtd7?sXF$>kb*IQd$1s9 zqJ!Z;l{(gmTYCd2u(+)5DgqKBF!1($60$krMt5G_my4d1v`ROXl*<8LLL;W9!ICkp za1mB|6Dl-poNOCZXs)$vjS9`ZBK2{ug~A}w4}%IV8SN$kqOZL#@k?nti*Q|UIm2GY ziy$J3bbu+^2B+!l1jNYMI89E*ijgfb)DdVc6HJk`SfQ-B1)vR3 zjcWRv?Ba^gVU{QB;~qeE$b$GF?4huGzMW6T1CW!mycY)u-=3PP3KxfV2Q}!$t{YHW zqw1Yz9|+XeO13qqEwrL_aqEnA-GSPN6K|r_fej;X0jN7G^qik`Jj3(&`zC}GA zrgf*ntJslHBfeQ(#?&Brah?|O3rwl0fHmv(5DLO6=DLU?fswSh$(>F(!~M30lh7#K z{iI5*!qRSpf^c|Un54xsrA$}x4DbHf>kYv|$B9%KA-;KfRRYW~TV;jtEQT;pvk#l-N|{_4KJ2=mTz7x$yo4Jl8XfuoFtJY8E4kfrOs;iOD!d) z(yiVYNt4y8&cek8G^i(&WiSMQ1qtdos~%I`u!LY4VWx?eyN~2NJfK9C8ts3vC0nPg zDYAcD;`1`Wb@Kp|9AQou%$QW=a0%5;!O*9s0Uy2%{6L%WYKyf4F-IB~Qs99rU23cC z?w45Ec!~oEEL+;4Qg~z9pieU4wpc;Qtf1LhGEYWvDXEx}-RPE8TJPGDD{17`i2_#| zOGtZ!KItJf1*yDg=~^CS5 z`hzL$(YNr>Gn^ShdR+4U8WfvMbLF!CWD}(1|-6*>fQ|B3)7Hcj0WpNrpLt+TibQ!;1C8p9z znB_bxFuWh11FCWgQj_ue`Y!Mo`otdac5db4#PBE-iE*ge2_}0VO~xmui}=JuNv)#+ z-7i4;9*{a7jR%p^G#u2F+4Oi4!;C*&ZT3ODpklhnkZe5DS==W(uihu#DRxR4F4Pv$ z0Z?Btdgn4+BjmriG7Y5t!PUuxTdY%%nq(i2*@h7uR}G~e&!PYD{NdL}R2324@FYt) zMp=CI^qVh;8cwqY0vw(e{Gv<|bW1${4OP}#IN~k7XMs98aXmTAz%pfTiW8hQ%-$3y zILD0GofDiRZn%*3jizzxTsWQBixXVDVoH}<-cTwrl%t6g9A3|A=)fSKd_@*JNwN{< z^kDo0Tq3{51KZqpH{P_mq6VZ|?KWgQj;bjhsn7FrjEIHprHXbO&(_y(rl=-W*X>Qa z8*)ZijR<&CbFo?9c8uW7g{oJPP^_j@XhbKafIMQCn3#4^*ZU{!NDXSq?6OUTpUWqSkmjooD|58 zNJSu>8qR7W#up#Dgfb(B2eVYKhQeEQW!zrV%4;ydN-6d;Q9VA0#;&j_Spj_6J4Jl}xj5F&}PG&=Yy_ex0bS;mOQ%LMhXhVvXO-*wh`sag>O zRt;xt(?hnnoj*$#r(ca{Z=MzT*?351c+gE5MkH@Vl;z%Pvx%rGBTEl+#!z3ZF<}Q6 zZCKm)@v4Foe#^`l4HvwPY0APPS`j6!;~|2u`NLuM{avWwjJF~~fdLfmY5!2Y5Qi^e z{dkzXtTOsnLOCd^@yWyLwYL#IeSP@?-qs2Co6=QO8^IbB{aVs3lhW}m(sR^^>(;N= z;7+1#JG=p5u*1zB{^h^@8<>&n=S5swlDuE%iQh!h6ovvJwKGwz@**Qq7`~(0uG2W3h=2M(N{+gzo}>imA56Snme=MRT~&~iP$??q~)7l7_}c?qrev)`zJpiRy6+Q zU;js>;z4|bGvQx%}FkH;sKrn)>nj{f24 zeEk9fYby*fGcRj5rSS=l_sh*mw4Rf>d^~FR&lFs(tH+w3167|dNBEHC=S+B#ji53{f)Dt-2^_*I7S~}ic+V?wkF)7_jJNA9}NG?R3H!4}n z`H)r#M3YlM*cK)OXZTE}=S=LL@oaryLMZKx3FLG;8jZCw$nULkiw%mLN_i$56;xwd zk)@Pn&exv!VX5)V)ZbypQqQaqjwsczFUa~BJ1phOc}tC}0am^~TL_dIyzcKET`FAt zRfww8E8SG88%s@NsckH+eH?>20gi!7a@%k6Y|OrfgrY3mcw|w+7 zhiA7Blao{UD~W$qp52(I``Pp4)$^~2elW9$xz^94r;m@mef{kD>z7Ynf{_y<#6oc6 zkm$lIh2EUHEWh6NbZ$*)E3^6ymKe`?eJ<&FjkNA;l=&>Ay9Gl`bLM4T3OV; zZfXeG2iwyaqqlvsyvQe{M_M+Cjy7gVB+G>z^UBP>z%z_hU|`6P7f}@+jeLHX%$Ji% zSPk^KUU@+hqj-WZ_7n_->R`33AJt&g7I0Yz(OM7P<4WtE{`|{7mfxamKTPal^5SB) zNZ-cm&xgsY*~$$Jr2XSE$A6WJqBgP?2XaCxJlVN2z2$6m83)sS%|RQ;F7xxoxj%gN z7$oDdz)f(T_1Fo zgZ!X4z!fhK?45t9r%1?{zGWwhJ`@q*s+in~h}p6(_!B!dEWTW^L#Oo}TB9|;4tEfc zpV;KPhcA*x&yOCyJbE1cIhWn7hl!~?gH$~(@^f^pfoSbhBZ0Me{i6)?Dq0+KJm3** zIE*_FZ|$Mm6a(*d2}Z{S%mwRBBqcU&pgia-TwkFsRa8uGY!jzP_^CHI^aR4mAovRr1#Cr~5C z0~W7Clh=U$*QZP9H<^q_+2CR@c`yI=!za&C)4Ye^>CHp9Pe_skzu|G_dodxv6J^Z+ zpDhZYAlyh8VnlknT;#nLFa=fJ zviexPs#L|$#OKQybP__ZLp%**j8kpE3Jm`F@6vI|Ql7*GPUq-YPABUp*N126YygT( zpX$hq`V7sXR7=auZNtjvufF)hlYe*zE)T+zeFi^bcb_wRj) zKne9N5TtE*&7duU{{Qygt;vlfOAq`ZdI_7hRgu#O(C?R&MpJASOIYX8WQinXTE{%A_alqyEGGl>P<#6ZRax1b|FLW`pQ?LTv*+GE^c5EbRx#1sUBY{vQ(;EqNp>*uo=eCjjT-3Ok zoJ+@tD`V(0V!8R5Z|CoqZdZ*8%$q^K96oKv@TGs_s0Qq2OlqFPid=!J|4{sY|Ls5j zA0TwTc>Utpi*K>;*!SN)fBii5jV3g`o2+(-H%0ONdWo^E`@5)sDQdHHrlA^%s~!)7 z*(@i^d1`-HkZY1W&@wBz{O7Vi+9-RJ+%gzW+)k+QSL^u<_7G;Hfl&3haD65wje|Cj z!E4}zQ;cGM`qke*{qET}-~HR-^@}s~HWp>^_3T6PBV*F*#Vr-)7RT{mQKxwhe4Kz( za!)nlt39->;JlQGm9_9O-p$b$j$SVp)x4>9S(5QGL7BXh16U1ARb?!%mdafv*yxGZ*vV~S#O+dd`X6nUal1ZFHJ~ZC>E^L|YT~IRlv*qQ*)9EYQLLBq`FS;Y zn;N&ITAo~Kcn_8icErmU-+uf2tKwHrpZ(ogteA-R5IWUo7dNSoH6#0M{VXkC;>*Tw zgYN@d2Q_pxTc_RzrAMY$)e_(FU0%^e0TzhqXw#(k5o5}7*)JUS*e1HIAa=QJ70*(? zjADur(l=0oQ-0d#BRZV`IhI>Bri^h{hq<2{_PTBwjDH>Cg{3))s~!_0;2P^;>W*0_ z)62S?z$Ka@j&)VDVkRdf+>s}#cb0Vq7+t~U!*F6sigmS`P0=c|fW!D?I%~p@h2uhv z(|^&DgXW~|dQzhV`8EbEg8G?D78mX;bjLk8+Ymw$JF)Kl)oflz(y&bH9(AiVVxwWg zlh>2t?|X!y0$lxi(gIHQlwM!rlZUSMg=(oNojFtNdvO}%|lQ(-HsVkvRorz{rU0f{Z(_BaT;W}m% ze`Um3qy;%;^6d=T=u9VJd1OebQ8CI0An9$y#bTr)iwGkuF@0o+Nws)aHNdMejCDwy zREF)FQM3UmSC~7SE;x5f52MvPjs^LXB3X19kUy2?L)a)Y%9-Rb^|+pNowlycm6Hg; z_*hb`&!ziII4n7RyeozUo*s{32`&W?z)cq6Mrq=_lDz|k)-`4b4hfT_If+3`j4_2< zDBbr0s4j2H-6fFXY^*cMJmdCcYOIWsN|1rkg`U20HXU%kgODI|Nd(c`bcfD2+x0tF zp%@*2+qJ9JcQIRIt}HsC&u@wb^FymSS_&tw$j_*lFB`OpE|ys9FUq!5A-StT?YQ?%bqq7L_j`1`t~z?`IR$D+;JOS;nrL%F-ZXnOgXQAPE#oI@th)CYkrGm`G>+$R=o7JtpWy>k1~kFjM1=Cd{JF|D8k6SO7?|| zvI^>%`!S*-j8i>RemrK}J>?(`eim!xTf^z|>$A6RzVc30U!zeDuFs<(UVgMva-FeS zu&yjUXU4et0Dcjl`L#HT5DsrE!m;See<=R^U;e4rKZ(|~Aii+k;pW-RM&dhbthep^ znjzo84SnV~Q0b=s_*bxV0uH|a>YK05{BA6(^9f}<@s$>?HfvG5s@K3o&Wacm6PdbYFv zBtQ46Yw^QsGs7JsuM5(oUY1h0m!CR#Kn!p@w$x8HPj>t4NwTd(f{=fzbx+{nhqs2V zP?KLm$LYJ5(F$u%@+o{z*^aGmja0dO%9Yf#AB#^xYufvp`|&AeskaCB;!_M^vs7O` zg|OE?X}&j~vP{a}e98i+O7C$CLa+Kc?QyHPk!e=7UAU?o=$*td?6vne)u5)nmfc6- zM^-0FLEuwEsg|;L&guR_Wrr!c{Fyvu07L^&pnD?5RO`*`qMD@9hq?@XI)^YbX2_?6 z7;5tNZpHH5teKt9=Ce)GJl~bTWk?pOARrROdL&WK7U#2~3^Cf~z{UYxiYTS} zBn9rKZ&j_+bJm%IEm;;#s>C_7k9-sGdcb zV;?W>UPrsh1y+&|&}h)!5dbt7Q0{{f1GM0Nq1xuETKmKWS7tsyK_K=ZoYrzSE#}MR z+wCeba%&koffA4JVdnMmM0EF5b2SL?_s5t&Uj`>u_+9#&{|i@W)s?+l)N_v#^ZeB_ zeD1(bT&>^uI5lVxMM=igX(f(l_`9dpa7rKO)M3F$zwesrUA@cCT1;;NV`D`uL{qL@ zQNb7waD2p^P_5bNdE^*d z5AHlbdXJH75Ez+Hx0Zh~>j zlg+l8&u={P{83%OjaX`KGUkmbb-cEPd&-#@RI>cX*>Mhc(WSi=ST|v@DVBZ)_cxoX z`|3c4cjba%9`VL#P_;Q)5ZDi1aGSyGIKqo1?+K3e>CFQ_w3WlJ4k|=M2%Ni`f9HtP zoEjSA%4SRumZf6p39_z!%7dIBIDH^rzzmh;Cg3N!yt4+QfbgDJ7T2@+d=~zm7@?$E z=WHa&9A0hesb=P(@~}`$vMd)%=P~i3TIDa&YC45Wb(*?c<)J2EZTa z?6hU^c6GM!%hCZItk>HW(1wn~)X=oC8S*MuMu{{l$3&fnDm*N&Q33^KyN`vX?GPM{c6Nx9vt|tEX~Dq)Z-1z>mMlwz`|BJT`v_%; zs%R$b*=iGSJ(M(1?m`UyRPh=dyC96x4E7H1S*PI^fBwJ!&sfRmtat_DvoAnXgrY`H zON*&P<5Xj?QN4}RW!kvvGG8=pH6g4VfQcqfIO=Exfr3(tDlL&r4u@{*2IB7wn${U; zFU*cX&?l^S$pDRM$!Nah&gO!tYXeB6B8x;24XNV1?<|5a6m3>6r+bLakL67wmVLGf zsK5-%S}Y*jZHwuQ^~(y^vS>5aT$%x^eKJD6E66Ped5DuS;nd^CWK##1eObY@&Nj8P z#=tviBb&OXjSVR@l31bpQ8>g-niItg^kBg{+b66oq&yHf8aQ{;j<7{0?+FS9^l8wR zQ;1~;v5M(!&-Hzmjl#CqiiqAEgmO{7zgkw;v))G%(+C93B2HU{6*K$d4Ef-+Q?Zks zVRV#Al|`a8R&#qV2vT<~r-rTYbm-yyor%1E)PYvudCVIqfa4SwQsNStvO8=X^!Yn&pkNQ^9T&IS}#|i{oc5$ z7AS6i@xOh6C4Su3)uyfgQ_vJS!bSjMcpX0*domvV>k8!7o9pGa`8SVKA1fJwv*F`A z^XHfEQLi!uZc2_UvR~Tf%#t&Vi-&d}ckLf#43RBm?){HK#__#&#-MA)60~UARxYW0 z>4vi>Pr9jDz<=KWr`pbeqJQx_aXnrwR=5Yn^^-4s(THFAZ@&~{Iu>_72ycZR{Fl=i z{96J8sGctT75&7Vl zs*F!1x0dxxB-!U(A3KK(XcAfprF#{LaV|U;c0MHYE~kV5&;^`~IEj_or?e8$CH8QZN1qXj+7k$}J3?JS%WAeLgCtU(ud8%Is_9C*G1t)j3e!X-ZCDJa?wY!6lPLCIm%Pc(itI43bspO+ZC)bCB$d23);_T4 z4>!F_R602OOtOsP$_y#c(H&K1?|YreF&qPs@cLf;-3jiU#;cqRDz2RNMfj;}HWE@! z*Ru<}`lqu^Z{Pwqc6`A24Sum*tm|rW<;uK9IMKX0G+Z*c@%kE^6-6-7va^ojJu}X^ z9y2WR(Wh#SvzdnaBXC>5(x>7%yNux`i#SOrVX_BtJ3Dv~Pf9GzmbPzoqG-@88nxDv z3!91S(G20cIjHE|fGlSOp|)?1C@NSV%4Yg@Na}gkjbO*xFzJA~@+_09%}u+2ouy$Y z=``(T$stIcq}0{Qy=UnzI1JP6B*c!*=NLG<0<~~b;VthA<25z>^veRVQ)1OF{MBOz zur*m?0l~w1UJlGH6!rT4o0r8eD3>DE@!~nfQr!p*(Wfd!W)xie@dudWWD#UcC}wTB z&;k6c`1Aku--{nY+J?idb{WR&*|aYH3Un=(M+WS=@+c(FJ#g zCgfC2M<5Uc<4s-8uB&7P?ap+f%xN;53fqrPZzd$&GhrK9)jAH?u zG_yQ+5t%$$?Yx~A=Qb*ri}{Wm4|$1bD)B}tHn6FRM{D?7)zoonn9hN`@f2xws29Q@DU_xQOZn0&p%XTq>Bud9`XwDrfH{4!=Etzk@)_=Uc32 zSzNYx8t$Co4u*wku0V)?+Y~;(`d2Sr{~FJQ;}uOT0llbQ4wj384V#JC+!@0Z7nOAo ztKxjV1X>Ww;uvU4CyIHq2%;pJ&AnbY<{_SvF~9%y(^t>meDnJG%U8Z05Zc#VM%6W@ z{UTRF+rtKpZGi%!c`gN5kMR@*`|E-@xEMJM6XD`v$`;N(L$4V?AG26!T-05~LGrx;vd{dd&N zk%6P>)8&(3?9+NyAr^15!NZ3Kt(~zv8ezVa(MinpF8L_{zG2chN-7wBpNa?jl`j$T z3t%p1MQmV)80T9+@T|?4=2ki0w9 z3L7r6Tn8mO+)ntP^*ucwl8=d?nw+#5gt(FO`l@<2TOtyDSJmwL*kv=Ji9?=gS=TX9F+2trMA)7gStlPuq8W#eyHe-W6z_? za%z;3L;M`UT+l446$syhR@UaMtQb=Q%6Oj?6AvuY5pcop%OwO>D>tbujy?b0beY^$ z=DrarKm}@aqEC{9|0M=&H@(qSGqb;dEQMt=WY65HP$x-gRfdv1WqTY>LM`3X<%cwD z%9Sv~VvsN*B9&w6z%o^SD5g4fyLZ+I&stRp4236S2lxH*&87t9bUrw*kLKWT13la_W9YEx zBbUW7C`l)a`^vF)=u0G(q;tGtN~nJ;sVsLWc6TT9wpQ>9{Gx!{1ciThBct$-x^fKD z@u(pv-`qve!1C@*@uXVK0zCbGWt-}ub~dNtE#f~``{i^tPOS$*0yJV1KJvsmNEXmn(3OiF`2s!hodDLj zuAOPAnm6O(`y~i>0WZfbV6}-OjHp*QC|5RmS6BM!x8J__?VH!nU%h_w?AsSVeD&rx zKfHeW`kNQu{SC!rD_6PNwE_EVJA+xiDX!~k;Q>FkA*mgP=Lzx=QK>amLhW)~5GP~3 zdF@v2H(QS+^>jW5N~b^I>b%BI`z1Bud6>`6J?E$w?_A0d!4f}@QzU7hx$Q~n-Pyek z-x&zxM^hO*WoADq<$D)@8K&&Y0p<_LEpdq%PbOE@>`OIfV|Mrye-HBHwEo}{EwnXf zGC6P7Mst5l9jISd%js{|7;v)w)Z~GFNX}T2KC}g@TlDQ^fDETDCbZNpGw@hKOH5|* zZ}>cx9odX3_)n5-#?0+SkMR4z^nUZ>&afnBoF_>|r2mIF!&0&{s*+?2X~;t)n@ZRi z>HRFNRuB2SX~Nh~$4AW>B}vNU@%rojsWD1?mZH@_xk@s92&bGQSN%n(dbmOr=uuKX zETKv&&E$g=s-)KP;R;pK63ZSilUKmw&F3dnDJr!6JcKI6l>H2aDkb>df&XlTDx=kd z7OIR@Og%`U$`V&UCiOH1#6M`E3a|ILTLh*}R&)Uw7%@t{O4P6(qvov0;%MVJ(2aMkIDDr#|pZjpz} z`Z*Dd$j?sH;zV-ud5Bt^C@IZnCu(t`HUIqVSD27T=mS1)lsJ*)r{bgbEX77Cb!o$? zr;H2w;3chDjQCq30N?oy-0C22fQ->>Tw3Z9lMbrc+@+|_tL7>-TIo=QhWZ5m_F}mP z*^2%}wt9H76+*}<^RQ$qYCQVVPs0-=Ps>)+K49M~c}f z{k_>Uu!w6ZwEj!L_CG7BN^2z^R%{W+0)RUo+}>s%$2EO)9~R+=2}s7SRUk;{w6LPB zl|P+sqc!l&1Z4(6eOk&^>m`=Adp(=HZJsWsFY7=0vOEnU!cFTlmllc%km1EAh%@*G zCKRUh%*ylx7^;Q1N4R6BLsnu*B%kYC>Y0l90UeMc4zwW<>B>Vc9LxdG;3xWEgx@{?ZQ*PC zHg4aJiUlG?`O$CT3Tw=R4s&_Ht(0I;aX zY#N5g9ZpL<*=}a<>f)k8*}#p>!KQ52b;NB^M95sCl~$~q=(h1@?r#22UE7uW1(Fep zB~0Rawh97?8zy||<9q^HhGgN%AX!qi`dfPAPEeyOR~2kSS1O=rgPSnPc!mo(Tg*`W zh`5sfs@*UfMAqKl8BXwl_VA>L@iT?pZ_*2-Q*%%lnm3cS)*y_yBk6dc*Tq4r}E06Ig#d zhOikx7JwbRfn6VP+UAv)!eKXEEK{U1Dh@IYBKB4NVUzqsBmi~bsUk!(_0h-va=W49=h#g8#iKdivb+}WRRQZxxy z=xsMTj~H~_T&6yd4pZ{@Q!R<pE-|kij~qm5F0LLi|KDcM*vZ4QomddgI8fB z=Xy6|Lbt{!^YC=xpk=p%vaP0$}8~Xe0do%1r8KO!ZB!aj`jS+jSInI72{K+< zL+&D#sw`5a;llK(SP)3nT9lySVs)zX?F`FOfmlC?%)BWzpaF8L`bf$1y4+OfC0gWM zO_xbM$fheZ=%Zs0Js6`ZS5EeYpn~0QfKYV0ww$91jx+b=iE|A#H;6!6R__t)6=eA# zB}J&oqD{ei>I8;Zx>62;hn4T@^>j8FRPRJ*(8k6LcpWGGObXuejRl~v_#pKTNl3&cka673byic3NNF7F&ZoHd{Sb9La6PQ7OcLSnqaVc=c#E6)Im$OsU%c;o8-ZqoH<|5gb;QdTo*L-g{>%;n+==AiAp8!^wtpy|f%c6C>mZSi z$KzxJR*2M;GR6&|Kd&ieOiHV|rZhsNCMOk|u-v95l@rdm{CQ0(+)(mRBMFoG?QHQD zz8=RQ0#_s1)zWdHLGPpSuvjmbn`GNYCTCkIDGB>|%{FMvRr3z>uIWCV~{c5K1Wg5jL!Guc~Z)998cUc z%L=`n>m};P=Uc36Fip;c)yKO2{`J4_)x8xevB5A!gVTq3?RevBVtegJd-WmpG57VPGi0yHyTAL#fBeT|vR7yO*6P$DfKvc6`OoJNxV19FNjtWhBxDr-Rlh7FRn;7d!mxu2|sjYyVK@de>h(LR3DtY)LwI3ReEstIt6%$4pRPFRIc%5Zeq{149y*Uy zODK~))*SAGAzm+?5uY6h0^yd3Q*-B%R9&u1U0~@^m;q!U=qh%TGo3Y4eUVO|LM-8(dTd6K@O7}bD zdzb6Lboc~cEb0&z-dZ-OjFYCc#8S@A)wGzR zKc{0VyWBaFWDzTwrle1mFqZ_dUBN=vE=w&`7aUTzNoP?AdaZ_=_=I=;YPl(~WcYwv zE%RU)Whm?8y2i3^7~Nb>t4&o_o8FOrSyi}zHZdlk4l~wc6)Vh}5yqAvk>ZU3|HtZP zLvGxxGb_fJ*h3Qa4o?;ea~TNb#X3zQ?W`q5GfA{Wlb#1RZoG1~2#Ii?Y72@nH-;Of z22=+FaSCJRf>b}?uu`2xp#r_v_HnE4R=EK+sGgP~OK(V_QkFN1N+1*-@6Ynuu*i-p z*xsuQpra4gi`qT`J2TWkOIEfak^^?2&O0KYEn(n9pSg1ZxiR`VCIa0Ixqn%Xm?g&q zc9XLS^smkt3)YoBeZ3hlkz8dFNwuby*q&n6^j)=>)Fq~3m96chT&E-5JB2NtMJbwE zo$avyk)AEyEhj$84J{6H*ayq)dQuNM+*!nOL9yrlXWPZMM+5bs+>pF=hwBn<>i)^W zk|+=eK_jU)%k^H>%=d#zI_J@K?oUt_vSQ zT@R6~NFI?IgR6R%*`-`~(}Bw_>oOfsd*8jMP%Q`U%ZAJ(-ec2HzI(b(n*PkaAO-@F zil{_A_*2?B6em@HBymDc54^s_&&>&cP$v!cgCF1GI||SN1C&HWfVE$j4`kgdvIqUZn9m24xFs*+?7zobB>1|6T*q5 zba?g%W>Ht0$yIj-$$oxF{K8=ywAuI`lc>!ZaTybs`LQW3uZq>${r!$Nt8|G9#bnmz zqxi&r_%xlEf)V?hWeNXF%?Rh@AadyYmuJN<1S=lxx3;d|&Fc4$Q*8*%;9B`+zk49{ zEM%NK0g@lx8tl6!KD?;WN?#}2_BgIzV+MWrp+)DoPsx8|#Bsp^o=Re@wF7fS zZtsBGOx$hKgziMLHYp>78qV^od}G;hkRled3rEuNGksruNsk2)glclK&UbZPZdX$T zq8i{_kRtCSOQDD{rzQ!m2eb&4aPY3H^`N5hS%xb;#v4uXe(^9-6E8hZM^u4Ryd}Q= z(!Kr4zkC<`4bIY{aR4h9GxR0_QYEe0xs0<}H?=k(YY5rj!lh(+#Yzw4P)_0DdU;b%2gu_z?-nsODVi2INQuI+pH-g$JVBNjHGFufGv!v&@=4FjrxXnc25u zu9mCq>XT)jnlY$=)`$!Qht`LeZ@$FVH8qDYP4_f9x;B}T$8`sIs;J=7W_rRiO?#b4 z8F}-}8Mi8NkoLvlGK65HaXUH@*z&d}NDY*5k*8oOBg@Hr2^!4X`Ub?sB)4JrbxWDC zCJK*q?=z#QEvp6(cc;Cn6DWq;ED{a1gz4S}r|E3HxhW@Cb$=8ahYdush_hT+W%|z_ zB~}DPRd?YJanI+No$Qzx#2-Ip6Kif3lTW4#T2SCouI}Lht1g_0c!r1t19*fsd1M(z zd?;hj)e{sL+4}8t2^;MkM1!|OoU&Pr%#NkTQ1d=63Lj2;F|X@QxjaXYMRh)#&o)EM zh%}E#MXb_;GVogoaDEu5awqIPzVP)jNaSslJ1hR*|3^N)E15eHGvM69)ee`Sb98xm zywwREvL|We@dg0dT zmWii=F=ik71V|~v98`1i|u@#8mCY~-D9q!?v-j@N=;LfEDcdi z^!IHXpW!}gjL17?h{CD+f;fgqoCqXdPL}ieY@oiC<)yc9FIsA8|A+#?bP2p4*RYKz z7;-meC|i__2{puuL7?citeK14y&9s>m^|_f(N>=b#|bX1Lz&ib6MrHM7> zw5i!8@B|UDQ^vj@c(5Z1#C^ox=%JeWQUK+oh8Y|b+n2Y?1h-7|@tC%lO*|%^tm{p3 zJv+}qjwc^TlnNC`*%u+&fiOBe#zCRhd7Di$MvNR33K8R|rn3f7EeCj>S>F5^$>8c9 zpayWxONTUEHrw?@7kHSurz!wcTB{|Or&NLIa)Ne{J%I9Nh&?IiY|t^465(L;C(cJ> zxWXMnCdy_}#gq{#@XsT`n(v#kSyZbAjT~?*^!J129aEso8W3J={%m2&;txqt)FvKLnK$23@VhPgG%_quf0iv`u)Sb2oIIF&cNTS!G!KtE}7Na(S4DY5KZ(&2-f-H`1e|CBK6?!wX#GI zmNzlTVq-)bMidQ-w9A_pi^47hvF`_vmE)EIvKS)W z+95_pj`@rjWQH^@lTotzo{_vjBTSpth73uk*hT#>9Y{k*i06In3P7j^jaX`v8L1`- z^R>&6$cy>%eHoIZhsZ>Eo6u6uqRx)*S#tg7|J@VmzPv4Ld$f%*Am!vK$HkwDovT6= zAXTka^)&Voh(1D{z%oTt$UeDx^#=4O^f5O#&8EH{Alv5bKrxmABwOA?5V_i1&*S+R zlJ>^4$kLh;Lr-}Tom|^3*7DBE8CeD)M-8>7-~*a^eo-lW)wtX4Pm~Dy`nn5SC6-Ij(4*Gq4z3XAVba?HF;cCrv^y)$toOF5Hc zZiOVL5D1RAAtI3fru3EZ2593di$Eca=;a{~nQ_yyUaV)6D->CML5~3on@6GosjD|Q zOdEq}_B3)=(7LSBIdkORyHHarGD>h1;%zgIdPY?>;;j zZOZpmrPm{AgC!#${Q;Oz+c+yc6q^g%gfN8$ZtJt>$oGr#B&D9)y&(i$~#6T=D7O zK29yKgl)u=%PS@C2~_=Qmlux!+jgLiUV(rSOUrbU>5;?cnb zB$rze$ysW-jkSvODz(ifJW#c>iOh(Qg5cA&ko7&)1cgpJqHS;gY&3ayIzaEMaVf-{5mub?oCW_%h)`Pt zCrn4+bgBnAP*6aX(GGE@hgD>UJd7G325(O50XAj{JTR9+P*zV=XZEPV4!5)g@AY?X z6)=4tEtZS_RIir~X!vmviO6#0Ay$Uwii!b%RICjjMK=#BJgp^b00+x4%j+?587izl zIxXIN8Q`?{Jq&uxxIjO4&tl6%EDTa-5yas}(riF%hXd}zu-md;yc-ZqYO~0b)G8`@ zrkjy4R=#WSV3r^ul$}94?B2Ug!6E8(Huj;RoZYJoukE7zhS+LI9<>S!R`>8Yy0hyX zjvm?){?{2*Z|=R=#xk3NM($=dq-8eM;s!I$hNLQG(P$;{<)eKJ<-WPohq@K^}A%YJiD)0Y{nQ_qg4-a>KgbRJV4%e zU2Q(m{b7Jd3%C_T?|Ax65@e3wpC-XnD%AQ2$W!V4aj0!q^SMVM?H~P-x4{hBAam2B zS0{3g`Mr;&CCbuX#mPa70G`_cR^E*qK=R09Al?$&Kc|Y5kZK%c^&!kRheSK3)YS>h zR8y5Ir=TmGFRS&`rBhvA0ikcxbr90~Oy`k@v15dsBoDSahTP3on;|(PDvK6S7?|$k zwDe2ZZH?)ep{!^{JLXz)X8L6tuZLhd$}Dm~cvM>U2FAn%oGiDCO)}o}o}@80b~ROf zRxqd6fBw(^^VUBh9*@Gmf?gLtEUqe7IjVS$`94UG2r+?O4WzoJ#Dg|+>iS5&=F(aJ z9%FAd#iZI|nJ-#wTUuQ1Q8C@ForN2neE1^V-Q!e?oN8#%0S;Zes$tM-Qx`5kyjU%p z*#_;FZgSCM;S`SB9X3AV=WShJCD-C+xn27&iVFUU&d6ABY6|K|LgzsVW!XD!C8Io8d{J4b+uS7ZmyTx=HC?K@e}xG0m|x|P}Shcc(qtvKk)>8 zDaLdx-h8>aS=G%i@wG1*raO$Oe)FY&<(Dq)8>OgsCcOT~X1cuol7aL+CU5YqFV$F# z+3{vJ%?J35;j4#q_~sEDYV%}rwOzbzTrx+snvLgQ-H7k(+h^lHHfQdx5<;H2_?7P6 zVFi`!^aSCI7f8z)0(NbDA0YLfx#D6cx_~1_j&||v3i#0XKFic~@x!A2uyXtQIhNo0 zXR;K8vaF@hmWaH08oU(75_`0Z*!n?NZ~So|FBh&Fn(GvSP1k7PpWvHsLXNN_f?4P( zW1N;Buf!dmYG129@Puc`J*UO_%~`R}9s)l=y5Q-y-8b`_j;*6&E zEnlO}(0h3cf2V^i#|-hcPN%yRj5{e5{8kDDF4-eyuy|^kc8TR>L;vp2YbU_m>R$re zJ57Kh$0z{($aIrnh1n;e6)g8ipr8}fp0UL46RlxJ#{8j>RT$Q zX}|qy8xXJkw^ogb@i79AHtTB9pz)FWEb@RA$G)!ob+B`-I9=5)?F8-IIO`Ulbcv!Kn9VIRldbdvROnNyq zF+w0$DewOI(~+N!{_;;pK2G%}CHfI@*B6I>{sbd>wYi3=!=HQ^b@$`f+r`BH^4r;> z#-DyzIIay>4F0D0MN_1DQv#uhex$Kk<6H($=W`eG(d>_{C^Q{g<$fxZ9Bbo$smE0K zpV(Mx|M=FQRBzT-00Euy&P?JTN=R&O_v5jja(aWel`EN1v{e+FPuUxlu1eCO`W}1Z zdMvGPj>)m5{+Dn`>-d=2UcDhiHI5A~=-s$tZwx)9zBx)`$LO5&>xQQumvuZIBUa*w z63t8ak52XmXBh4LAqH!&Ker9YSaaVS{22F!9%~&RyMCcJBS(VTQZPD_nk9P^hGR?! zeBezW(fr_=Q)Dce?@fXfBmFOd@rn4@@Ln@1JseB1hl1C%V>2{%u{R?8i#&KNXemTi zZ^*%2Z<|epX^74KOd8S7d zdJ8)Oen;a^g|%ZOhBA{mlh(T~iU0<4c-|=6EsTvjk(zOsN!X`u1)*VSM$jkh64TWi z95`pkE+*^!qaAvKVVH2B6!gY#o$2@(W|ET;D6a6Y9#NS5WN#ktwwRtD9yfoWK&}pZ z65bYJZ0(%7H+*kQlyLWt7>5o-@&r(-i0=)nB;P*WR*t0!d(yNWTQc?owr0$6yh>-D z018^Q>ut6;Uw*i~OU!OVz#TyJt(i2aNx)Brlmh0k zUMzCPCYb|rC}Q!}W7^MBUgrl?q^c!cnFmk^Kj!>RdT{d zoavJD^m$X#b7hxf8!{eUr+@P7io~O3$k?B5X~)F(X7t!{-KH*0U-0U-NmKVW0VBJvU(lNo_7SaPA$y;&E+RD|ftJ14+|B~p7FWN)Lo z$FL!V9!WA%?8LempwXc7#w6_htX%?vf?qd}*XU7^n#j4t*n}I$a?hm>xH60v z=ae%7`vdj~)7`xxM!(}FF!%c?hF($tj>uGdzeySwlhXKDhuah>7h z6!)a&?QIJ3BvrQ$VYnAyY$I4PgImOwl&5!z6|C*;u3Q4*fo%6AzkRylYIn2iuA7{7 z16Hb?6cl`x^W}EB8yl_+Z>1n5;$re3S@JOroXgmfSvYkBI2cXxNlv+g2gEP-0SWzi z)2&!{AS>_hlE9Uj8A(w}H8QeHp8#$xk?q*-o&Yrt>sEkVY@&|CBdhjT1oT5upo9Sm zK{{qq5bJN8_gBm6dUopsSksQ}*t*4^P%=kO0x~5C4ZsYOy#YZa%p{iZ^#H$dCOKlU z`-TN?Md2Cn-R)O2DeIQgGEVHs%W=V`tHo>#p(0b_H*RNGWdbN#iBFY?(_4 zkE7)BEH>3*(T=%grj%&ycvHlNUBF5ab)zSwAVV0 zO0RpD8tf85Z;uERE~J1f3vwhMX{k>1rjekrYh(d4D-(|OWiPL-FlFXb* zqdD=tEinOX*Ik&ZRzQYv&?7U66HP72?9Iw31a!;$Sc!?C&c$sT*RbN+kX7VHV23m<5n{(3Lz281^Z%VOX zRogoN^O-*Z#_skcMhQkeDo6^z*n+l`+nW|H&KtX%f?zUVF5h}x=TE+iRx)oTsrks-uHF#9$6KC2{%O+!nol&7R;Xy5*##puyYtra$$cqO9?Fvx1Z6rB z@?>v{a2J(T9H62!cXx?7qEW@4l*}BPLa>u4mu$-OYVvk@aS_*TuVLNMn*?5xWert~ zYT4><^@qt-wYY5cH>q0N>yCr$>%@if)|rw^P=-r^Qe{_focPlnW##sc`xLuG*#LL2 z!ED$0UUz}Xt9ag8FMtv8#z7Nbu$O>}ZzL!lAjBG#l*~0zP7{U?4lzUgr0vd|yX&F_ zBWS|-T=W^6)AOAbdr{4AHnYirglBClbD5(Sj8;~r7W8~G;u`KZW7563M140ngps^* zssC{%fl0N!u_&UzoKhPZn=wu<;Uh+5$rwE`HpX0ytzuFw-c?ORSZQ`Lmu)Q=)WpRM zD+s@1LC29|#H0q<9JDHtUN`0eE8W;G%l8Nlw;LN3tsZdf?xxn;%mpb8&006kz21Eb zKBkF|n&|D@9X(A!3OV3hN|##Y^=34W7W#5p&u8ykt%T;bqv06mijOIknR`+Q7vl?D z8ODUk{Lz*db)4M=W>cq|-m-;u^yaHL7nh+Dyz`zO;gcVQmuM-44!h*T-I?U=-3_v` zF$dEB&dno~D20+E&|rY0(2m{|@%J0_1|nDaeztb0x?yZw#1nuvtsvo<=m`eh)CPMA zXgS<)xaJ6`-s#hgv6W833kf3*zwVRGB)SBgg&th|Fnz(a9aSJ4 zn;B_sN0#W)lJwZLqH*2!>t8f?mZ-hoj(8t({W#iC5%jkO%3PNRS_hl}Dcj!edc=bD zTu`3bo8{gE4P)Df_4+jHX0RGh=nV@c#H_pB321$bt;uUQHsL+)#*Qc)0J>wAsop4& zZ12Wtw&&Ng%_kchRVMa^2DMvblkK_zFDCMqebNS(G3BbH2U{i88{<6QILMb*)wK0?b0w=CV8F_Clmd@2x`o4xxj znq;JoF&GDN3O2+kP@sB5xm55UmF`VQFDc+mmVtfk?f@8&X(ha9dlk`^hEJnEfR~cW z5n@SzN|@|TGO{PVB7j0r8XeoUaX?PT4qO@QY=vKvF^xVdi4vEUCHzOFd(*r4 z9h=bYp5(3gC1mUk`i&kj;I=T66s@qJfhK!X!s|GGY+hpC_x44b0*VSc+OC7lU`*@n zFlIP24UysNT|iBAbG-R0f+u zYf*6188^EPr(3tfG_57U$QUV+-J5X{R)x1;pxWIW9b4CJDtTsamdI8La$!NOp8qLk z(o#y}9Z<%a-8%_yU+KN=3HC;A+ecEcGXK!7#6T>eZrxnD?oD-`_CD-vGIw?f%biIc z`~jD<&D}_ykG@L;<8GIPLX5Y39Xnov_Bwf2!gID};EuVyY0pa%kvboFY;eE1Jqgm? z<$=4;T`6?u1ZeeN-g@Fh7%yCGr(sXQ>CyMZ6M|uye)7u>GkW!F75n z2E!`mfH{i2c!<4q6RjcUs2?Xh7AAKlmG-@P6VToT7vxE@({Wm*crOOSDU*23D70Rj z>`k-a$2k^X!a>ytU0?tpi_xyc&nf4vMx|*9S~W~4+;eGP2c|Yf`=mpR0Qq%u^~t!h z36VM|;o*H6DoB5#-VhBk_^8^D*IksYY8}Cf#t_&&LplnXJ$hq=NH3XJ)8*<;e2HmY z>W3Cr-NvL!5#zZ=l<*Pbz@S0APW48Ka=rPQcX#7UV&H`mF_L#LgwuFOr;{fI1$k0w zkSCQ%_olp#xP)C-eR~}@A)2zs18P7{$A*EYx6~}qFU({BTdg>e2L>bK?s{qW z#e*ol9-IKjsTsqVUQ~ojszldJnqV_WJ)r)uSyz!=2FIxBkQCx$iNUT}>t>acUiagI z9K=Ip7Bcb@9d&!76y0x98S zj|2fNU)W@Cl=)cfTNo9eTP%V4Y1b{r^n%iT zBtu5+8IucRjzn0w%qE@Nn^MB@Ne>xymn+A0gEf+OdLup#4`ZI-&X#nTx| zLYWN39W8f3 z{K7DP)rK?oi3?&Nn(LUiSRU+@Tk(_99PtT<$GMs?^k%I|@<~);_)d9J(JHCiY&)jf zyhV^BL6#J7>m>IiOr^Kb$I75^?DqTt=qko7AWn+xZsv z>l!w_vVR09ZCNwPg_nYkYE*avXdCfC+Dz6UK*M9J==w-_XUDEsUQV*Y?wk-JNT^4`A7+2(3{etVagmY3)@aC6rioR6qcNm`=INEkYT z>p9t*B#-PejmLmI!8zv%4q2dBiBuBM><<#-@e0V642a`>;1S$T6k* zSRPgM5hz<~k~Nctw))!=-F>a@5mPBt>!ppm#6~>;ffb~S{8lB`8n}rrg|n@tdb6yx zbFF6=n>!~US}^p^J-lzj^E1g%n#`F=Yea~9tu8K>^Xb8D>h8;SmoR;|Hy%!a^Si-q z%48{7QVN3oFXHK*E#56B)h1-;e+nrGgja794RD?W2+sg@g#Upqi$+i{Y|R4OLi8o7||$IDI{obhQQuT-T@NMn`z^M z_7&71!bMh$>0n1BJSRvn%p+GBcehlmok?IKd%H45gzlnj@93uP{&Rj)ShtKkD(KC+ znkvOXEuK$x0;JaB$H77Z@r{DG#9){F;upVjiBx|Gh;bM!y#mRhp89%l4mJ);P!x20 zyd|cD#!Q3n@FhmD_|$ogu&>j4wXP?wMxe)a#gz4n`SLx!f>p8bCD)1nR21hoMX2I4 zF21=y{MKSo10|cKyM>qQYCW6Z6o?BOdK0dGXz=H&4I)_Qh}C z{P61en;*V=_59fnFQ32p?!{Nn-~9dam#^T3zuBy}^#SzPJ7~N*pV#B!>*cz*234iN zdS2)$N6fLNuFs0A&1Th{J$VvD9Z)YWpZGKm{0XKgEdc5bE_Qg%lFG>Q~Ysx-h#~>eVtigG^pHt53Rb=Wle$2l5 zV4y6GN#cDA#}>MJy|*qHg|(hxo%-U}FMjj2!UB&?by4p=JKi5B+#nY_KJ}rsvJ9=4Wi_3;cE>w% zI?MefdPN{%eX>_qG6DXNF?l7{>_kdxbWU8>A6D?*y8ivRX|A66R*v^2h%{KP?|5Gr z^zDFA(dJcym=0aQz_zfH-ql3|f1MP6)rY7rZd3I|@h|^nakPc)0uDcxp7tLdeDkM2 z6=#2Sd_bC8i3M5iH+jesem{1M;rD01fAagl8-9;ILip?0xh1daYFe)oWEgJH5{gZX zZ!s8X<4Hyf)>6!#%N)pVx`ua&r4beV@is$`SIKTe#o#>G!O!^;V| z?RHfx*DGMpMNl3RTv`#Owf0iv`)ZBf{sAxc@gJKYNgT=$IKDXu5ZFh!KtuKKV(+u_ z?Z)Btdnp8FWa32#U$A)hrZ|g@JaK;L<#zMsJ27g1=>VtP)$M-mV5j}F(mjIT{zS6E N{|`vbuIT600{{Ux_3!`y literal 0 HcmV?d00001 diff --git a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts index 8f28620c0..be69dbae8 100644 --- a/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts +++ b/packages/codev/src/__tests__/pir-13-ci-concepts.test.ts @@ -265,6 +265,44 @@ describe.skipIf(!hasJq())('#13 — extraction returns the assertion, not the log expect(got.text).toContain('FAIL\tforgejo.org/modules/lfs'); }); + it('does NOT anchor on a PASSING test whose captured stderr contains a real Error line', () => { + // Found by running ci-failures against this repository's own red CI run + // (32536232930) while this PR was open. The extractor returned + // `Error: Refusing to POST /api/tunnel/disconnect ...` from inside a + // `stderr |` block belonging to a test that PASSED — that test asserts the + // error is thrown, so the text is the suite working as designed — while the + // real failure sat 355 lines further down. Same decoy class as the + // artifact-canvas one, arriving through a different door: this decoy IS + // anchored at the start of its line, so anchoring alone does not stop it. + const raw = fixtureLog('github-vitest-worker-crash'); + expect(raw, 'the fixture no longer carries the decoy').toContain('Refusing to POST /api/tunnel/disconnect'); + + const got = extract(raw)!; + expect(got.text).not.toContain('Refusing to POST'); + expect(got.rung).toBe('vitest-unhandled'); + expect(got.text).toContain('Unhandled Error'); + expect(got.text).toContain('Worker forks emitted error'); + }); + + it('reads a captured block as capture until the blank line that ends it', () => { + const log = [ + 'stderr | some.test.ts > a suite > a passing case', + 'Error: this is expected output from a passing test', + ' at somewhere', + '', + ' ✓ some.test.ts (3 tests) 4ms', + ' ✓ other.test.ts (1 test) 2ms', + ' ✓ third.test.ts (2 tests) 3ms', + ' ✓ fourth.test.ts (2 tests) 3ms', + 'Error: this one is real', + 'more', + ].join('\n'); + const got = extract(log)!; + expect(got.rung).toBe('first-error'); + expect(got.text).toContain('this one is real'); + expect(got.text).not.toContain('expected output from a passing test'); + }); + it('returns NOTHING when nothing is recognisable, rather than an arbitrary slice', () => { const noise = Array.from( { length: 400 }, From 781747f819e77e912b31ba870334f5435bbc29d7 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 17:45:06 -0600 Subject: [PATCH 28/30] [PIR #13] docs: correct the --log-failed overstatement, and record all four lane verdicts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The claude lane's third pass returned APPROVE with four documentation-accuracy notes. The one that mattered: this PR claimed gh run view --log-failed ALWAYS tags lines UNKNOWN STEP. Verified, and it does not — on run 32515040122 all 2528 lines are UNKNOWN STEP, on 32536232930 all 1193 are attributed correctly. The attribution is unreliable, not absent. Nothing about the design changes, since attributed or not the output is a whole job or a whole step (293 KB and 108 KB) and never the assertion — but the strong claim was wrong and is now corrected here, in arch.md, and in both script comments that stated it. Also: fixture count 2 -> 3, commit list regenerated, and an explicit reason why the unsupported-server envelope says failingJobs rather than failures (those entries carry no extract; reusing the key would shape an unsupported server like a successful extraction with the details missing). Coverage table now records all four lanes with their real outcomes, and the rotation as it stood when the review ran. Co-Authored-By: Claude Opus 5 --- codev/resources/arch.md | 2 +- codev/reviews/13-ci-forge-concepts.md | 49 +++++++++++++++------ packages/codev/scripts/forge/_ci-extract.sh | 5 ++- packages/codev/scripts/forge/github/_lib.sh | 6 ++- 4 files changed, 45 insertions(+), 17 deletions(-) diff --git a/codev/resources/arch.md b/codev/resources/arch.md index 6291bbb8b..e5cbf0211 100644 --- a/codev/resources/arch.md +++ b/codev/resources/arch.md @@ -2114,7 +2114,7 @@ Shared implementation, so the two providers cannot drift: `scripts/forge/_ci-ext **CI behavior verified live against GitHub (gh 2.87.0), Forgejo 15.0.2 and Forgejo 16.0.0-dev** (PIR #13). Each of these looks like something else and is not: -- **`gh run view --log-failed` does not narrow to the failing step.** It selects the failing JOB and returns all of it — 2528 lines / 293 KB on the reference run, with every line tagged `UNKNOWN STEP` because gh's filename-to-step mapping had missed. Both providers therefore fetch `actions/jobs/{id}/logs` and codev extracts; the failing step NAME comes from `gh run view --json jobs`. +- **`gh run view --log-failed` does not return an extract, and its step attribution is unreliable.** Measured on two runs of this repository: on 32515040122 all 2528 lines came back tagged `UNKNOWN STEP` (gh maps log files to steps by name and falls back when that misses); on 32536232930 the same command attributed all 1193 lines to the failing step correctly. **Either way the output is a whole job or a whole step** — 293 KB and 108 KB respectively — never the assertion. So codev extracts on both providers, and takes the failing step NAME from `gh run view --json jobs`, which is structured and does not depend on that mapping. - **Forgejo has no Actions job-log API before 16.0** (released 2026-07-16). On 15.x there is no token-reachable log by any route: `tea actions runs view` / `runs logs` both 404, and the web UI's log route is session-only, rejecting an API token and basic auth alike. `ci-failures` / `ci-run-log` return `unsupported-server` there naming both versions; `ci-runs` / `ci-run-view` keep working. - **Forgejo ignores `limit` unless `page` is also sent** — `actions/runs?limit=3` returned all 6922 runs. `status=` filters server-side; **`branch=` and `event=` are silently ignored**. - **A `pull_request` run records `head_branch` as `#`**, not a branch name, so branch filtering resolves the branch to its PR first (the #12 base/head lookup) and filters client-side. diff --git a/codev/reviews/13-ci-forge-concepts.md b/codev/reviews/13-ci-forge-concepts.md index 67f71f04f..6bea30bc5 100644 --- a/codev/reviews/13-ci-forge-concepts.md +++ b/codev/reviews/13-ci-forge-concepts.md @@ -17,6 +17,8 @@ Unit Tests UNKNOWN STEP 2026-08-21T18:47:09.5820646Z Current runner version: '2. Every one of those 2528 lines is tagged `UNKNOWN STEP`. `--log-failed` selects the failing **job** and returns all of it; `gh` maps log files to steps by name and falls back to `UNKNOWN STEP` when that mapping misses. The architect independently reproduced this on run `32448538074`: 919 lines, all 919 tagged `UNKNOWN STEP` — and had read that same output earlier the same day while diagnosing #6 without registering what it meant. +**One precision, raised by the claude lane and verified:** that attribution is *unreliable*, not always absent. On run `32536232930` the same command attributed all 1193 lines to the failing step correctly. It changes nothing about the design — attributed or not, what comes back is **a whole job or a whole step**, 293 KB and 108 KB respectively, and never the assertion — but "always UNKNOWN STEP" would have been an overstatement, so it is not claimed here or in `arch.md`. + So codev extracts on **both** providers, and neither uses `--log-failed`. Both fetch `actions/jobs/{id}/logs` — one job, no invented step column, the same shape Forgejo 16 serves — which is also why they share one cache and one extractor. The failing step *name* comes from `gh run view --json jobs`, which is structured and reliable. Anyone reading #13 later should read this section instead of its "Provider notes". @@ -85,7 +87,7 @@ No log lines at all. A builder handed 50 arbitrary lines treats them as the diag Every ci-* concept prints one JSON object on stdout on success **and** failure, so `timeout` / `not-found` / `unsupported-server` / `forge-error` / `bad-input` stay distinguishable after `executeForgeCommand` has flattened everything else to `null`. `executeForgeCommandDetailed` is added for callers that need the distinction in TypeScript: it returns `{ok, data, stdout, stderr, exitCode, timedOut, unavailable, durationMs}` and keeps stdout on the failure path. -On a Forgejo below 16 the response is `unsupported-server`, naming the version found and the version needed **and still listing the failing job names it could determine** — never an empty `failures` array. "Your CI is fine" and "I cannot see your CI at all" are opposite facts and must not be the same observation. +On a Forgejo below 16 the response is `unsupported-server`, naming the version found and the version needed **and still listing the failing job names it could determine** — never an empty `failures` array. It calls that list `failingJobs`, not `failures`, and the difference is deliberate: a `failures` entry carries an extract (`matchedBy`, `text`, `from`/`to`, `returnedLines`), and these have none — only a name and an id. Reusing the key would make an unsupported server shaped like a successful extraction with the details missing, which is a smaller version of the same lie the envelope exists to prevent. (Raised as an inconsistency by the claude lane; kept, with the reason stated here.) "Your CI is fine" and "I cannot see your CI at all" are opposite facts and must not be the same observation. ### `codev forge ` @@ -106,7 +108,7 @@ Added at the architect's direction at the dev-approval gate, and the reasoning i - `packages/codev/src/commands/forge.ts` (+97 / -0) — `codev forge ` - `packages/codev/src/cli.ts` (+19 / -0) - `packages/codev/src/__tests__/pir-13-ci-concepts.test.ts` (+976 / -0) -- `packages/codev/src/__tests__/fixtures/pir-13/{github-vitest-failure,forgejo-go-failure}.log.gz` (2 files, 57 KB) +- `packages/codev/src/__tests__/fixtures/pir-13/{github-vitest-failure,forgejo-go-failure,github-vitest-worker-crash}.log.gz` (3 files, 98 KB) — the third is this branch's own red CI run, which produced the capture-block decoy - `packages/codev/src/__tests__/forge.test.ts` (+10 / -2) — concept count 18 → 22 - `packages/codev/src/__tests__/spec-1280-measurement-instrument.test.ts` (+18 / -25) — timeout ceiling, see Flaky Tests - `.claude/skills/forge/SKILL.md`, `.codex/skills/forge/SKILL.md` (+133 / -0 each, byte-identical twins) @@ -125,7 +127,13 @@ Added at the architect's direction at the dev-approval gate, and the reasoning i - `8e26a661a` fix(forge): reject a non-numeric run or job id before it reaches a URL - `067f7b179` test(forge): the concept count is 22, not 18 - `9893db9cd` test: give the prompt-surface instrument a ceiling above its own cost -- `3334bf9e8` feat(cli): codev forge \ — run a concept through the real resolver +- `3334bf9e8` feat(cli): codev forge — run a concept through the real resolver +- `5f57f107e` Review + retrospective +- `79fb7b664` docs: record the review-lane coverage gap and porch's wrong remedy +- `8a57b262c` fix: the two defects the claude review lane found +- `9a5a19bac` docs: test counts after the review-lane fixes (5572 passed, 67 new) +- `5a61226d6` fix: an unusable TMPDIR must not be reported as a missing run +- `b1f8c7fce` fix(forge): the extractor pointed at a passing test ## Test Results @@ -134,7 +142,9 @@ Added at the architect's direction at the dev-approval gate, and the reasoning i ### Branch CI: red, and NOT because of a failing test -**Read this before reading `npm test: ✓ pass` above.** That line is the local suite and it is true; the branch's own CI is a separate claim, and at the time of writing it was **red**. The claude review lane caught the review presenting it as resolved when it was not. Here is what it actually is. +**Read this before reading `npm test: ✓ pass` above.** That line is the local suite; the branch's own CI is a separate claim, and for several commits it was **red** while this file said nothing about it. The claude review lane caught that. + +**CI is green at HEAD** (`b1f8c7fc`: `Tests` ✓, `CLI Integration Tests` ✓), which confirms the diagnosis below — the red was an intermittent worker-teardown crash, not a failing test. The guard defect that turned that flake into a hard failure is still there for whoever hits it next. The last red run (`32536232930`) reports: @@ -160,7 +170,7 @@ if grep -q "Test Files.*passed" /tmp/vitest-output.txt && ! grep -q "failed" /tm | 1657, 1663, 1664 | `spec-1470` test names and stdout — `reentry-failed`, `clear-failed`, "reports a **failed** Tower send" | | 2202, 2205 | `git fetch … failed` warnings printed by consult tests | -So *any* worker crash in this repository is a hard CI failure regardless of the test results, and the tolerance the workflow author wrote has never been reachable. That is a defect in `test.yml`, not in this diff, and it is **not fixed here** — fixing another team's CI gate to turn an unrelated red green is the scope creep the review phase warns against. It is reported to the architect with this evidence. +So *any* worker crash in this repository is a hard CI failure regardless of the test results, and the tolerance the workflow author wrote has never been reachable — which is why a flake that later cleared on a re-run cost this PR two red runs and a diagnosis. That is a defect in `test.yml`, not in this diff, and it is **not fixed here** — fixing another team's CI gate to turn an unrelated red green is the scope creep the review phase warns against. It is reported to the architect with this evidence. What this PR *did* cause, and has fixed, is the earlier red: the `TMPDIR` harness bug above, which failed 31 tests on Linux while passing locally. @@ -204,13 +214,18 @@ Also verified against a second real run (`32448538074`, the architect's): 919 li ## ⚠ Review lane coverage — read this before trusting the review depth -**As of 2026-08-21 ~17:00 MDT, two of the three lanes could not run, and the PR is being HELD rather than merged on the remainder.** +**The rotation this review ran under was `["gemini", "codex", "claude", "opencode"]`**, the four-lane list in effect at the time (2026-08-21, ~16:50–17:30 MDT). The owner removed `gemini` from the rotation shortly afterwards — `["codex", "claude", "opencode"]` — so a later reader will see three names where this table has four. The table records what actually ran, not the current config. -| Lane | Verdict | Why | +The first pass had only ONE lane available and the PR was **held** rather than merged on it; the `opencode` lane (PR #24) was merged and installed mid-flight, which is what made a second live reviewer possible. + +| Lane | Verdict | Notes | |---|---|---| -| **codex** (gpt-5.6-sol) | **NEVER RAN** | Provider quota. Refused in seconds, before any model work: *"You've hit your usage limit… try again at Aug 27th, 2026 4:01 PM."* The same quota blocked codex on #2, #4, #11 and #12 the same day. | -| **gemini** (agy) | **NEVER RAN** | Provider quota, **not** the reason porch reported — see below. Resets ~2026-08-28. | -| **claude** (opus-5) | in progress at time of writing | — | +| **claude** (`claude-opus-5`) | **APPROVE**, HIGH | Ran twice more than required. First pass found the timeout inversion and the `sed -n "1,0p"` portability bug; second pass, on the corrected code, found the extractor pointing at a passing test and this file claiming green CI while the branch was red. Third pass, after those fixes: APPROVE, "no blocking issues", shellcheck and tsc clean, review claims verified independently. | +| **opencode** (`xai/grok-4.6`) | **APPROVE**, HIGH | No key issues. Checked the three documented deviations rather than the code alone. | +| **codex** (gpt-5.6-sol) | **NEVER RAN** | Provider quota, refused in seconds: *"You've hit your usage limit… try again at Aug 27th, 2026 4:01 PM."* Same quota blocked #2, #4, #11 and #12 the same day. | +| **gemini** (agy) | **NEVER RAN** | Provider quota, **not** the reason porch reported — see below. Resets ~2026-08-28. Removed from the rotation by the owner after this review ran. | + +Every finding from every lane was reproduced before being acted on, and none was argued down. The four that changed the code are described in **What the lanes found** below. **Porch's own gate summary will say otherwise, and it is wrong.** Both lane files carry `VERDICT: SKIPPED`, which `parseVerdict` (`porch/verdict.ts`) does not recognise — it knows only `APPROVE`, `REQUEST_CHANGES`, `COMMENT` — so it falls through to the "treat as COMMENT" default, and `allApprove` counts `COMMENT` as approval. Porch will print **"All reviewers approved!"** over two reviewers that read nothing. That is **#20**, filed by PIR #12; it is porch behaviour, not anything in this diff, and it is not fixed here. Read this table, not the summary line. @@ -236,9 +251,9 @@ rc=1 Reinstalling and signing in again would have changed nothing and cost whoever followed the advice their afternoon. A confidently printed remedy that cannot work is the same defect class as #21, where the stuck-mailbox alert names a command that cannot clear a composer. The architect is filing it separately. -### What the one lane that did run found +### What the lanes found -**claude (opus-5) — VERDICT: COMMENT, CONFIDENCE: HIGH.** Two real defects, both fixed before this was written, and both in precisely the class the two absent lanes exist to catch. +**First pass — claude, COMMENT/HIGH.** Two real defects, both in precisely the class the absent lanes exist to catch. **1. The timeout layering was inverted, and my own test hid it.** `executeForgeCommandDetailed` defaults to a 30s ceiling; the scripts default to a 60s `CODEV_FORGE_TIMEOUT`. **At the defaults the outer kill fires first**, so a stalled forge arrived as a generic Node kill and the script's *named* timeout envelope — the entire point of the inner watchdog, and the thing #17 and #8 are about — never printed. The comment in `forge.ts` asserted the opposite ordering. @@ -250,7 +265,15 @@ The reason it survived to review is worth more than the fix: **the timeout test A third finding was cosmetic (a misindented `exit` and a trailing space in `gitea/ci-runs.sh`), fixed. -A fourth was noted rather than requested: the `Ci*` contracts in `forge-contracts.ts` are documentation-only, with no conformance test tying them to actual script output. **Deliberately not done here.** Adding it for the four CI contracts alone would leave the other eighteen forge contracts untested while implying they were covered — worse than uniformly untested. The architect is filing it as its own issue across all forge contracts. +**Second pass — claude, REQUEST_CHANGES/HIGH**, on the code after those fixes. Three findings, all correct: + +- **This file claimed CI was green when the branch was red.** Addressed in the CI section above: disclosed, diagnosed, and the guard defect reported rather than fixed. +- **`ci-failures` pointed at a passing test** — the capture-block decoy, above. The best finding of the review, because it was found by *running this PR's tool against this PR's own failing CI*. +- **"How to Test Locally" gave commands that do not work here.** The globally installed `codev` predates this PR and has no `forge` subcommand; the worktree build predated the `opencode` lane and rejected the workspace config. `main` is now merged (fixing the second) and the instructions build the branch and invoke its own CLI (fixing the first). + +**Third pass — claude, APPROVE/HIGH.** No blocking issues; four minor notes, all documentation accuracy, all applied: the fixture count (2 → 3), the commit list, the `--log-failed` overstatement corrected above and in `arch.md`, and the reason `failingJobs` is deliberately not spelled `failures`. + +A further note was noted rather than requested: the `Ci*` contracts in `forge-contracts.ts` are documentation-only, with no conformance test tying them to actual script output. **Deliberately not done here.** Adding it for the four CI contracts alone would leave the other eighteen forge contracts untested while implying they were covered — worse than uniformly untested. The architect is filing it as its own issue across all forge contracts. ### What CI on this PR found that the local suite could not diff --git a/packages/codev/scripts/forge/_ci-extract.sh b/packages/codev/scripts/forge/_ci-extract.sh index baeb75329..7e70b7d9c 100644 --- a/packages/codev/scripts/forge/_ci-extract.sh +++ b/packages/codev/scripts/forge/_ci-extract.sh @@ -13,7 +13,10 @@ # JOB and returned all of it. (The architect independently reproduced this on # run 32448538074: 919 lines, all 919 tagged UNKNOWN STEP.) `gh` attributes log # lines to steps by matching filenames, and falls back to UNKNOWN STEP when that -# mapping fails. So codev extracts on GitHub too, and the concepts fetch +# mapping fails — unreliably, not always: on run 32536232930 it attributed all +# 1193 lines correctly. Attributed or not, the output is a whole job or a whole +# step and never the assertion, which is 108 KB at best. So codev extracts on +# GitHub too, and the concepts fetch # `repos/{owner}/{repo}/actions/jobs/{id}/logs` instead — one job, no invented # step column, and the same shape Forgejo 16 serves. # diff --git a/packages/codev/scripts/forge/github/_lib.sh b/packages/codev/scripts/forge/github/_lib.sh index 679eab799..f62d99b08 100755 --- a/packages/codev/scripts/forge/github/_lib.sh +++ b/packages/codev/scripts/forge/github/_lib.sh @@ -33,8 +33,10 @@ GH_FAILED_JOBS_JQ='[.jobs[] | select(.conclusion == "failure" or .conclusion == # Uses `gh api repos/{owner}/{repo}/actions/jobs/{id}/logs` — NOT # `gh run view --log-failed`. The reason is measured, and it contradicts the # issue that asked for this work: --log-failed returned 2528 lines / 293 KB for -# one failing job on run 32515040122, every line tagged "UNKNOWN STEP", because -# it selects the failing JOB and cannot always attribute lines to steps. The +# one failing job on run 32515040122, every line tagged "UNKNOWN STEP". Its step +# attribution is unreliable rather than always absent — on run 32536232930 the +# same command attributed all 1193 lines correctly — but attributed or not, what +# comes back is a whole job or a whole step, never the assertion. The # per-job endpoint returns the same bytes without the invented step column, for # exactly the job asked for, and mirrors the shape Forgejo 16 serves at # `actions/jobs/{id}/logs` — so both providers share one cache and one From f833b0629a77d6816783e084766dd2d095a8db58 Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 17:45:10 -0600 Subject: [PATCH 29/30] chore(porch): 13 pr gate-requested --- .../projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 826b90413..7179808b8 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -15,13 +15,15 @@ gates: approved_at: '2026-08-21T22:39:29.289Z' pr: status: pending + requested_at: '2026-08-21T23:45:10.673Z' iteration: 1 -build_complete: true +build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T22:50:10.139Z' +updated_at: '2026-08-21T23:45:10.673Z' pr_history: - phase: review pr_number: 23 branch: builder/pir-13 created_at: '2026-08-21T22:50:04.111Z' +pr_ready_for_human: true From 514cd776dd7d7f10046fba33bcd951767000fc1c Mon Sep 17 00:00:00 2001 From: pseudo Date: Fri, 21 Aug 2026 17:55:10 -0600 Subject: [PATCH 30/30] chore(porch): 13 pr gate-approved --- .../projects/13-add-ci-concepts-to-the-forge-l/status.yaml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml index 7179808b8..a1dbc0741 100644 --- a/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml +++ b/codev/projects/13-add-ci-concepts-to-the-forge-l/status.yaml @@ -14,16 +14,17 @@ gates: requested_at: '2026-08-21T22:38:21.754Z' approved_at: '2026-08-21T22:39:29.289Z' pr: - status: pending + status: approved requested_at: '2026-08-21T23:45:10.673Z' + approved_at: '2026-08-21T23:55:10.108Z' iteration: 1 build_complete: false history: [] started_at: '2026-08-21T20:35:23.798Z' -updated_at: '2026-08-21T23:45:10.673Z' +updated_at: '2026-08-21T23:55:10.109Z' pr_history: - phase: review pr_number: 23 branch: builder/pir-13 created_at: '2026-08-21T22:50:04.111Z' -pr_ready_for_human: true +pr_ready_for_human: false