ci: Comment the Vercel build log on PRs whose build fails by marcleblanc2 · Pull Request #1946 · sourcegraph/docs

marcleblanc2 · 2026-09-11T17:24:47Z

Problem

When a PR's Vercel build fails, the Vercel bot comment and the Vercel check link to the Vercel inspector, which only Vercel team members can open
Contributors who aren't Vercel team members see a red X and a login wall, with no hint what broke
Example: test: PR checks on a broken PR, left broken (do not merge) #1913

Solution

On vercel.deployment.error (the same repository_dispatch channel preview-links.yml uses), comment the tail of the build log on the PR Vercel built the deployment for (meta.githubPrId), like the links / redirects / CSpell checks do. Each PR gets its own deployment, so two PRs at the same commit each get their own comment
On the next vercel.deployment.success, update that comment to ✅ on every open PR at the commit that has one. The success path only reads PR comments; it does not touch Vercel
Lines that look like credentials (JWTs, vcp_ / ghp_ / github_pat_ / AKIA… tokens, Bearer …, *TOKEN= / *SECRET= / *KEY= assignments) are redacted before the log is written, since the comment and artifact are public and the build gets VERCEL_OIDC_TOKEN, VERCEL_DEPLOYMENT_KEY and VERCEL_ENV_ENC_KEY. The code fence is sized longer than any backtick run in the log, so a log line cannot close it and inject Markdown

Token

VERCEL_TOKEN repo secret is a project-scoped token for sourcegraph-docs only, expiring 2026-12-10. Vercel has no read-only scope; project scope is the narrowest it offers, and it can read and write everything in that one project
Enumerated with read-only calls:
- Allowed: project settings, environment variables (/env, including ?decrypt=true; the project has none), deployments list, deployment metadata, build log events, domains, custom environments, project list (returns only this project)
- Denied: team, team members, user, log drains, webhooks, Edge Config, other projects, minting tokens
- Writes within the project (env vars, deployments, domains, settings) are allowed per Vercel's docs; not exercised
fetch-log asks GitHub for an open PR from this repository at the commit before it contacts Vercel, so a dispatch for a fork PR or a stale commit never uses the token. repository_dispatch runs the workflow and script from main, so a PR can't change the code the token is handed to
Anyone with write access can read any repo secret by pushing a workflow; this token limits what that buys them to one Vercel project

Tested

Run locally against the two example PRs, which is how the comments there got posted (from my account, since GitHub neither delivers repository_dispatch nor resolves workflow_dispatch for a workflow that isn't on main yet):
- test: PR checks on a broken PR, left broken (do not merge) #1948: ❌ build log comment
- test: PR checks on a broken PR, then remediated (do not merge) #1949: ❌ posted on a broken revision, then updated to ✅ after the fixed revision built
- A PR with no failed build exits with has no failed build to resolve
That test found a bug: with two open PRs at the same head SHA, only the first PR the commits/{sha}/pulls API listed got the comment. Vercel records the PR a deployment was built for (meta.githubPrId), so fetch-log now reads it and the comment lands on that PR only. The success path stays off Vercel and keeps the commit lookup, since it only updates comments that already exist
Re-ran fetch-log after the reorder: test: PR checks on a broken PR, left broken (do not merge) #1948's head → pull_request=1948; a commit with no open PR stops at No open PR with head … with VERCEL_TOKEN=invalid, proving Vercel was not contacted. Redaction and fence sizing checked against JWT, vcp_, ghp_, AKIA…, Bearer, KEY: value lines and a log line of six backticks
After merge, re-run on a PR by hand: gh workflow run vercel-build-report.yml -f id=dpl_... -f state=error -f sha=<pr head sha>

Dry-run output (abridged)

❌ The Vercel build failed for this PR

Vercel only shows build logs to members of its team, so here is the end of the log.
Run npm run build locally to reproduce.

Build log
Running build in Cleveland, USA (East) – cle1
...
❌ Found 2 dead link(s) in 1 file(s):
📄 docs/code-search/features.mdx
   Line 154: /code-search/no-such-page
...
❌ Failed checks: links, filenames
 ELIFECYCLE  Command failed with exit code 1.
Error: Command "pnpm run build" exited with 1

Amp thread

Vercel build failure report

Merge order for the PR-check stack

Trial-merged onto main in this order with no conflicts:

ci: Comment the Vercel build log on PRs whose build fails #1946 Vercel build log comment — independent; first so the other PRs' Vercel failures get a readable log
check-links: one suggestion per fix, synced with findings; one fact per line in reports #1916 check-links report format — adds dev/sync-review-comments.sh, which ci/redirects: Add Redirect check for PRs #1935 calls
ci/redirects: Add Redirect check for PRs #1935 redirect check — needs check-links: one suggestion per fix, synced with findings; one fact per line in reports #1916 merged first
spell check: update inline comments whose text changed #1947 spell check comment updates — independent
check-links: don't fail the generated-docs sync PR over absolute self-links it can't fix #1944 check-links, generated-docs sync PR — conflicts with check-links: one suggestion per fix, synced with findings; one fact per line in reports #1916 on dev/check-links.mjs; rebase after check-links: one suggestion per fix, synced with findings; one fact per line in reports #1916 merges

Squash-merge each, then rebase the next onto main.

#1948 (broken) and #1949 (fixed) are the example PRs that exercise every check; never merge, close them once the stack has landed.

vercel · 2026-09-11T17:24:52Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
sourcegraph-docs	Ready	Preview	Sep 11, 2026 6:29pm UTC

Vercel shows build logs only to members of its team, so contributors saw a red X and a login wall. On vercel.deployment.error this posts the tail of the build log on the PR; on the next successful build the same comment is updated to say the failure is fixed. Fork PRs are skipped so the project-scoped Vercel token is never used on their behalf.

…dispatch needs the file on main Two open PRs at the same head SHA got one comment on whichever PR the commits/{sha}/pulls API listed first. A deployment belongs to a commit, so comment on each open PR at that head, fetching the build log once. GitHub only resolves workflow_dispatch for workflows on the default branch (gh workflow run --ref <branch> 404s before merge), so the header now says to run the script locally until then. Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>

Amp-Thread-ID: https://ampcode.com/threads/T-01a09014-dfa8-740c-95b4-9e28c43cae51 Co-authored-by: Amp <amp@ampcode.com>

…omment cannot hold it The comment says how many lines the log has and that the last 100 are shown, matching tail -n 100. When that leaves lines out, the full log is uploaded as a workflow artifact and linked from the comment; the artifact ID rides in the comment marker so the run that reports the build passing deletes it. Amp-Thread-ID: https://ampcode.com/threads/T-01a09014-dfa8-740c-95b4-9e28c43cae51 Co-authored-by: Amp <amp@ampcode.com>

…e log

… for A deployment is built for one branch, and Vercel records that PR in the deployment's meta.githubPrId. fetch-log reads it and hands it to the comment step as pull_request, so two PRs at the same commit no longer both get the log. The success path stays off Vercel and keeps the commit lookup, since it only updates comments that already exist. The open / same-repo / head-is-this-commit checks apply either way. Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>

…using the Vercel token fetch-log asked Vercel which PR the deployment was for, then checked GitHub that an open PR from this repository is at the commit. Swap the order, so a dispatch for a fork PR or a stale commit never spends the Vercel token. Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>

…dential shapes A log line of four or more backticks closed the fixed fence and let the rest of the log render as Markdown in the bot comment. Make the fence one backtick longer than the longest run in the tail. The comment and artifact are public and the build gets VERCEL_OIDC_TOKEN, VERCEL_DEPLOYMENT_KEY and VERCEL_ENV_ENC_KEY, so a build that prints its environment would publish them. Redact JWTs, known token prefixes, Bearer values and TOKEN/SECRET/PASSWORD/KEY assignments before the log is written. Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>

github-actions · 2026-09-11T18:28:17Z

⚠️ Spell check found 3 issue(s) in this PR

Only findings on lines added by this PR are shown.

dev/report-vercel-build.mjs

line 162, column 45
- abpr → abbr
line 162, column 65
- AKIA → kaia
line 167, column 24
- PASSW → pass

Please correct the spelling, or add words which are correct to cspell-allow-list.txt.

Run npx cspell@10 --no-progress --dot '**/*' locally to check the full repository.

github-actions · 2026-09-11T18:28:18Z

+const REDACTIONS = [
+	[/\beyJ[\w-]{10,}\.[\w-]{10,}\.[\w-]+/g, '[redacted-jwt]'],
+	[
+		/\b(?:vcp_|gh[pousr]_|github_pat_|sk-|xox[abpr]-)[\w-]{16,}|\bAKIA[0-9A-Z]{16}\b/g,


abpr is not in the dictionary.

Did you mean abbr?

Suggested change

/\b(?:vcp_|gh[pousr]_|github_pat_|sk-|xox[abpr]-)[\w-]{16,}|\bAKIA[0-9A-Z]{16}\b/g,

/\b(?:vcp_|gh[pousr]_|github_pat_|sk-|xox[abbr]-)[\w-]{16,}|\bAKIA[0-9A-Z]{16}\b/g,

Please correct the spelling, or add the word to cspell-allow-list.txt if it is correct.

github-actions · 2026-09-11T18:28:18Z

+const REDACTIONS = [
+	[/\beyJ[\w-]{10,}\.[\w-]{10,}\.[\w-]+/g, '[redacted-jwt]'],
+	[
+		/\b(?:vcp_|gh[pousr]_|github_pat_|sk-|xox[abpr]-)[\w-]{16,}|\bAKIA[0-9A-Z]{16}\b/g,


AKIA is not in the dictionary.

Did you mean kaia?

Suggested change

/\b(?:vcp_|gh[pousr]_|github_pat_|sk-|xox[abpr]-)[\w-]{16,}|\bAKIA[0-9A-Z]{16}\b/g,

/\b(?:vcp_|gh[pousr]_|github_pat_|sk-|xox[abpr]-)[\w-]{16,}|\bkaia[0-9A-Z]{16}\b/g,

Please correct the spelling, or add the word to cspell-allow-list.txt if it is correct.

github-actions · 2026-09-11T18:28:18Z

+	],
+	[/(\bBearer\s+)\S+/gi, '$1[redacted]'],
+	[
+		/(\w*(?:TOKEN|SECRET|PASSW(?:OR)?D|CREDENTIALS?|API_?KEY|PRIVATE_KEY|ENC_KEY|DEPLOYMENT_KEY)\w*["']?\s*[=:]\s*["']?)\S+/gi,


…er line in reports (#1916) Follow-ups from testing the PR checks on #1913 / #1914. - Review comments: one suggested change per finding (not one per line), no review body. Each comment starts with an HTML marker; the workflow deletes suggestions whose finding is gone (or that GitHub could no longer place, `line: null`) and skips ones already posted, so resolved suggestions disappear like the spell check's do. - Summary comment and review comments put line, link, problem, and fix each on their own line. - Absolute links to this site get their own **Absolute links** section instead of being lumped into Outbound. - Case-mismatch findings now come with a fix (`/Code-Search/queries` → `/code-search/queries`). - Wording: "Write links on this site as relative paths", "fix the inbound links on these other pages", dropped `https://docs.sourcegraph.com/…`; reproduce command matches `package.json` (`pnpm check links …`). Tested locally against the `test-pr-checks-broken` branch with the CI recipe (baseline from `origin/main`, `--diff`, `--review`); build-mode run (`node dev/check-links.mjs`) still clean. Trade-off: when several fixes sit on one line, applying one suggestion outdates the others until the next run re-posts them, because GitHub will not batch overlapping suggestions.  ## Merge order for the PR-check stack Trial-merged onto `main` in this order with no conflicts: 1. #1946 Vercel build log comment — independent; first so the other PRs' Vercel failures get a readable log 2. #1916 check-links report format — adds `dev/sync-review-comments.sh`, which #1935 calls 3. #1935 redirect check — needs #1916 merged first 4. #1947 spell check comment updates — independent 5. #1944 check-links, generated-docs sync PR — conflicts with #1916 on `dev/check-links.mjs`; rebase after #1916 merges Squash-merge each, then rebase the next onto `main`. #1948 (broken) and #1949 (fixed) are the example PRs that exercise every check; never merge, close them once the stack has landed. --------- Co-authored-by: Amp <amp@ampcode.com>

marcleblanc2 mentioned this pull request Sep 11, 2026

ci: Comment the Vercel build log on PRs whose build fails #1918

Closed

vercel Bot deployed to Preview September 11, 2026 17:43 View deployment

marcleblanc2 and others added 6 commits September 11, 2026 12:01

ci/vercel-build-report: Reword the failure comment

b33004f

Amp-Thread-ID: https://ampcode.com/threads/T-01a09014-dfa8-740c-95b4-9e28c43cae51 Co-authored-by: Amp <amp@ampcode.com>

ci/vercel-build-report: Plain wording when the comment holds the whol…

f96dedb

…e log

marcleblanc2 force-pushed the marc/ci/vercel-build-failure-report branch from b665669 to e27f008 Compare September 11, 2026 18:01

vercel Bot deployed to Preview September 11, 2026 18:04 View deployment

marcleblanc2 and others added 2 commits September 11, 2026 12:27

github-actions Bot reviewed Sep 11, 2026

View reviewed changes

vercel Bot deployed to Preview September 11, 2026 18:29 View deployment

Navigation Menu

Uh oh!

ci: Comment the Vercel build log on PRs whose build fails - #1946

ci: Comment the Vercel build log on PRs whose build fails#1946
marcleblanc2 wants to merge 8 commits into
mainfrom
marc/ci/vercel-build-failure-report

marcleblanc2 commented Sep 11, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Sep 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 11, 2026

Uh oh!

github-actions Bot Sep 11, 2026

Uh oh!

github-actions Bot Sep 11, 2026

Uh oh!

github-actions Bot Sep 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	/\b(?:vcp_\|gh[pousr]_\|github_pat_\|sk-\|xox[abpr]-)[\w-]{16,}\|\bAKIA[0-9A-Z]{16}\b/g,
	/\b(?:vcp_\|gh[pousr]_\|github_pat_\|sk-\|xox[abbr]-)[\w-]{16,}\|\bAKIA[0-9A-Z]{16}\b/g,

	/(\w(?:TOKEN\|SECRET\|PASSW(?:OR)?D\|CREDENTIALS?\|API_?KEY\|PRIVATE_KEY\|ENC_KEY\|DEPLOYMENT_KEY)\w["']?\s[=:]\s["']?)\S+/gi,
	/(\w(?:TOKEN\|SECRET\|pass(?:OR)?D\|CREDENTIALS?\|API_?KEY\|PRIVATE_KEY\|ENC_KEY\|DEPLOYMENT_KEY)\w["']?\s[=:]\s["']?)\S+/gi,

Uh oh!

Conversation

marcleblanc2 commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Token

Tested

❌ The Vercel build failed for this PR

Amp thread

Merge order for the PR-check stack

Uh oh!

vercel Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 11, 2026

⚠️ Spell check found 3 issue(s) in this PR

Uh oh!

github-actions Bot Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

marcleblanc2 commented Sep 11, 2026 •

edited

Loading

vercel Bot commented Sep 11, 2026 •

edited

Loading