Skip to content
Merged
45 changes: 38 additions & 7 deletions .github/actions/ci-test-notify/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,20 @@ Replaces the nightly-specific `ci-notify-nightly-tests` action with a generic in

<!-- AUTO-DOC-INPUT:START - Do not remove or modify this section -->

| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION |
|-------------|--------|----------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| details | string | false | | Markdown text appended after the build <br>URL (test results, versions, artifact links, etc.) |
| status | string | true | | Run status, typically `needs.<job>.result` or `job.status`. <br>`success` and `failure` notify; `cancelled` and <br>`skipped` are treated as no-ops and <br>send nothing. |
| test-name | string | true | | Test suite name for the header <br>(e.g. "E2E Ginkgo Nightly Tests"). Keep under ~130 chars — <br>Slack header blocks have a 150-char <br>limit and the status suffix takes <br>~15 chars. |
| webhook-url | string | true | | Slack incoming webhook URL |
| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION |
|-------------------|--------|----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| details | string | false | | Markdown text appended after the build <br>URL (test results, versions, artifact links, etc.) |
| run-link-position | string | false | `"top"` | Where the immutable workflow-run link goes, <br>and how it reads. `top` (default) <br>puts a bare `Build URL: <url>` line above <br>`details`, unchanged from before this input <br>existed. `bottom` puts a linked `Workflow: View workflow run` <br>line below `details`, so the content <br>leads and the link trails. Invalid <br>values fall back to `top`. |
| status | string | true | | Run status, typically `needs.<job>.result` or `job.status`. <br>`success`, `failure`, and `warning` notify; `cancelled` <br>and `skipped` are treated as no-ops <br>and send nothing. |
| test-name | string | true | | Test suite name for the header <br>(e.g. "E2E Ginkgo Nightly Tests"). Keep under ~130 chars — <br>Slack header blocks have a 150-char <br>limit and the status suffix takes <br>~15 chars. |
| webhook-url | string | true | | Slack incoming webhook URL |

<!-- AUTO-DOC-INPUT:END -->

## Message format

With `run-link-position: top` (the default), unchanged from before that input existed:

```
[emoji] [test-name] [status]
─────────────────────────────
Expand All @@ -29,6 +32,28 @@ Build URL: <link to workflow run>
<repo> · Run #<number>
```

With `run-link-position: bottom`, for messages whose `details` are the point and
should be read first:

```
[emoji] [test-name] [status]
─────────────────────────────
<details if provided>

Workflow: View workflow run
─────────────────────────────
<repo> · Run #<number>
```

The link is not merely moved: `top` prints the bare URL after `Build URL:`, while
`bottom` renders a linked label. `top` is left exactly as it was so that switching
position is opt-in for the roughly thirty existing call sites.

The section is capped at Slack's 3000-character limit and the header at 150. Both
are measured in characters rather than bytes, so multi-byte text is not truncated
early or cut mid-character; with `bottom`, the run link is always preserved and the
`details` are what give way.

## Usage

### Nightly E2E tests
Expand Down Expand Up @@ -79,7 +104,13 @@ The action only notifies on actionable outcomes. A `status` of `cancelled` or
`skipped` is treated as a no-op: the action logs a notice and sends nothing.
This means callers can pass `needs.<job>.result` or `job.status` straight
through without a guard. A cancelled run (aborted by a human or superseded) or a
skipped job never produces a Slack alert; only `success` and `failure` do.
skipped job never produces a Slack alert.

Everything else notifies. `success` and `failure` are the usual pair; `warning`
is for an advisory result that is worth reporting but is not a failure, such as a
CVE scan running on the default non-blocking posture. An unrecognised status also
notifies, under a `❓ Unknown (<status>)` header, on the grounds that a status
nobody anticipated is more useful surfaced than swallowed.

An empty `webhook-url` (fork PRs, where secrets are unavailable) also suppresses
the notification.
Expand Down
7 changes: 6 additions & 1 deletion .github/actions/ci-test-notify/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ inputs:
description: 'Test suite name for the header (e.g. "E2E Ginkgo Nightly Tests"). Keep under ~130 chars — Slack header blocks have a 150-char limit and the status suffix takes ~15 chars.'
required: true
status:
description: 'Run status, typically `needs.<job>.result` or `job.status`. `success` and `failure` notify; `cancelled` and `skipped` are treated as no-ops and send nothing.'
description: 'Run status, typically `needs.<job>.result` or `job.status`. `success`, `failure`, and `warning` notify; `cancelled` and `skipped` are treated as no-ops and send nothing.'
required: true
details:
description: 'Markdown text appended after the build URL (test results, versions, artifact links, etc.)'
Expand All @@ -17,6 +17,10 @@ inputs:
webhook-url:
description: 'Slack incoming webhook URL'
required: true
run-link-position:
description: 'Where the immutable workflow-run link goes, and how it reads. `top` (default) puts a bare `Build URL: <url>` line above `details`, unchanged from before this input existed. `bottom` puts a linked `Workflow: View workflow run` line below `details`, so the content leads and the link trails. Invalid values fall back to `top`.'
required: false
default: 'top'

runs:
using: "composite"
Expand All @@ -40,6 +44,7 @@ runs:
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
REPO: ${{ github.repository }}
RUN_NUMBER: ${{ github.run_number }}
RUN_LINK_POSITION: ${{ inputs.run-link-position }}
run: ${{ github.action_path }}/build-payload.sh

- name: Send Slack notification
Expand Down
69 changes: 60 additions & 9 deletions .github/actions/ci-test-notify/build-payload.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,25 @@
set -euo pipefail

# Required env vars: TEST_NAME, STATUS, DETAILS, PAYLOAD_FILE, RUN_URL, REPO, RUN_NUMBER
# Optional env vars: RUN_LINK_POSITION (top or bottom; defaults to top)

command -v jq >/dev/null || { echo "::error::jq is required but not found"; exit 1; }

# Slack's block limits are in characters. bash's ${#var} and ${var:0:n} follow
# the locale: characters under a UTF-8 locale, bytes under POSIX. Runners are
# not guaranteed to set one, so measuring in bash truncates roughly three times
# too early on non-ASCII text and can cut a UTF-8 sequence mid-character. jq
# always counts codepoints, so measure and cut there instead.
str_len() { printf '%s' "$1" | jq -Rs 'length'; }
clip_to() {
printf '%s' "$2" | jq -Rrs --argjson n "$1" \
'if length > $n then .[0:($n - 3)] + "..." else . end'
}

case "$STATUS" in
success) EMOJI="✅"; STATUS_TEXT="Success" ;;
failure) EMOJI="❌"; STATUS_TEXT="Failed" ;;
warning) EMOJI="⚠️"; STATUS_TEXT="Warning" ;;
cancelled) EMOJI="⚠️"; STATUS_TEXT="Cancelled" ;;
skipped) EMOJI="⏭️"; STATUS_TEXT="Skipped" ;;
*) EMOJI="❓"; STATUS_TEXT="Unknown ($STATUS)" ;;
Expand All @@ -16,20 +29,58 @@ esac
HEADER="${EMOJI} ${TEST_NAME} ${STATUS_TEXT}"

# Slack header blocks reject >150 chars
if [[ ${#HEADER} -gt 150 ]]; then
echo "::warning::Header exceeds 150-char Slack limit (${#HEADER} chars), truncating"
HEADER="${HEADER:0:147}..."
HEADER_LEN=$(str_len "$HEADER")
if [[ $HEADER_LEN -gt 150 ]]; then
echo "::warning::Header exceeds 150-char Slack limit (${HEADER_LEN} chars), truncating"
HEADER=$(clip_to 150 "$HEADER")
fi

# Normalise first, so the two positions are each written once and every later
# reader (the truncation branch below included) sees a value it can trust.
RUN_LINK_POSITION="${RUN_LINK_POSITION:-top}"
if [[ "$RUN_LINK_POSITION" != "top" && "$RUN_LINK_POSITION" != "bottom" ]]; then
echo "::warning::invalid RUN_LINK_POSITION '$RUN_LINK_POSITION', defaulting to top"
RUN_LINK_POSITION="top"
fi

SECTION="Build URL: ${RUN_URL}"
if [[ "$DETAILS" =~ [^[:space:]] ]]; then
SECTION="$(printf '%s\n\n%s' "$SECTION" "$DETAILS")"
# The two positions render the link differently, not just in a different place:
# `top` keeps the bare `Build URL:` line every existing caller already gets, and
# `bottom` uses a linked label that reads better as a footer. Changing `top`
# would alter the message for ~30 call sites, so the difference is documented in
# the input rather than smoothed over here.
RUN_LINK="Workflow: <${RUN_URL}|View workflow run>"
if [[ "$RUN_LINK_POSITION" == "bottom" ]]; then
SECTION="$RUN_LINK"
if [[ "$DETAILS" =~ [^[:space:]] ]]; then
SECTION="$(printf '%s\n\n%s' "$DETAILS" "$SECTION")"
fi
else
SECTION="Build URL: ${RUN_URL}"
if [[ "$DETAILS" =~ [^[:space:]] ]]; then
SECTION="$(printf '%s\n\n%s' "$SECTION" "$DETAILS")"
fi
fi

# Slack section blocks reject >3000 chars
if [[ ${#SECTION} -gt 3000 ]]; then
echo "::warning::Section exceeds 3000-char Slack limit (${#SECTION} chars), truncating"
SECTION="${SECTION:0:2997}..."
SECTION_LEN=$(str_len "$SECTION")
if [[ $SECTION_LEN -gt 3000 ]]; then
echo "::warning::Section exceeds 3000-char Slack limit (${SECTION_LEN} chars), truncating"
if [[ "$RUN_LINK_POSITION" == "bottom" ]]; then
# Reserve the run link and the blank line above it, so truncation never
# costs the one immutable piece of the message.
DETAILS_LIMIT=$((3000 - $(str_len "$RUN_LINK") - 2))
if [[ $DETAILS_LIMIT -lt 4 ]]; then
# A run URL long enough to leave no room for details is not reachable from
# github.server_url/run_id, but an unfloored budget here would go negative
# and a negative slice reads as "all but the last n", overshooting 3000
# and getting the whole message rejected. Keep the link, drop the details.
SECTION=$(clip_to 3000 "$RUN_LINK")
else
SECTION="$(printf '%s\n\n%s' "$(clip_to "$DETAILS_LIMIT" "$SECTION")" "$RUN_LINK")"
fi
else
SECTION=$(clip_to 3000 "$SECTION")
fi
fi

jq -n \
Expand Down
10 changes: 5 additions & 5 deletions .github/actions/ci-test-notify/should-notify.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ set -euo pipefail
# `notify=true|false` to $GITHUB_OUTPUT for the composite action to gate on.
#
# Callers pass the run conclusion straight from `needs.<job>.result` or
# `job.status`, which can be success, failure, cancelled, or skipped. Only
# success and failure are actionable: a cancelled run was aborted by a human
# (or superseded), and a skipped job never executed. Neither warrants a Slack
# alert, so both are silenced here rather than in every caller.
# `job.status`, which can be success, failure, warning, cancelled, or skipped.
# Cancelled and skipped runs are silenced: a cancelled run was aborted by a
# human (or superseded), and a skipped job never executed. A warning is an
# advisory result and should notify without being labelled as a failure.
#
# An empty webhook (fork PRs, where secrets are unavailable) also suppresses
# the notification, same as before.
Expand All @@ -22,7 +22,7 @@ if [[ -z "${WEBHOOK_URL:-}" ]]; then
echo "::warning::webhook-url is empty (expected on fork PRs where secrets are unavailable), skipping notification"
notify=false
elif [[ "${STATUS:?STATUS is required}" == "cancelled" || "$STATUS" == "skipped" ]]; then
echo "::notice::status is '$STATUS' — only success and failure notify, skipping Slack notification"
echo "::notice::status is '$STATUS' — cancelled and skipped runs do not notify, skipping Slack notification"
notify=false
fi

Expand Down
89 changes: 89 additions & 0 deletions .github/actions/ci-test-notify/test/build-payload.bats
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ setup() {
export RUN_URL="https://github.com/org/repo/actions/runs/12345"
export REPO="org/repo"
export RUN_NUMBER="42"
export RUN_LINK_POSITION="top"
}

teardown() {
Expand Down Expand Up @@ -41,6 +42,12 @@ payload_field() {
[[ "$(payload_field '.text')" == *"Failed"* ]]
}

@test "warning status produces an advisory header" {
STATUS="warning" run bash "$SCRIPT"
[ "$status" -eq 0 ]
[ "$(payload_field '.blocks[0].text.text')" = "⚠️ My Test Suite Warning" ]
}

@test "cancelled status produces correct emoji and text" {
STATUS="cancelled" run bash "$SCRIPT"
[ "$status" -eq 0 ]
Expand Down Expand Up @@ -110,6 +117,22 @@ payload_field() {
[[ "$section" == *"Line three"* ]]
}

@test "bottom run link position appends the workflow link after details" {
RUN_LINK_POSITION="bottom" DETAILS="High findings: 6" run bash "$SCRIPT"
[ "$status" -eq 0 ]

local section
section=$(payload_field '.blocks[1].text.text')
[[ "$section" == "High findings: 6"$'\n\n'"Workflow: <https://github.com/org/repo/actions/runs/12345|View workflow run>" ]]
}

@test "invalid run link position safely falls back to the top" {
RUN_LINK_POSITION="hidden" run bash "$SCRIPT"
[ "$status" -eq 0 ]
[[ "$output" == *"invalid RUN_LINK_POSITION"* ]]
[ "$(payload_field '.blocks[1].text.text')" = "Build URL: https://github.com/org/repo/actions/runs/12345" ]
}

# --- Block Kit structure ---

@test "payload has correct block structure" {
Expand Down Expand Up @@ -178,6 +201,72 @@ payload_field() {
[[ "$section" == *"..."* ]]
}

@test "bottom run link is retained when details exceed the section limit" {
RUN_LINK_POSITION="bottom" DETAILS="$(printf 'X%.0s' {1..3000})" run bash "$SCRIPT"
[ "$status" -eq 0 ]

local section
section=$(payload_field '.blocks[1].text.text')
[ "${#section}" -le 3000 ]
[[ "$section" == *"Workflow: <https://github.com/org/repo/actions/runs/12345|View workflow run>" ]]
}

# The limits Slack enforces are in characters, but bash measures in bytes under
# a POSIX locale, so an ASCII-only fixture cannot tell the two apart. These run
# the truncation path with multi-byte text, where a byte-based cut both fires
# far too early and can split a character in half.
#
# LC_ALL=C is pinned deliberately. Under a UTF-8 locale bash already counts
# characters, so these would pass whatever the script did and quietly stop
# testing anything — the same "green here, red there" trap that unsetting
# GITHUB_EVENT_NAME avoids in the cve-scan helpers. Pinning the byte locale
# reproduces the hazard wherever the suite runs.

@test "a multi-byte section is measured in characters, not bytes" {
# 2000 three-byte bullets: 2000 characters, 6000 bytes. Under the limit by
# Slack's count, so nothing should be truncated.
LC_ALL=C DETAILS="$(printf '•%.0s' {1..2000})" run bash "$SCRIPT"
[ "$status" -eq 0 ]

local chars
chars=$(jq -r '.blocks[1].text.text | length' "$PAYLOAD_FILE")
[ "$chars" -le 3000 ]
[[ "$(jq -r '.blocks[1].text.text' "$PAYLOAD_FILE")" != *"..."* ]]
}

@test "truncating a multi-byte section never splits a character" {
LC_ALL=C DETAILS="$(printf '•%.0s' {1..4000})" run bash "$SCRIPT"
[ "$status" -eq 0 ]

# U+FFFD is what a half-written UTF-8 sequence decodes to, so its absence is
# the assertion: the cut landed on a character boundary.
local section
section=$(jq -r '.blocks[1].text.text' "$PAYLOAD_FILE")
[ "$(jq -r '.blocks[1].text.text | length' "$PAYLOAD_FILE")" -le 3000 ]
[[ "$section" != *'�'* ]]
}

@test "a bottom-positioned run link survives truncation intact" {
LC_ALL=C RUN_LINK_POSITION="bottom" DETAILS="$(printf '•%.0s' {1..4000})" run bash "$SCRIPT"
[ "$status" -eq 0 ]

local section
section=$(jq -r '.blocks[1].text.text' "$PAYLOAD_FILE")
[ "$(jq -r '.blocks[1].text.text | length' "$PAYLOAD_FILE")" -le 3000 ]
[[ "$section" == *"Workflow: <${RUN_URL}|View workflow run>" ]]
}

# A run URL this long is not reachable from github.server_url and github.run_id,
# but an unfloored budget would go negative here, and a negative slice length
# reads as "all but the last n" — overshooting 3000 and losing the whole message
# to a Slack rejection.
@test "an absurdly long run URL still yields a section within the limit" {
RUN_URL="https://github.com/org/repo/actions/runs/$(printf '9%.0s' {1..3200})"
RUN_LINK_POSITION="bottom" DETAILS="some findings" run bash "$SCRIPT"
[ "$status" -eq 0 ]
[ "$(jq -r '.blocks[1].text.text | length' "$PAYLOAD_FILE")" -le 3000 ]
}

@test "section is not truncated when under 3000 chars" {
DETAILS="Short details" run bash "$SCRIPT"
[ "$status" -eq 0 ]
Expand Down
6 changes: 6 additions & 0 deletions .github/actions/ci-test-notify/test/should-notify.bats
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ notify_value() {
[ "$(notify_value)" = "true" ]
}

@test "warning notifies" {
STATUS="warning" run bash "$SCRIPT"
[ "$status" -eq 0 ]
[ "$(notify_value)" = "true" ]
}

# --- Statuses that must stay silent (the bug this fixes) ---

@test "cancelled does not notify" {
Expand Down
Loading
Loading