Skip to content

docs: state which images CertificateAudit can assess #175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
stevebeattie wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: state which images CertificateAudit can assess #175

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 65 additions & 6 deletions docs/certificate-audit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,60 @@ apko's own comment on `writeCABundleChecksums` states the purpose: "so
downstream tooling (e.g. OpenSCAP) can verify they were not modified
post-build." This rule is that downstream tooling.

### Which images this rule can assess

Because the expected digest comes from the sidecar, an image that has no sidecar
has nothing to compare against — and the rule **fails** rather than skipping.
That is deliberate: `tst:4`/`tst:6`/`tst:11` exist so a missing sidecar cannot
pass vacuously, and `certificate_audit/fail_missing_stamp` pins it.

The consequence is a floor on which images the rule can meaningfully assess:

| image | outcome |
| --- | --- |
| built by apko **v1.2.30 or later** | assessable |

@0xDom-S 0xDom-S Sep 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: The apko v1.2.30 sidecar-floor claim here (and the related version claims at :90, :187, and :407 for the ECS bundle bump v1.2.35→v1.2.43) are the only apko version references in the tree and nothing pins them in-repo. They rest entirely on the author's stated manual verification, which could not be checked offline in this review. Not a defect — flagged so reviewers know which claims carry no regression guard.

| built by apko **v1.2.29 or earlier** | fails — no sidecars exist |
| not built by apko at all | fails — no sidecars exist |

v1.2.30 is the first release carrying `writeCABundleChecksums`; v1.2.29 does not
have it. Every current Chainguard image is well past that, so this is not a
concern for scanning what the registry serves today. It matters when scanning
something older: an archived release, a customer's pinned image from before the
change, or an image built by other tooling.

**A failure caused by the floor is not distinguishable from a real one by the
rule verdict alone** — both are `fail`. It *is* distinguishable from the scan
artifact, in the per-test OVAL results, so no access to the image is needed and
an archived results file can be read after the fact. Scan with `--oval-results`

@0xDom-S 0xDom-S Sep 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: Two issues with the --oval-results guidance here.

(a) Invocation: this is the only occurrence of --oval-results/ARF anywhere in the repo; in OpenSCAP, --oval-results is functional only alongside --results. Worth spelling out the full invocation so a reader doesn't get an empty result — the repo's own harness (tests/oscap-offline/internal/scan/command.go:148-150, tests/e2e/run.sh:150-152) passes only --results and won't produce this per-test detail.

(b) Sensitivity: full OVAL results/ARF serialize textfilecontent54_item matched text for every rule in the profile, not just CertificateAudit — including /etc/shadow-matching rules co-selected in the same profile (e.g. UserPasswordConfiguredTest.xml, NoUsersCheck.xml use a regex whose [^:\n]* group captures the crypt field). This is bounded (check_existence="none_exist", instance=1, first match only, image already failing that rule) and not a vulnerability, but since :98-99 explicitly anticipates the file being read by someone with no image access, a clause noting full OVAL output is credential-bearing (or scoping the guidance to oscap oval eval --id oval:org.CABundleHash:def:1) is cheap and correct.

(or keep the ARF) and compare two tests:

| | no usable sidecar | trust store actually drifted |
| --- | --- | --- |
| `tst:4` — sidecar exists and parses | `false` | `true` |
| `tst:2` — bundle matches the sidecar digest | `error` | `false` |

The `error` on `tst:2` is itself the tell: the variable behind the comparison
collected no values, because there was no sidecar to read one from. A `false`
there means a sidecar was read and disagreed.

So `tst:4 false` says the rule *could not assess* this image — it predates the

@0xDom-S 0xDom-S Sep 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: The disjunction "tst:4 false says the rule could not assess this image — it predates the mechanism, or the sidecar is malformed" omits post-build sidecar deletion.

Because tst:2's comparator comes from var:1←obj:4 and not from the bundle, the tst:4 false / tst:2 error signature is content-independent on the bundle side. A sidecar deleted after build — maliciously or via an image-slimming step — produces the identical signature to a pre-v1.2.30 image. The doc's own measurement at :117-118 was taken with the bundle intact, so it doesn't separate the two cases.

Note that in-image corroboration is not available either: sidecars are unowned (apk info -W → "Could not find owner package"), and apk audit always reports benign A for them, so the X ("owned but missing") code can never fire for a deleted sidecar (apk audit --system will NOT surface this).

Suggested fix: add "or the sidecar was removed after the build" to :111-112 and state that tst:4 false is not an exoneration. As a disambiguator, the bundle's actual SHA-256 is collected by obj:2 (filehash58_object, :100-103) and can be compared out-of-band against the signed build.

mechanism, or the sidecar is malformed — which is a different statement from
"this image's trust stores were modified". `tst:4 true` with `tst:2 false` is
the real finding. The same reading applies to `tst:6`/`tst:7` for the Java

@0xDom-S 0xDom-S Sep 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: The generalization "The same reading applies to tst:6/tst:7 for the Java truststore and tst:11/tst:12 for a /kaniko copy" is unqualified and will mis-triage ordinary images.

The reading only transfers when the truststore or /kaniko copy actually exists. On a non-Java image (the common case): tst:5 is none_exist over obj:5 (CertificateAuditTest.xml:64-66) → true, satisfying the Java OR at :39-45. But tst:6 is only_one_exists over obj:6 (:67-69, :117-122) → false, and tst:7's state ste:3 resolves via var:2←obj:6 (:152-155, :168-170) → error. That is byte-for-byte the "no usable sidecar" column of the new triage table (:102-105) appearing on a fully assessable, passing scan. The same holds for non-kaniko images with tst:11/tst:12.

An analyst triaging a fail per :111-115 could wrongly conclude "the Java truststore / /kaniko copy could not be assessed" on an image that has neither.

Suggested fix: qualify the sentence — "…where that truststore or copy is present. On an image with neither, tst:6/tst:11 are false and tst:7/tst:12 error as a matter of course; tst:5 and tst:3 carry those branches and the absence is not a finding."

truststore and `tst:11`/`tst:12` for a `/kaniko` copy.

Measured, not inferred: scanning an image with its sidecar removed and its
bundle intact gives `tst:4 false`, `tst:2 error`; scanning one with the sidecar
intact and the bundle appended to gives `tst:4 true`, `tst:2 false`. Both report
the rule as `fail`.

In-image, `ls -l /etc/ssl/certs/.ca-certificates.crt.sha256` answers the same
question more directly, where you have a shell and the image to hand.

Note this is a change in which images are assessable, not only in how. Under the
previous design the expected digest was pinned in the datastream, so an old image
could pass if its bundle happened to match that pin — no sidecar required.

## Why not a pinned digest

The rule previously pinned the CA bundle's SHA-256 in the datastream. That had
Expand Down Expand Up @@ -128,7 +182,16 @@ Then an `OR` for the Java truststore:
to the absent-truststore branch.
- **`tst:13` before falling back to `tst:9`.** A sidecar beside the `/kaniko`
copy takes precedence, so a divergent copy cannot sidestep its own sidecar by
appealing to the system one.
appealing to the system one. Which branch a real image takes has

@0xDom-S 0xDom-S Sep 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: This new text (asserting the /kaniko copy now ships its own sidecar and real images have moved onto tst:11/tst:12) contradicts the unchanged deferral rationale later in the file at docs/certificate-audit.md:399-402, which still reads: "The copy is currently byte-identical to its system bundle, so the drift being guarded against is remote… Revisit if the /kaniko copy ever starts diverging, or gains a sidecar of its own." This PR's own evidence fires that revisit trigger verbatim, yet :193-194 concludes "nothing needs changing."

Two follow-ups worth doing in this PR (both landing in unchanged lines outside the diff, so not separately anchorable):

  • Update :399-402 so the file doesn't argue both sides.
  • The Known gap at :413-416 should note that the branch production images now take (obj:10) is the unguarded one — tests/stamps/run.sh:61 marks obj:10 required=no, and .github/workflows/update-ca-cert.yaml:35 stamps only cgr.dev/chainguard/jre:latest daily. Deferral may still be the right call; the rationale just needs restating on the new facts.

changed. `kaniko/ssl/certs/ca-certificates.crt` was **not** in apko's
`caBundlePaths` at v1.2.35 but **is** at v1.2.43, and the published kaniko

@0xDom-S 0xDom-S Sep 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: Minor wording ambiguity: this line says "the published kaniko image", while the unchanged line at docs/certificate-audit.md:370 says the /kaniko copy exists "only" on cgr.dev/chainguard-private/kaniko, and :413-414 says the daily workflow "inspects only public images." Both readings are individually defensible, but one word here (e.g. "published private image", or naming the ref explicitly) would remove the ambiguity for future readers.

image now ships `/kaniko/ssl/certs/.ca-certificates.crt.sha256` where in
August 2026 it did not. So real kaniko images have moved off the fallback and
onto `tst:11`/`tst:12`. Confirmed by running the guard against the image: the
copy matches its own sidecar, and that sidecar matches `obj:10`'s pattern —
the first time that pattern has been checked against anything other than a
synthetic fixture. Both branches remain fixture-covered, so nothing needs
changing; the fallback is now the path an *older* kaniko image would take.
- **`tst:5` uses `none_exist`** rather than testing for Java some other way,
because a non-Java image must not fail for lacking a truststore.

Expand Down Expand Up @@ -341,12 +404,8 @@ against is remote; the trade was judged not worth it for now. Revisit if the
## Known gaps

- apko also stamps `var/lib/ecs/deps/execute-command/certs/tls-ca-bundle.pem`
(still in `caBundlePaths` as of apko v1.2.35). This rule neither checks that
(still in `caBundlePaths` as of apko v1.2.43). This rule neither checks that
bundle nor its sidecar.
- `CertificateAuditTest.xml` is the only OVAL component referenced by the
datastream with no standalone file under
`gpos/xml/scap/ssg/content/ssg-chainguard-xccdf/OvalDefinitions/`, so
`make validate_checks` does not validate this definition.
- `<ind:instance>1</ind:instance>` with `only_one_exists` means a sidecar
containing two matching lines silently uses the first.
- Deleting `/etc/ssl/certs/java/cacerts` outright satisfies the Java `OR` via
Expand Down