OCPBUGS-122255: tests-extension: fix PolarionID:68521 and Polarion:27680

[perdasilva]

Summary

Two independent fixes to the OLM v0 QE test extension, both addressing tests that fail against current catalog/product behavior.

1. PolarionID:68521 — startupProbe failureThreshold assertion (10 → 15)

The test PolarionID:68521-Check failureThreshold of redhat-operators catalog failed with:

error: expected content 10 not found by [pods -n openshift-marketplace -l olm.catalogSource=redhat-operators -o=jsonpath={..spec.containers[0].startupProbe.failureThreshold}]

Root cause: The catalog pod's StartupProbe.FailureThreshold was restored from 10 back to 15, returning the registry-server startup budget from 100s to 150s (the 100s budget was too tight for large CatalogSources — OCPBUGS-78095). The product value in reconciler.go is now 15, but the QE test still asserted 10.

Origin of the change:

• Upstream: OCPBUGS-78095: Restore registry-server startup probe FailureThreshold to 15 operator-framework/operator-lifecycle-manager#3876 ("Restore registry-server startup probe FailureThreshold to 15", upstream-commit aca76a50)
• Synced downstream as commit 16f83128c via OCPBUGS-23954,OCPBUGS-78095: Synchronize From Upstream Repositories #1355 (synchronize-upstream)

Fix: Update the assertion in tests-extension/test/qe/specs/olmv0_nonallns.go to expect 15.

2. ClusterPackageExists — treat NotFound as absent, not an error

The test PolarionID:27680-OLM Bundle support for Prometheus Types failed with:

failed to query packagemanifest for odf-prometheus-operator in openshift-marketplace
Error from server (NotFound): packagemanifests.packages.operators.coreos.com "odf-prometheus-operator" not found

Root cause: odf-prometheus-operator is not present in the redhat-operators catalog on the cluster under test. The test is written to g.Skip in that case, but ClusterPackageExists returned a non-nil error on NotFound (oc get exits 1). Since the test asserts o.Expect(err).NotTo(o.HaveOccurred()) before the skip check, it failed instead of skipping.

Origin of the regression: commit b51eec3ac ("tests-extension: replace etcd usage in 32613/47181 and update 27680 Prometheus flow"), merged via #1241, which rewrote test 27680 to use the real redhat-operators catalog + odf-prometheus-operator and added the o.Expect(err) assertion that is incompatible with how ClusterPackageExists reports NotFound.

Fix: ClusterPackageExists now returns (false, nil) when the packagemanifest is NotFound, and only surfaces genuine query errors. A missing packagemanifest is a legitimate "does not exist" result. The other callers in olmv0_defaultoption.go already discard the error and are unaffected.

Verification

• make build passes
• make update-metadata produces no changes (test titles/labels unchanged)

🤖 Generated with Claude Code

[openshift-ci]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 58222fce-7917-466d-9d71-010ec0dcc722

📥 Commits

Reviewing files that changed from the base of the PR and between e235699 and 8a165a0.

📒 Files selected for processing (1)

• tests-extension/test/qe/util/olmv0util/helper.go

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The package lookup helper now treats missing package manifests as a normal nonexistence result and propagates other errors. The redhat-operators catalog test now expects a startup probe failureThreshold of 15.

Changes

Catalog and package handling

Layer / File(s)	Summary
Package lookup handling and validation tests-extension/test/qe/util/olmv0util/helper.go, tests-extension/test/qe/specs/olmv0_nonallns.go	ClusterPackageExists returns (false, nil) for “NotFound” or “not found” output, returns other query errors, and returns (true, nil) for successful lookups. The catalog test expectation changed from 10 to 15.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 8a165

The package lookup correctly handles absent manifests, and the probe expectation matches the restored product value.

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The pull request does not introduce or modify a Ginkgo test title. The target title remains the static string `PolarionID:68521-[OTP][Skipped:Disconnected]Check failureThreshold of redhat-operators ca…
Test Structure And Quality	✅ Passed	The pull request does not introduce a test-structure or quality failure. The Ginkgo change only updates the expected value from 10 to 15 in an existing It block. That block creates no resources,…
Microshift Test Compatibility	✅ Passed	PASS: The pull request does not add a new Ginkgo test. It changes one assertion in the existing PolarionID:68521 test from 10 to 15 and updates error handling in ClusterPackageExists. The test…
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The pull request does not add a Ginkgo test. It changes one existing assertion from 10 to 15 and updates ClusterPackageExists error handling. The affected test only reads the redhat-operators …
Topology-Aware Scheduling Compatibility	✅ Passed	PASS: The PR changes only two Go test/helper files. The diff updates a probe assertion from 10 to 15 and changes ClusterPackageExists error handling. It does not add or modify deployment manifes…
Ote Binary Stdout Contract	✅ Passed	PASS: The pull request changes only a test assertion and ClusterPackageExists error handling. The assertion is inside a g.It body, which the check explicitly excludes. ClusterPackageExists conta…
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PASS. The pull request changes an existing Ginkgo test assertion from 10 to 15; it does not add a new test. The affected test queries cluster resources through oc and adds no IPv4 address, IPv4-…
No-Weak-Crypto	✅ Passed	PASS. The pull request changes only a test value from 10 to 15 and ClusterPackageExists error handling. No changed line introduces MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB mode, custom cryptogra…
Container-Privileges	✅ Passed	PASS. The PR changes only a test assertion and ClusterPackageExists error handling. The added lines introduce no privileged: true, hostPID, hostNetwork, hostIPC, SYS_ADMIN, or `allowPrivil…
No-Sensitive-Data-In-Logs	✅ Passed	PASS. The pull request changes one test literal from 10 to 15 and changes ClusterPackageExists error handling. It adds no logging calls and does not log command output, tokens, credentials, PII,…
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title identifies the OCP bug and the two Polarion issues addressed by the test update and ClusterPackageExists error-handling change.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: perdasilva

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• DOWNSTREAM_OWNERS [perdasilva]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@perdasilva: This pull request references Jira Issue OCPBUGS-122255, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.1.0) matches configured target version for branch (5.1.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

Two independent fixes to the OLM v0 QE test extension, both addressing tests that fail against current catalog/product behavior.

1. PolarionID:68521 — startupProbe failureThreshold assertion (10 → 15)

The test PolarionID:68521-Check failureThreshold of redhat-operators catalog failed with:

error: expected content 10 not found by [pods -n openshift-marketplace -l olm.catalogSource=redhat-operators -o=jsonpath={..spec.containers[0].startupProbe.failureThreshold}]

Root cause: The catalog pod's StartupProbe.FailureThreshold was restored from 10 back to 15, returning the registry-server startup budget from 100s to 150s (the 100s budget was too tight for large CatalogSources — OCPBUGS-78095). The product value in reconciler.go is now 15, but the QE test still asserted 10.

Origin of the change:

• Upstream: OCPBUGS-78095: Restore registry-server startup probe FailureThreshold to 15 operator-framework/operator-lifecycle-manager#3876 ("Restore registry-server startup probe FailureThreshold to 15", upstream-commit aca76a50)
• Synced downstream as commit 16f83128c via OCPBUGS-23954,OCPBUGS-78095: Synchronize From Upstream Repositories #1355 (synchronize-upstream)

Fix: Update the assertion in tests-extension/test/qe/specs/olmv0_nonallns.go to expect 15.

2. ClusterPackageExists — treat NotFound as absent, not an error

The test PolarionID:27680-OLM Bundle support for Prometheus Types failed with:

failed to query packagemanifest for odf-prometheus-operator in openshift-marketplace
Error from server (NotFound): packagemanifests.packages.operators.coreos.com "odf-prometheus-operator" not found

Root cause: odf-prometheus-operator is not present in the redhat-operators catalog on the cluster under test. The test is written to g.Skip in that case, but ClusterPackageExists returned a non-nil error on NotFound (oc get exits 1). Since the test asserts o.Expect(err).NotTo(o.HaveOccurred()) before the skip check, it failed instead of skipping.

Origin of the regression: commit b51eec3ac ("tests-extension: replace etcd usage in 32613/47181 and update 27680 Prometheus flow"), merged via #1241, which rewrote test 27680 to use the real redhat-operators catalog + odf-prometheus-operator and added the o.Expect(err) assertion that is incompatible with how ClusterPackageExists reports NotFound.

Fix: ClusterPackageExists now returns (false, nil) when the packagemanifest is NotFound, and only surfaces genuine query errors. A missing packagemanifest is a legitimate "does not exist" result. The other callers in olmv0_defaultoption.go already discard the error and are unaffected.

Verification

• make build passes
• make update-metadata produces no changes (test titles/labels unchanged)

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@perdasilva: This pull request references Jira Issue OCPBUGS-122255, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.1.0) matches configured target version for branch (5.1.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Summary

Two independent fixes to the OLM v0 QE test extension, both addressing tests that fail against current catalog/product behavior.

1. PolarionID:68521 — startupProbe failureThreshold assertion (10 → 15)

The test PolarionID:68521-Check failureThreshold of redhat-operators catalog failed with:

error: expected content 10 not found by [pods -n openshift-marketplace -l olm.catalogSource=redhat-operators -o=jsonpath={..spec.containers[0].startupProbe.failureThreshold}]

Root cause: The catalog pod's StartupProbe.FailureThreshold was restored from 10 back to 15, returning the registry-server startup budget from 100s to 150s (the 100s budget was too tight for large CatalogSources — OCPBUGS-78095). The product value in reconciler.go is now 15, but the QE test still asserted 10.

Origin of the change:

• Upstream: OCPBUGS-78095: Restore registry-server startup probe FailureThreshold to 15 operator-framework/operator-lifecycle-manager#3876 ("Restore registry-server startup probe FailureThreshold to 15", upstream-commit aca76a50)
• Synced downstream as commit 16f83128c via OCPBUGS-23954,OCPBUGS-78095: Synchronize From Upstream Repositories #1355 (synchronize-upstream)

Fix: Update the assertion in tests-extension/test/qe/specs/olmv0_nonallns.go to expect 15.

2. ClusterPackageExists — treat NotFound as absent, not an error

The test PolarionID:27680-OLM Bundle support for Prometheus Types failed with:

failed to query packagemanifest for odf-prometheus-operator in openshift-marketplace
Error from server (NotFound): packagemanifests.packages.operators.coreos.com "odf-prometheus-operator" not found

Root cause: odf-prometheus-operator is not present in the redhat-operators catalog on the cluster under test. The test is written to g.Skip in that case, but ClusterPackageExists returned a non-nil error on NotFound (oc get exits 1). Since the test asserts o.Expect(err).NotTo(o.HaveOccurred()) before the skip check, it failed instead of skipping.

Origin of the regression: commit b51eec3ac ("tests-extension: replace etcd usage in 32613/47181 and update 27680 Prometheus flow"), merged via #1241, which rewrote test 27680 to use the real redhat-operators catalog + odf-prometheus-operator and added the o.Expect(err) assertion that is incompatible with how ClusterPackageExists reports NotFound.

Fix: ClusterPackageExists now returns (false, nil) when the packagemanifest is NotFound, and only surfaces genuine query errors. A missing packagemanifest is a legitimate "does not exist" result. The other callers in olmv0_defaultoption.go already discard the error and are unaffected.

Verification

• make build passes
• make update-metadata produces no changes (test titles/labels unchanged)

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[perdasilva]

/payload-aggregate periodic-ci-openshift-operator-framework-olm-release-5.1-periodics-e2e-aws-ovn-extended-f1 10

[openshift-ci]

@perdasilva: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-operator-framework-olm-release-5.1-periodics-e2e-aws-ovn-extended-f1

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/47ce5ac0-b0f8-11f1-94b6-9f49072cc97a-0

[perdasilva]

/hold for aggregate jobs

[perdasilva]

/payload-aggregate periodic-ci-openshift-operator-framework-olm-release-5.1-periodics-e2e-aws-ovn-extended-f1 10

[openshift-ci]

@perdasilva: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-operator-framework-olm-release-5.1-periodics-e2e-aws-ovn-extended-f1

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/2a81c1b0-b0fc-11f1-88b3-18a5b8ed27f7-0

[perdasilva]

/payload-aggregate periodic-ci-openshift-operator-framework-olm-release-5.1-periodics-e2e-azure-ovn-techpreview-extended-f1 10

[openshift-ci]

@perdasilva: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-operator-framework-olm-release-5.1-periodics-e2e-azure-ovn-techpreview-extended-f1

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/0a878a40-b0ff-11f1-88dc-aba9fade0369-0

[perdasilva]

/test ci/prow/e2e-upgrade

[perdasilva]

/test e2e-upgrade

[perdasilva]

/test e2e-gcp-own

[perdasilva]

/test e2e-aws-olmv0-ext

[openshift-ci]

@perdasilva: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.