Skip to content

Fail loudly when renew-certificate --restart cannot restart the control plane; include scheduler#1670

Open
Mukuwul wants to merge 2 commits into
dapr:masterfrom
Mukuwul:fix/renew-cert-restart-errors
Open

Fail loudly when renew-certificate --restart cannot restart the control plane; include scheduler#1670
Mukuwul wants to merge 2 commits into
dapr:masterfrom
Mukuwul:fix/renew-cert-restart-errors

Conversation

@Mukuwul

@Mukuwul Mukuwul commented Jul 18, 2026

Copy link
Copy Markdown

Description

dapr mtls renew-certificate -k --restart had three defects that combined to swallow every failure mode of the post-rotation control-plane restart:

  1. The error returned by restartControlPlaneService() was discarded, and the if err != nil beneath it tested a stale variable that is provably nil at that point (a non-nil value would already have exited via logErrorAndExit). A failed restart still printed "Certificate rotation is successful!" and exited 0.
  2. The restart list omitted statefulsets/dapr-scheduler-server, which is part of the control plane since runtime 1.14 — so the scheduler kept running with pre-rotation certificates (previously reported from the field in dapr mtls renew-certificate -k --valid-until <days> --restart did not restart statefulsets/dapr-scheduler-server #1574, closed stale).
  3. A GetDaprNamespace() failure was printed and then ignored, producing kubectl rollout restart -n "".

Changes:

  • Capture and handle the restartControlPlaneService() error: print to stderr and exit 1.
  • Return early on namespace lookup failure instead of continuing with an empty namespace.
  • Add statefulsets/dapr-scheduler-server to the restart list — probed with kubectl get first so clusters on runtime < 1.14 (no scheduler) are unaffected.

Issue reference

Fixes #1669
Related: #1574 (field report of the missing scheduler restart, closed by the stale bot), #807 (mTLS rotation reliability epic)

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • Code compiles correctly
  • Created/updated tests (restart path shells out to kubectl; no existing harness for it — happy to add coverage if maintainers point at a preferred pattern)
  • Extended the documentation (n/a — behavior fix)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes dapr mtls renew-certificate -k --restart so that control-plane restart failures are no longer silently ignored, and ensures the scheduler (introduced as a control-plane component in newer runtimes) is included in the restart when present.

Changes:

  • Propagates and handles restartControlPlaneService() errors, printing to stderr and exiting non-zero on failure.
  • Fails early if GetDaprNamespace() cannot be determined (instead of continuing with an empty namespace).
  • Adds a conditional restart for statefulsets/dapr-scheduler-server by probing for its presence first.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmd/renew_certificate.go Outdated
if err != nil {
print.FailureStatusEvent(os.Stdout, err.Error())
if err := restartControlPlaneService(); err != nil {
print.FailureStatusEvent(os.Stderr, err.Error())

@Mukuwul Mukuwul Jul 18, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 339975f - switched to a constant format string. Side note: go vet on Go 1.26 flags ~50 pre-existing call sites across cmd/ with this same pattern, so a separate cleanup PR might be worth doing at some point.

Comment thread cmd/renew_certificate.go
Comment on lines +183 to 187
// The scheduler control plane service only exists for runtime 1.14 onwards,
// so restart it only when it is present in the cluster.
if _, err := utils.RunCmdAndWait("kubectl", "get", "statefulsets/dapr-scheduler-server", "-n", namespace); err == nil {
controlPlaneServices = append(controlPlaneServices, "statefulsets/dapr-scheduler-server")
}

@Mukuwul Mukuwul Jul 18, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 339975f - reworded both errors to be resource-agnostic since the list now includes statefulsets.

Mukuwul added 2 commits July 19, 2026 03:02
…ol plane

dapr mtls renew-certificate --restart discarded the error returned by
restartControlPlaneService and checked a stale err variable that is
always nil at that point, so a failed control-plane restart still
reported success with exit code 0. A namespace lookup failure was also
ignored, producing 'kubectl rollout restart -n ""'.

The restart list additionally omitted statefulsets/dapr-scheduler-server,
which is part of the control plane since runtime 1.14 (reported in dapr#1574).
The scheduler is restarted only when present so older clusters are
unaffected.

Fixes dapr#1669

Signed-off-by: Mukul <nmukul32@gmail.com>
…errors

Signed-off-by: Mukul <nmukul32@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dapr mtls renew-certificate --restart silently ignores restart failures and never restarts the scheduler statefulset

2 participants