docs: Improve self-hosted Kubernetes troubleshooting

[warp-agent-staging]

What this feature does

Self-hosted Kubernetes workers run each task as a Kubernetes Job. These docs show platform engineers how to protect active task pods, plan capacity, and diagnose startup, scheduling, image, Secret, networking, initialization, and eviction failures. Shipped in oz-agent-worker v2026-09-18-23-15-02 (2026-09-18).

Summary

This PR updates the existing self-hosting documentation with production-backed Kubernetes troubleshooting and lifecycle guidance. It keeps operational procedures on the managed Kubernetes and troubleshooting pages, with focused corrections to the related reference, monitoring, and security pages.

Changes

• Adds Karpenter disruption protection, PDB limitations, task-pod lifecycle behavior, resource headroom, scheduling, taint, and toleration guidance.
• Adds a Kubernetes task-failure decision tree for unschedulable pods, OOM kills, image pulls, missing Secrets and volumes, init failures, task-pod egress, deadlines, and eviction.
• Corrects preflight, RBAC, Job cleanup, connection log, and OpenTelemetry exporter behavior against the current worker implementation.
• Documents current Kubernetes config fields and the classified task-failure metric.
• Defines Kubernetes diagnostic placeholders and removes repeated lifecycle and metrics explanations.

Content design plan

Audience and JTBD: External platform and DevOps engineers operating self-hosted cloud agent workers on Kubernetes who need to keep active runs stable and diagnose failures from standard Kubernetes signals.

Problem: Existing pages did not explain the production failure modes operators see across pod scheduling, resources, images, initialization, credentials, egress, and node disruption. They also contained stale lifecycle and metrics defaults.

Goals:

• Help operators map Kubernetes pod states and events to a specific remediation.
• Explain which pods require disruption protection and why an interrupted task does not resume.
• Keep configuration defaults and cleanup behavior aligned with the current worker release.

Purpose and value: Operators can diagnose failures without incident-specific context or knowledge of Warp's internal systems.

Content type: Troubleshooting, with targeted feature, reference, monitoring, and security corrections.

Skill and template: draft_troubleshooting / .agents/templates/troubleshooting.md.

High-impact scenarios:

• Covers worker startup, preflight, capacity, unschedulable pods, CPU and memory, image pulls, Secrets, init containers, networking, Karpenter, PDB limitations, and mid-run eviction.
• Excludes universal PDB manifests, provider-specific controls other than verified Karpenter behavior, and any claim that a replacement pod resumes task state.

Verification

• python3 .agents/skills/style_lint/style_lint.py --changed — passed; 5 files scanned, 0 issues.
• python3 .agents/skills/check_for_broken_links/check_links.py --internal-only — passed; 4,242 internal links checked, 0 broken.
• npm run build — passed; Astro build, prerendering, Markdown generation, sitemap, and search index completed.
• python3 .agents/skills/doc_quality_policy/check_compression_contract.py ... — passed for troubleshooting, monitoring, reference, and security pages. The managed Kubernetes page remains above the combined-feature budget at 1,941 words, down from 2,236 in the previous revision; it was already above budget at 1,791 words on main. This PR removes three callouts and keeps the requested operational guidance on the existing canonical page instead of creating a duplicate page.
• trunk check / trunk fmt — unavailable because Trunk is not installed in the environment.

Source verification

• oz-agent-worker@530bbe934a632910dfe8818ab7b4c6027a3473ef: internal/worker/kubernetes.go, internal/worker/worker.go, internal/config/config.go, internal/metrics/metrics.go, charts/oz-agent-worker/values.yaml, and charts/oz-agent-worker/templates/deployment.yaml.
• warp-server@2916aa67b511e8ef1f556766aa3d2dfe42a42f74: logic/ai/ambient_agents/workers/selfhosted/websocket.go.
• Karpenter disruption controls.

Unverified claims

None — worker behavior, config fields, defaults, failure classifications, and Kubernetes lifecycle claims were verified against the sources above. Karpenter behavior was verified against its current public documentation.

Documentation risk

Risk: engineering-review-required
Rationale: Changes commands, configuration examples and defaults, RBAC, task lifecycle and disruption, networking, security, and monitoring claims.
Source files consulted: oz-agent-worker@530bbe934a632910dfe8818ab7b4c6027a3473ef (worker, config, metrics, and Helm sources); warp-server@2916aa67b511e8ef1f556766aa3d2dfe42a42f74 (self-hosted worker failure contract); Karpenter disruption documentation
Requested engineering reviewers: none (ownership resolved to a team, not one unambiguous human owner)
Engineering review status: pending
Docs override: none

Co-Authored-By: Oz oz-agent@warp.dev

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 20, 2026 5:33pm UTC

[warp-agent-staging]

This PR was generated with Warp.

Comment @warp-staging-factory on this PR to send it follow-up work.

  

[warp-for-oss]

@hongyi-chen

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[warp-for-oss]

Overview

This PR expands the self-hosted Kubernetes documentation with operational troubleshooting, lifecycle, metrics, reference, and security updates. I found one docs-style issue around unexplained command placeholders; no security-specific or spec-drift concerns surfaced from the attached context.

Concerns

• src/content/docs/platform/self-hosting/troubleshooting.mdx introduces all-caps placeholders in Kubernetes diagnostic commands without defining the values readers should substitute.
• spec_context.md reports that no approved or repository spec context was found, so spec-drift review did not apply.

Verdict

Found: 0 critical, 0 important, 1 suggestions

Approve with nits

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[github-actions]

Review summary

The independent agent completed its review for this commit.

Findings

• src/content/docs/platform/self-hosting/reference.mdx:156 — [CRITICAL] The config-file field active_deadline_seconds is documented as "Defaults to eight hours," but oz-agent-worker@530bbe9 internal/worker/kubernetes.go applies no fallback default for this field when set via the raw YAML config (only the Helm chart values.yaml defaults it to 28800s); an operator using the config file directly gets an unbounded task Job. Requested change: state that the field has no binary-level default and that eight hours is a Helm-chart-only default (see kubernetesBackend.activeDeadlineSeconds on managed-kubernetes.mdx).
• src/content/docs/platform/self-hosting/reference.mdx:150 — [SUGGESTION] preflight_resources does not document its actual default values (requests 10m CPU/16Mi memory, limits 100m CPU/64Mi memory per oz-agent-worker@530bbe9 KubernetesBackendConfig.PreflightResources). Requested change: add the concrete default resource values to this bullet and its Helm-value counterpart in managed-kubernetes.mdx.
• src/content/docs/platform/self-hosting/managed-kubernetes.mdx:236 — [SUGGESTION] Page is 2,236 words against the 1,500-word feature-doc budget (justified in the PR body by avoiding a duplicate page for the new Karpenter/capacity sections). Requested change: no immediate action; consider splitting "Protect active task pods from disruption" and "Plan capacity and scheduling" into a dedicated page if the page grows further.

Verdict

Request changes

[github-actions]

Review summary

The independent agent completed its review for this commit.

Findings

• src/content/docs/platform/self-hosting/reference.mdx:156 — [CRITICAL] The active_deadline_seconds config-file bullet claims "Defaults to eight hours," but that default only exists in the Helm chart's values.yaml (kubernetesBackend.activeDeadlineSeconds: 28800); the raw config file / worker binary applies no such default (internal/worker/kubernetes.go:330 passes the nilable pointer through unchanged, unlike ttl_seconds_after_finished which has a real coded default). Requested change: remove the false default claim or clarify it applies only to the Helm chart deployment path.
• src/content/docs/platform/self-hosting/managed-kubernetes.mdx — [SUGGESTION] Page is 1941 words, still ~440 words over the 1500-word feature-doc budget per check_compression_contract.py (was 1791 on main); the PR's own reported count of 2,236 doesn't match either measurement. Justified in the PR body, but requested follow-up: move the full Helm value reference list to reference.mdx to bring the page back under budget.
• src/content/docs/platform/self-hosting/security-and-networking.mdx:46 — [NIT] "Docker Hub — for pulling task images..." lost its bold lead-in while sibling paragraphs below it (Google Cloud Storage, GitHub, Linux package repositories) kept theirs. Requested change: restore Docker Hub bold for consistency.

Verdict

Request changes

[github-actions]

Review summary

The independent agent completed its review for this commit.

Findings

• src/content/docs/platform/self-hosting/security-and-networking.mdx:46 — [NIT] The "Docker Hub — for pulling task images..." line lost its bold lead-in, inconsistent with the sibling "Google Cloud Storage (...)" and "GitHub (...)" entries in the same list. Requested change: restore the bold to **Docker Hub** — for pulling task images (managed architecture only).
• src/content/docs/platform/self-hosting/managed-kubernetes.mdx:230 — [SUGGESTION] Page word count (1,941) exceeds the 1,500-word feature-doc compression budget; already justified in the PR body (reduced from 2,236, three callouts removed, avoids a duplicate page) so non-blocking. Requested change: none required now, but consider splitting the new "Protect active task pods from disruption" and "Plan capacity and scheduling" sections onto a dedicated page if this content grows further in a follow-up.

Verdict

Approve with nits