Skip to content

fix(supervisor): tolerate non-empty bounding set when CAP_SETPCAP is unavailable#2075

Open
waynesun09 wants to merge 2 commits into
NVIDIA:mainfrom
waynesun09:fix-2069-cap-bounding-set-rootless-podman
Open

fix(supervisor): tolerate non-empty bounding set when CAP_SETPCAP is unavailable#2075
waynesun09 wants to merge 2 commits into
NVIDIA:mainfrom
waynesun09:fix-2069-cap-bounding-set-rootless-podman

Conversation

@waynesun09

@waynesun09 waynesun09 commented Jun 30, 2026

Copy link
Copy Markdown

Summary

When running under rootless Podman (or any container runtime that drops CAP_SETPCAP), cap_drop_bound() returns EPERM for every capability still in the bounding set. Since v0.0.73 this is fatal — the supervisor crashes on sandbox startup, breaking all rootless Podman deployments.

This PR adds a third match arm to validate_capability_bounding_set_clear() that tolerates EPERM when the bounding set is non-empty, logging a warning instead of returning an error. The sandbox still relies on seccomp and Landlock for confinement in this case.

Related Issue

Fixes #2069

Changes

  • crates/openshell-supervisor-process/src/process.rs:
    • Add EPERM + non-empty bounding set tolerance branch between the existing EPERM + empty (success) and catch-all (error) arms
    • Import warn from tracing
    • Update capability_bounding_set_clear_tolerates_nonempty_eperm test to assert is_ok() instead of is_err()
    • Simplify drop_privileges_succeeds_for_current_group test — remove conditional cfg(target_os) branching
  • .github/workflows/branch-checks.yml: Add rootless-caps CI job that runs supervisor capability tests as a non-root user without CAP_SETPCAP on ubuntu-24.04

Local Build & Test

Built and tested the fix locally before pushing:

  1. Cross-compiled supervisor as a static aarch64-unknown-linux-musl binary inside a rust:1.95-bookworm Podman container (macOS host cannot build Linux-only deps: capctl, landlock, seccompiler)
  2. Packaged into local OCI image localhost/openshell-supervisor:fix-2069 via Dockerfile.supervisor
  3. Configured gateway to use the local supervisor image (supervisor_image = "localhost/openshell-supervisor:fix-2069" in gateway.toml)
  4. Tested sandbox creation — sandbox reached Phase: Ready with the expected warning:
    CAP_SETPCAP is unavailable and the child capability bounding set is non-empty;
    the child process relies on seccomp and Landlock for confinement
    
  5. Ran a fullsend agent code review pipeline against the fixed supervisor — sandbox created (7.4s), bootstrapped, agent code copied into sandbox — all stages succeeded

Testing

  • cargo test -p openshell-supervisor-process --lib -- capability_bounding drop_privileges passes
  • Local supervisor build + sandbox creation test (rootless Podman on macOS)
  • Fullsend agent pipeline run with fixed supervisor image — sandbox lifecycle (create, bootstrap, code copy) verified
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@copy-pr-bot

copy-pr-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown

All contributors have signed the DCO ✍️ ✅
Posted by the DCO Assistant Lite bot.

@waynesun09

Copy link
Copy Markdown
Author

I have read the DCO document and I hereby sign the DCO.

@waynesun09

Copy link
Copy Markdown
Author

recheck

@maxamillion

Copy link
Copy Markdown
Collaborator

Few points of note:

  • Security boundary behavior changed but architecture docs were not updated.
    architecture/sandbox.md still says child capability bounding-set clearing is fail-closed and that EPERM is tolerated only when the set is already empty. This PR intentionally changes that invariant. The architecture doc should be updated in the same PR to describe the degraded rootless mode and its reliance on seccomp/Landlock.

  • The degraded path relies on Landlock, but Landlock may be best-effort.
    The warning says the child relies on “seccomp and Landlock,” but elsewhere Landlock can run in best-effort mode and continue unavailable or failed. If the bounding set remains non-empty and Landlock is unavailable/best-effort, the actual confinement story is weaker than the warning implies. Consider tightening the message or adding an explicit check/comment explaining acceptable residual risk.

  • Consider emitting this as an OCSF security/config event, not only tracing::warn!.
    Per project logging guidance, degraded sandbox controls and unavailable confinement primitives are operator-visible security posture events. This warning represents a confinement degradation and may warrant a structured OCSF finding or config-state event so it appears in sandbox security telemetry.

  • Commit metadata includes Assisted-by: Claude.
    Project instructions say commits must not mention Claude or AI agents. The commit bodies in this PR include Assisted-by: Claude; those should be removed before merge.

…unavailable

On rootless Podman with AppArmor unprivileged user namespace restrictions
(Ubuntu 24.04 default), CAP_SETPCAP is granted by Podman but blocked at
the kernel level by AppArmor. This causes drop_capability_bounding_set()
to fail with EPERM while the bounding set is non-empty — a case that was
unhandled, crashing sandbox creation.

Add a match arm that logs a warning and continues when EPERM is returned
with a non-empty bounding set. The child process is still confined by
seccomp.

Add a parent-side OCSF DetectionFinding alert
(log_capability_bounding_set_readiness) emitted before fork, following
the same pattern as Landlock's log_sandbox_readiness(). This ensures the
degraded confinement posture appears in structured security telemetry.

Fixes: NVIDIA#2069
Signed-off-by: Wayne Sun <gsun@redhat.com>
@waynesun09 waynesun09 force-pushed the fix-2069-cap-bounding-set-rootless-podman branch from 7320552 to 1dc253d Compare June 30, 2026 22:13
@TaylorMutch

Copy link
Copy Markdown
Collaborator

/ok to test 1dc253d

@waynesun09 waynesun09 force-pushed the fix-2069-cap-bounding-set-rootless-podman branch from 1dc253d to ad6106c Compare June 30, 2026 22:29
Run supervisor capability unit tests on a bare GHA ubuntu-24.04 runner
as an unprivileged user, where CAP_SETPCAP is unavailable and the
bounding set is non-empty. This exercises the EPERM + non-empty path
that caused the NVIDIA#2069 crash.

Update architecture/sandbox.md to describe the degraded rootless mode
and its reliance on seccomp when the bounding set cannot be cleared.

Refs: NVIDIA#2069
Signed-off-by: Wayne Sun <gsun@redhat.com>
@waynesun09 waynesun09 force-pushed the fix-2069-cap-bounding-set-rootless-podman branch from ad6106c to 3f95d51 Compare June 30, 2026 22:34
@elezar elezar assigned elezar and alangou and unassigned elezar Jul 1, 2026
@elezar

elezar commented Jul 1, 2026

Copy link
Copy Markdown
Member

@alangou could this have been introduced in #2001? Do you mind having a look?

@NVIDIA NVIDIA deleted a comment from copy-pr-bot Bot Jul 1, 2026
@alangou

alangou commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

/ok to test 3f95d51

@alangou

alangou commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

The new OCSF degraded-mode alert may not fire.

The parent-side probe returns early if CAP_SETPCAP is present in the effective set, but the commit message says Podman can grant CAP_SETPCAP while AppArmor still makes bounding::clear() fail with EPERM. Then we skip the parent DetectionFinding, and only hit the warn! inside pre_exec, which is exactly the context this patch says cannot reliably emit structured logs.

Could we make the readiness probe test the actual bounding-set clear behavior ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

supervisor v0.0.73 crashes in rootless Podman: drop_capability_bounding_set() EPERM with non-empty bounding set

5 participants