feat(supervisor-middleware): add in-process egress middleware

[pimlock]

Summary

Implements the first usable RFC 0009 supervisor middleware slice: proto-backed, host-selected HTTP egress middleware for HttpRequest/pre_credentials, with both in-process built-ins and statically registered operator-run gRPC services.

The implementation covers RFC 0009 Phase 1 and adds basic external-service support from Phase 2. It establishes the contract, policy plumbing, ordered chain execution, built-in secret redaction, static gateway registration, relay integration, validation before policy persistence, body limits, audit events, and user-facing configuration and operations documentation.

Related Issue

Closes #2010
Part of #1733
Design/RFC: #1738

Changes

• Add the documented openshell.middleware.v1 protobuf contract with Describe, ValidateConfig, and EvaluateHttpRequest, plus generated Rust module wiring.
• Add the openshell-supervisor-middleware crate with a common service abstraction for built-in and operator-run middleware, ordered chain execution, fail-open/fail-closed handling, safe append-only header mutation, body replacement, metadata propagation, findings, and the built-in openshell/secrets redactor.
• Add static operator-run service registration through gateway TOML. The gateway connects to each service at startup, validates its manifest, rejects duplicate or reserved binding IDs, and supports plaintext http:// and TLS https:// endpoints using platform trust roots.
• Add operator-owned body limits for registered services. Each service advertises its capability through Describe, the gateway rejects configured limits above that capability, and built-ins use their code-owned advertised limits.
• Add the top-level ordered network_middlewares policy list. Middleware selection is independent of the network policy that admitted a request and uses case-insensitive destination-host include and exclude selectors, with exclusions taking precedence.
• Validate built-in and operator-run middleware configuration before policy persistence, reject duplicate config names and invalid selectors, and prevent required middleware from covering uninspectable tls: skip endpoints.
• Deliver only the operator-run services required by the effective policy to sandbox supervisors. Supervisors validate a new registry before installing it and retain the last-known-good registry when reload fails.
• Integrate middleware before credential injection across inspected HTTP relay paths, including L7 REST and JSON-RPC requests, credential-injection passthrough HTTP, and WebSocket upgrades before the connection becomes a raw stream.
• Buffer requests using the smallest limit in the selected chain. Declared over-capacity bodies may pass through only when every selected stage is fail-open and no body bytes were consumed; over-capacity chunked bodies are denied after consumption. The same per-stage limit applies to replacement bodies.
• Emit OCSF HTTP activity, configuration state changes, and detection findings for middleware decisions, failures, fail-open bypasses, over-capacity bodies, and findings without logging raw request values.
• Add the Supervisor Middleware extensibility documentation and update gateway configuration, policy schema, policy operations, and architecture documentation.

Testing

• mise run pre-commit passes
  • Rust formatting, clippy, workspace checks, workspace and server tests
  • Python formatting, linting, type checks, and tests
  • Protobuf generation, Markdown and Mermaid linting, Helm checks, license headers, packaging, SBOM, and docs website tests
• Unit and integration-style tests added or updated
  • cargo test -p openshell-supervisor-middleware
  • cargo test -p openshell-supervisor-network
  • cargo test -p openshell-server --features test-support
• E2E coverage considered
  • No separate gateway E2E was added. The request relay, policy validation, registry, and remote gRPC paths are covered by automated unit and integration-style tests.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated
• Published docs updated

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[pimlock]

/ok to test 9ebcdd5

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2027.docs.buildwithfern.com/openshell

[pimlock]

/ok to test 1fbcdbc

[pimlock]

/ok to test

[pimlock]

/ok to test

[pimlock]

/ok to test