Skip to content

docs: add Seal as an RL reward - #196

Merged
abrichr merged 2 commits into
mainfrom
claude/docs-seal-reward
Sep 2, 2026
Merged

docs: add Seal as an RL reward#196
abrichr merged 2 commits into
mainfrom
claude/docs-seal-reward

Conversation

@abrichr

@abrichr abrichr commented Sep 2, 2026

Copy link
Copy Markdown
Member

Adds docs/commercial/seal-reward.md, "Seal as an RL reward", to the Commercial section beside The Seal and the Execute page, with back-links from The Seal and Effect verification.

What the page states

  • The rule: reward 1 only on a tier-2 or tier-3 verified; screen-only evidence is never certified; a reward receipt is not an Execute Seal; unscored episodes are dropped, never paid 0.
  • What runs where: organization worker, OpenAdapt control service, trainer node.
  • The outcome-to-scalar table and the certified rule, including calibration_scope (synthetic today; production needs the unpublished Phase-1 calibration). No epsilon or delta value appears on the page.
  • Wiring: the OpenAI grader route, TRL CertifiedRewardFunction, verl CertifiedRewardManager, and the openadapt-mockmed-extradup verifiers environment.
  • The certificate fields, expiry in policy updates, re-certification, and a link to the public preregistration.
  • The MockMed run and the evals proof command.

Where every command comes from

Item Source
RewardContractV1, RewardCertificateV1, RewardOutcomeV1, RewardEvidenceReceiptV1, score, the outcome table openadapt-types#37 (merged, 0.16.0)
calibration_scope, issuer, production_certified openadapt-types#38 (merged, 0.17.0)
HttpRewardEndpoint, CertifiedRewardFunction, metadata_columns, as_async, CertifiedRewardManager, register_with_verl, python -m openadapt_evals.reward.proof openadapt-evals#326 (merged)
environments/openadapt_mockmed_extradup, vf-eval, scripted_policy.py serve, check_fails_closed.py openadapt-evals#324 (merged); the git-subdirectory install was run locally on 2026-09-02
openadapt-flow serve-reward, --contract, --port, --seed-mockmed, the routes, the envelope fields, the MockMed descriptors openadapt-flow#452 (merged, f90f723b)

Checks run locally

uv run mkdocs build --strict, uv run python scripts/validate_docs.py, and tests/test_validate_docs.py pass.

Opened by an agent session, not the founder.

🤖 Generated with Claude Code

abrichr and others added 2 commits September 1, 2026 20:47
Adds docs/commercial/seal-reward.md to the Commercial section beside The
Seal and the Execute page, and links it from The Seal and Effect
verification. Types and evals commands come from merged code; the flow
worker commands are pinned to openadapt-flow#452 until it merges.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@abrichr
abrichr merged commit cf2d198 into main Sep 2, 2026
8 checks passed
@abrichr
abrichr deleted the claude/docs-seal-reward branch September 2, 2026 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant