feat(types): add reward contract, certificate, and evidence receipt

[abrichr]

Adds openadapt_types/reward.py: three strict, versioned contracts for training a policy against a verified terminal effect, plus the pure score() helper, packaged JSON Schemas, tests, and docs/REWARD.md.

What it adds

RewardContractV1 (openadapt.reward-contract/v1) binds task and environment by opaque id and digest, the required and forbidden effect contracts by digest, the independent oracle (an OracleChannel plus identity keys and an oracle contract digest), the reward components with weights, a scoring policy, and a certificate policy (epsilon, delta, threshold, calibration corpus digest, expiry in policy updates). Its digest is the canonical SHA-256 used elsewhere in this package; components sort by name so the digest does not depend on author order. The pinned digest in tests/test_reward.py will fail if the canonical form ever changes.

RewardCertificateV1 (openadapt.reward-certificate/v1) is the signed bound with issued_at_policy_update and expiry_policy_updates. is_current(policy_update) and state_at() answer whether a trainer may still use it. satisfies(policy) checks it against the contract's certificate policy.

RewardOutcomeV1 has seven members, each with one scoring class in REWARD_SCORING_CLASS: verified (admitted positive), halted_before_effect, refused, rejected_policy, wrong_effect (zero or a declared penalty), reconciliation_required and failed_platform (unscored). refused is separate from a runtime halt so a trainer can weigh them apart. wrong_effect is the silent wrong action the certificate bounds.

RewardEvidenceReceiptV1 (openadapt.reward-evidence-receipt/v1) binds the contract digest, policy checkpoint and update number, episode id, oracle tier, evidence digest, component vector, scalar (absent when unscored), certificate reference and state, uncertainty state, certified, development_only, issuer, nonce, timestamp, and an ed25519 signature.

score(outcome, tier, certificate, policy_update, *, scoring=DEFAULT_REWARD_SCORING) returns (scalar | None, certified, development_only).

Rules the types enforce

• An unscored outcome never becomes 0.0. score() returns None, the receipt refuses a scalar_reward for it, and RewardScoringPolicyV1 fixes uncertain_episodes and platform_failures to the literal "unscored".
• certified is true only at oracle tier 2 or 3 with a certificate whose state is current at the receipt's policy_update. A tier-0 or tier-1 receipt marked certified fails validation through RewardCertificationRefused.
• development_only must equal oracle_tier < 2.
• A verified scalar must be positive; a zero-or-penalty scalar must be at most zero; verified cannot carry an uncertainty state; reconciliation_required must.

Not an Execute Seal

The reward receipt states that OpenAdapt verified the terminal effect of one episode. It does not state that Flow governed the policy's actions. It has its own schema id and shares only generic plumbing with ExecuteEvidenceReceiptV1 (receipt_id, oracle_tier, evidence_digest, nonce, issued_at, schema_version). test_reward_receipt_is_not_an_execute_seal pins that shared set, asserts no Seal-specific field exists on the reward receipt, and checks that neither receipt validates as the other.

Reviewer checks

• uv run pytest is green locally (428 passed). uv build plus scripts/check_source_boundary.py --require-dist pass.
• Three new schema files ship under openadapt_types/schemas/; the README count moves from thirty-one to thirty-four and test_readme_claims.py extends its number table.
• No corpus contents, thresholds, or oracle recipes are in this package. Every one of those is a digest.
• The version bump and CHANGELOG entry come from python-semantic-release on the feat: squash commit, as with the previous releases in this repository.

Preregistration this follows: openadapt-evals docs/preregistrations/PREREGISTRATION_CERTIFIED_REWARD_RL_2026_08_25.md, section 3.6 and section 6.

Opened by an agent session, not the founder.

🤖 Generated with Claude Code