Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/CONTRACTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,10 @@ vector, the scalar, and the certificate state.

`reconciliation_required` and `failed_platform` are unscored. They carry no
scalar and the contract cannot map them to zero. `certified` is true only at
oracle tier 2 or 3 with a current certificate. Tier 0 and 1 receipts are
`development_only`.
oracle tier 2 or 3 with a current certificate, a calibration corpus digest,
and a stated `calibration_scope`. A self-signed certificate may carry only
`synthetic` scope, and today that is the only scope anyone can compute. Tier
0 and 1 receipts are `development_only`.

The reward receipt is not an Execute Seal. It has its own schema id and none
of the Seal's fields. It says OpenAdapt verified one episode's terminal
Expand Down
34 changes: 25 additions & 9 deletions docs/REWARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,32 @@ SHA-256 over sorted JSON. Components sort by name, so two authors who list them
in a different order get the same digest.

`RewardCertificateV1` is the bound: `epsilon`, `delta`, `threshold`, the
calibration corpus digest, the checker configuration digest, the policy update
it was issued at, and its expiry in policy updates. It is signed. Expiry
counts updates, not hours, because on-policy training breaks the
exchangeability the bound assumes. `is_current(policy_update)` answers whether
a trainer may still use it.
calibration corpus digest, the calibration scope, the checker configuration
digest, the issuer, the policy update it was issued at, and its expiry in
policy updates. It is signed. Expiry counts updates, not hours, because
on-policy training breaks the exchangeability the bound assumes.
`is_current(policy_update)` answers whether a trainer may still use it.

`RewardEvidenceReceiptV1` binds the contract digest, the policy checkpoint and
update number, the episode, the oracle tier, the evidence digest, the
component vector, the scalar, the certificate reference and its state, and two
booleans a trainer must read: `certified` and `development_only`.
component vector, the scalar, the certificate reference and its state, the
calibration corpus digest and scope, and two booleans a trainer must read:
`certified` and `development_only`.

## Synthetic scope is the only scope today

Today the only certificate anyone can compute is against the synthetic
MockMed/ExtraDup corpus, so its `calibration_scope` is `synthetic`. A
`production` scope needs the Phase-1 calibration, which is not published.
Until that changes, the word "certified" in any public text about a reward
must sit next to the word "synthetic".

The types hold that line. A certificate with `issuer: self_signed` may carry
only `synthetic` scope; `self_signed` plus `production` does not validate. A
receipt is `certified` only when the oracle tier is 2 or 3, the certificate is
current, the calibration corpus digest is present, and the scope is stated.
`production_certified` is true only when that scope is `production`, which no
self-signed certificate can reach.

## Outcome to scalar

Expand All @@ -58,8 +74,8 @@ failure this contract exists to stop.
| --- | --- | --- |
| 0 (visual, OCR) | yes | never |
| 1 (second session) | yes | never |
| 2 (API, DB, file, ack) | no | with a current certificate |
| 3 (counterparty) | no | with a current certificate |
| 2 (API, DB, file, ack) | no | with a current certificate, corpus digest, and stated scope |
| 3 (counterparty) | no | with a current certificate, corpus digest, and stated scope |

The tier comes from the oracle channel, as it does for every Seal.
`refuse_development_certification` raises `RewardCertificationRefused` for
Expand Down
4 changes: 4 additions & 0 deletions openadapt_types/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,8 @@
REWARD_EVIDENCE_RECEIPT_SCHEMA,
REWARD_SCORING_CLASS,
UNSCORED_REWARD_OUTCOMES,
RewardCalibrationScopeV1,
RewardCertificateIssuerV1,
RewardCertificatePolicyV1,
RewardCertificateStateV1,
RewardCertificateV1,
Expand Down Expand Up @@ -645,6 +647,8 @@
"REWARD_EVIDENCE_RECEIPT_SCHEMA",
"REWARD_SCORING_CLASS",
"UNSCORED_REWARD_OUTCOMES",
"RewardCalibrationScopeV1",
"RewardCertificateIssuerV1",
"RewardCertificatePolicyV1",
"RewardCertificateStateV1",
"RewardCertificateV1",
Expand Down
79 changes: 74 additions & 5 deletions openadapt_types/reward.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,30 @@ class RewardUncertaintyStateV1(str, Enum):
ORACLE_UNAVAILABLE = "oracle_unavailable"


class RewardCalibrationScopeV1(str, Enum):
"""What corpus the certificate was calibrated against.

``synthetic`` is the only scope anyone can compute today. ``production``
requires the Phase-1 calibration, which is not published. A consumer
must show the scope beside the word certified.
"""

SYNTHETIC = "synthetic"
PRODUCTION = "production"


class RewardCertificateIssuerV1(str, Enum):
"""Who signed the certificate.

``self_signed`` is a certificate the trainer computed for itself. It may
carry only ``synthetic`` scope. ``organization`` is an organization node
that holds the calibration corpus and the signing key.
"""

SELF_SIGNED = "self_signed"
ORGANIZATION = "organization"


class RewardCertificationRefused(ValueError):
"""Raised when a tier-0 or tier-1 reward receipt is marked certified."""

Expand Down Expand Up @@ -335,9 +359,11 @@ class RewardCertificateV1(_StrictContract):
delta: StrictFloat = Field(gt=0.0, lt=1.0, allow_inf_nan=False)
threshold: StrictFloat = Field(allow_inf_nan=False)
calibration_corpus_digest: StrictStr = Field(pattern=_SHA256_PATTERN)
calibration_scope: RewardCalibrationScopeV1
issued_at_policy_update: StrictInt = Field(ge=0, le=_MAX_POLICY_UPDATES)
expiry_policy_updates: StrictInt = Field(ge=1, le=_MAX_POLICY_UPDATES)
issued_at: StrictStr = Field(pattern=_TIMESTAMP_PATTERN)
issuer: RewardCertificateIssuerV1
issuer_key_id: StrictStr = Field(pattern=_OPAQUE_ID_PATTERN)
signature_algorithm: Literal["ed25519"] = "ed25519"
signature: StrictStr = Field(min_length=88, max_length=88)
Expand All @@ -350,6 +376,13 @@ def _signature(cls, value: str) -> str:
@model_validator(mode="after")
def _issue_window(self) -> RewardCertificateV1:
_parse_timestamp(self.issued_at, "issued_at")
if (
self.issuer is RewardCertificateIssuerV1.SELF_SIGNED
and self.calibration_scope is not RewardCalibrationScopeV1.SYNTHETIC
):
raise ValueError(
"a self-signed reward certificate may only carry synthetic scope"
)
if self.issued_at_policy_update + self.expiry_policy_updates > _MAX_POLICY_UPDATES:
raise ValueError("reward certificate expiry overflows the update counter")
return self
Expand Down Expand Up @@ -423,7 +456,10 @@ def score(
* ``scalar`` is ``None`` for ``RECONCILIATION_REQUIRED`` and
``FAILED_PLATFORM``. A trainer must drop or hold those episodes.
* ``certified`` is true only at tier 2 or 3 with a certificate that is
current at ``policy_update``.
current at ``policy_update``, names its calibration corpus by digest,
and states its calibration scope. A self-signed certificate can state
only ``synthetic`` scope, so a self-signed certificate alone never
yields a production-scope certification.
* ``development_only`` is true at tier 0 or 1. A tier-0 reward can train
a local experiment. It can never be certified.
"""
Expand All @@ -432,7 +468,13 @@ def score(
raise ValueError("policy_update must be non-negative")
development_only = int(tier) < REWARD_CERTIFIED_MINIMUM_TIER
state = certificate_state(certificate, policy_update)
certified = not development_only and state is RewardCertificateStateV1.CURRENT
certified = (
not development_only
and certificate is not None
and state is RewardCertificateStateV1.CURRENT
and bool(certificate.calibration_corpus_digest)
and certificate.calibration_scope in RewardCalibrationScopeV1
)
scalar = scoring.scalar_for(RewardOutcomeV1(outcome))
return RewardScoreV1(scalar, certified, development_only)

Expand Down Expand Up @@ -470,6 +512,10 @@ class RewardEvidenceReceiptV1(_StrictContract):
certificate_id: StrictStr | None = Field(default=None, pattern=_OPAQUE_ID_PATTERN)
certificate_digest: StrictStr | None = Field(default=None, pattern=_SHA256_PATTERN)
certificate_state: RewardCertificateStateV1
calibration_corpus_digest: StrictStr | None = Field(
default=None, pattern=_SHA256_PATTERN
)
calibration_scope: RewardCalibrationScopeV1 | None = None
uncertainty: RewardUncertaintyStateV1
certified: StrictBool
development_only: StrictBool
Expand Down Expand Up @@ -505,14 +551,28 @@ def _scoring_contract(self) -> RewardEvidenceReceiptV1:
refuse_development_certification(self.oracle_tier)
if self.certificate_state is not RewardCertificateStateV1.CURRENT:
raise ValueError("a certified reward requires a current certificate")
if self.calibration_corpus_digest is None:
raise ValueError("a certified reward requires a calibration corpus digest")
if self.calibration_scope is None:
raise ValueError("a certified reward requires a stated calibration scope")
has_reference = (
self.certificate_id is not None or self.certificate_digest is not None
self.certificate_id is not None
or self.certificate_digest is not None
or self.calibration_corpus_digest is not None
or self.calibration_scope is not None
)
if self.certificate_state is RewardCertificateStateV1.ABSENT:
if has_reference:
raise ValueError("an absent certificate cannot carry a reference")
elif self.certificate_id is None or self.certificate_digest is None:
raise ValueError("a referenced certificate requires id and digest")
elif (
self.certificate_id is None
or self.certificate_digest is None
or self.calibration_corpus_digest is None
or self.calibration_scope is None
):
raise ValueError(
"a referenced certificate requires id, digest, corpus digest, and scope"
)

scoring_class = REWARD_SCORING_CLASS[self.reward_outcome]
if scoring_class is RewardScoringClassV1.UNSCORED:
Expand Down Expand Up @@ -551,6 +611,15 @@ def _scoring_contract(self) -> RewardEvidenceReceiptV1:
def scoring_class(self) -> RewardScoringClassV1:
return REWARD_SCORING_CLASS[self.reward_outcome]

@property
def production_certified(self) -> bool:
"""True only for a certified receipt whose scope is ``production``."""

return (
self.certified
and self.calibration_scope is RewardCalibrationScopeV1.PRODUCTION
)

def unsigned_payload(self) -> dict[str, Any]:
return self.model_dump(
mode="json",
Expand Down
28 changes: 28 additions & 0 deletions openadapt_types/schemas/reward-certificate-v1.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,24 @@
{
"$defs": {
"RewardCalibrationScopeV1": {
"description": "What corpus the certificate was calibrated against.\n\n``synthetic`` is the only scope anyone can compute today. ``production``\nrequires the Phase-1 calibration, which is not published. A consumer\nmust show the scope beside the word certified.",
"enum": [
"synthetic",
"production"
],
"title": "RewardCalibrationScopeV1",
"type": "string"
},
"RewardCertificateIssuerV1": {
"description": "Who signed the certificate.\n\n``self_signed`` is a certificate the trainer computed for itself. It may\ncarry only ``synthetic`` scope. ``organization`` is an organization node\nthat holds the calibration corpus and the signing key.",
"enum": [
"self_signed",
"organization"
],
"title": "RewardCertificateIssuerV1",
"type": "string"
}
},
"additionalProperties": false,
"description": "A signed, expiring bound on one reward contract's false-accept rate.\n\nExpiry counts policy updates, not wall-clock time. A certificate issued\nat update ``i`` with expiry ``n`` is current for updates ``i`` through\n``i + n - 1``. Revocation is a separate list keyed by ``certificate_id``\nand is checked by the issuer, as it is for every other admission.",
"properties": {
Expand All @@ -7,6 +27,9 @@
"title": "Calibration Corpus Digest",
"type": "string"
},
"calibration_scope": {
"$ref": "#/$defs/RewardCalibrationScopeV1"
},
"certificate_id": {
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$",
"title": "Certificate Id",
Expand Down Expand Up @@ -46,6 +69,9 @@
"title": "Issued At Policy Update",
"type": "integer"
},
"issuer": {
"$ref": "#/$defs/RewardCertificateIssuerV1"
},
"issuer_key_id": {
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$",
"title": "Issuer Key Id",
Expand Down Expand Up @@ -87,9 +113,11 @@
"delta",
"threshold",
"calibration_corpus_digest",
"calibration_scope",
"issued_at_policy_update",
"expiry_policy_updates",
"issued_at",
"issuer",
"issuer_key_id",
"signature"
],
Expand Down
33 changes: 33 additions & 0 deletions openadapt_types/schemas/reward-evidence-receipt-v1.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
{
"$defs": {
"RewardCalibrationScopeV1": {
"description": "What corpus the certificate was calibrated against.\n\n``synthetic`` is the only scope anyone can compute today. ``production``\nrequires the Phase-1 calibration, which is not published. A consumer\nmust show the scope beside the word certified.",
"enum": [
"synthetic",
"production"
],
"title": "RewardCalibrationScopeV1",
"type": "string"
},
"RewardCertificateStateV1": {
"enum": [
"absent",
Expand Down Expand Up @@ -38,6 +47,30 @@
"additionalProperties": false,
"description": "A signed statement that one episode's terminal effect was verified.\n\nThis receipt binds a reward contract, a policy checkpoint, an episode,\nand the oracle read. It is not an Execute Seal. It carries no\n``execution_id``, ``workflow_digest``, ``qualification_id``, or Execute\ncontract block, and it does not claim that Flow governed the policy.",
"properties": {
"calibration_corpus_digest": {
"anyOf": [
{
"pattern": "^sha256:[a-f0-9]{64}$",
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Calibration Corpus Digest"
},
"calibration_scope": {
"anyOf": [
{
"$ref": "#/$defs/RewardCalibrationScopeV1"
},
{
"type": "null"
}
],
"default": null
},
"certificate_digest": {
"anyOf": [
{
Expand Down
Loading
Loading