Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@

### Features

- Add `window_tree` structural observations and a fail-closed authoring
projector. The raw tree may persist on disk for compile, except password
and secure-field values. Native AX, UIA, AT-SPI, and ARIA roles map onto
the types ElementRole enum. The vendor-wire payload is
`openadapt.authoring.observe/v1` without values, titles, or screenshots.
The observe fixture is pinned to openadapt-types PR 35. OS-injected input
still does not persist.
- Add attended authentication handoffs. Capture suppresses sensitive source
data, seals a bounded method marker, and retains a fresh exact frame before
native input resumes. The same retry-safe operations are available through
Expand Down
12 changes: 12 additions & 0 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,18 @@ typelib/runtime, and an interactive desktop accessibility bus.
The native provider describes the local accessibility tree. It does not
describe controls inside an RDP or Citrix pixel stream.

Authoring adds `query_kind: "window_tree"`. The raw tree may persist on disk
for compile. Password and secure-field values are omitted from that tree.
`openadapt_capture.authoring_project` projects the tree into
`openadapt.authoring.observe/v1` for a vendor wire: no values, titles,
screenshots, or extra keys. Native AX, UIA, AT-SPI, and ARIA roles map onto
the types `ElementRole` enum; unmapped roles never reach the wire. The
observe fixture is pinned to `openadapt-types` PR 35. Names and automation
ids that fail the coach-hint bar (length, `://`, `@`, six or more digits,
phone, SSN, email) or the types projected-label grammar are dropped. RDP and
Citrix observe payloads are coach-only with an empty tree. Capture still
drops OS-injected input; there is no `record_injected` API.

## Video and frame timing

Capture does not import, link, or bundle FFmpeg, and it downloads nothing on
Expand Down
15 changes: 15 additions & 0 deletions openadapt_capture/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@
FreshFrameProof,
load_authentication_handoffs,
)
from openadapt_capture.authoring_project import (
AUTHORING_OBSERVE_SCHEMA_VERSION,
AuthoringObserve,
AuthoringProjection,
project_authoring_observe,
)
from openadapt_capture.browser_events import (
BoundingBox,
BrowserClickEvent,
Expand Down Expand Up @@ -135,9 +141,11 @@
StructuralObservationRequest,
StructuralObserver,
StructuralProcessIdentity,
StructuralTreeNode,
StructuralWindowIdentity,
create_structural_observer,
observe_structural_action,
observe_window_tree,
)

# Visualization
Expand Down Expand Up @@ -188,9 +196,16 @@
"StructuralObservationRequest",
"StructuralObserver",
"StructuralProcessIdentity",
"StructuralTreeNode",
"StructuralWindowIdentity",
"create_structural_observer",
"observe_structural_action",
"observe_window_tree",
# Authoring observe projector (vendor wire; no values/titles/screenshots)
"AUTHORING_OBSERVE_SCHEMA_VERSION",
"AuthoringObserve",
"AuthoringProjection",
"project_authoring_observe",
# Window-scoped capture
"WindowTarget",
"TargetWindow",
Expand Down
Loading