Fix benign OmniRtx*API schema errors with newton_rtx + ovphysx by matthewtrepte · Pull Request #7653 · isaac-sim/IsaacLab

matthewtrepte · 2026-09-08T21:20:24Z

Summary

Fixes benign [omni.rtx] FindAppliedAPIPrimDefinition(...) returned nothing errors when newton_rtx is used after another backend has initialized USD.
Treats the current newton_rtx viewer and OVRTXRendererCfg as OVRTX runtime consumers during the existing config scan.
Rejects incompatible Isaac Sim / Kit combinations before loading OVRTX, then calls OVRTX's own register_schema_paths() once before physics or user code can initialize USD.
Removes the public Isaac Lab setup helper, duplicate path resolution, visualizer-to-isaaclab_ov dependency, and best-effort fallback. Registration failures now fail at the selected runtime boundary.
Removes the obsolete NewtonViewerRTX environment-path workaround; pinned OVRTX 0.4.1 resolves its native library and schema paths itself.

The resulting PR is +21 LOC overall and -18 production LOC against its develop base.

Root cause

USD's plugin registry is initialized once per process. ovrtx.register_schema_paths() must therefore run before a physics backend or renderer first accesses USD. Previously, OVPhysX could initialize USD before Newton constructed ViewerRTX, leaving the OmniRtx*API schemas undiscoverable.

OVRTX already owns native-library discovery and schema-path publication. Isaac Lab only needs to invoke that primitive at the launch composition root after validating that no incompatible Kit runtime was selected.

Test plan

uv run --frozen --extra test python -m pytest source/isaaclab/test/app/test_launch_simulation_require_kit.py -q — 9 passed.
Related launcher and visualizer suites — 74 passed.
Verified a config-declared newton_rtx registers OVRTX before storage setup and user code.
Verified CLI newton_rtx with Isaac Sim PhysX is rejected before OVRTX is loaded.
With LD_LIBRARY_PATH and OMNI_USD_PLUGINS_BASE_PATH unset, verified ovrtx.register_schema_paths() publishes the USD plugin path and ovrtx.Renderer() constructs successfully.
Changed-file formatting, lint, spelling, RST, and changelog checks pass.

Release backport

Backport this pull request to the active release branch after it merges into develop

Fixes benign "[omni.rtx] FindAppliedAPIPrimDefinition(...) returned nothing" error logs when the newton_rtx visualizer is used alongside physics=ovphysx. USD's plug registry initializes once per process, typically triggered by the physics backend touching pxr well before any visualizer is constructed; registering ovrtx's schema paths later (inside NewtonViewerRTX) is too late. launch_simulation now prepares the ovrtx runtime up front, whenever newton_rtx is requested via CLI or config, before any physics backend can touch pxr. The prior per-visualizer patch is kept as a defense-in-depth fallback for callers that construct a viewer without going through launch_simulation, and both call sites now share one helper (isaaclab_ov.renderers.prepare_ovrtx_runtime) instead of duplicating the logic.

greptile-apps · 2026-09-08T21:23:15Z

Confidence Score: 4/5

The implementation is not ready to merge until the existing visualizer-intent test is updated for the newly forwarded key.

Findings

Stale Intent Assertion ▶

Summary

Adds early detection of CLI- and configuration-selected Newton RTX visualization.
Centralizes native library path setup and USD schema registration in prepare_ovrtx_runtime.
Preserves viewer-level preparation for construction paths that bypass launch_simulation.
Leaves one existing visualizer-intent test with a stale exact mapping expectation.

Diagram

sequenceDiagram
    participant Caller
    participant Launcher as launch_simulation
    participant OVRTX as prepare_ovrtx_runtime
    participant Physics
    participant Viewer as NewtonViewerRTX

    Caller->>Launcher: Request newton_rtx
    Launcher->>Launcher: Scan CLI/config visualizer intent
    Launcher->>OVRTX: Prepare library paths and schemas
    OVRTX-->>Launcher: Best-effort completion
    Launcher->>Physics: Initialize selected backend
    Launcher->>Viewer: Construct RTX visualizer
    Viewer->>OVRTX: Repeat preparation as fallback
    Viewer->>Viewer: Open render product

greptile-apps · 2026-09-08T21:23:19Z

        "has_kit_streaming_view": any(
            getattr(c, "visualizer_type", None) == "kit" and bool(getattr(c, "streaming_view", False)) for c in cfgs
        ),
+        "has_newton_rtx_visualizer": any(getattr(c, "visualizer_type", None) == "newton_rtx" for c in cfgs),


Stale intent assertion

Adding has_newton_rtx_visualizer changes the forwarded visualizer_intent mapping from three keys to four. However, test_launch_simulation_passes_visualizer_intent_to_applauncher still compares it with the exact three-key mapping, so the test now fails. Update the expected mapping to include the new key.

isaaclab-review-bot

Isaac Lab Review Bot

The early ovrtx schema registration addresses the USD plug-registry ordering issue, but the shared helper introduces an undeclared runtime dependency that can break supported package installations.

Design and architecture: Calling the helper from both launch_simulation and NewtonViewerRTX appropriately covers normal launch ordering and direct viewer construction. However, both paths now unconditionally cross into isaaclab_ov, although the affected packages do not declare isaaclab-ov as a dependency.
API: prepare_ovrtx_runtime is exported and documented consistently, and visualizer intent detection covers CLI and config requests. Its use is not dependency-safe: an installation containing isaaclab-visualizers, Newton, and ovrtx but not isaaclab-ov can now fail with ImportError rather than applying this best-effort mitigation.
Implementation: The helper preserves the previous Linux path setup and performs schema registration before ViewerRTX initialization. Guard the isaaclab_ov imports or declare the required package dependency so the optional log-noise fix does not become a hard runtime requirement.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

isaaclab-review-bot · 2026-09-08T21:25:09Z

+        # its USD schema/plugin paths with USD's plug registry before ``super().__init__()``
+        # (below) reaches ``ViewerRTX``, which imports ``ovrtx`` and opens a stage referencing
+        # ``OmniRtx*API`` schemas. See :func:`~isaaclab_ov.renderers.prepare_ovrtx_runtime` for why.
+        from isaaclab_ov.renderers import prepare_ovrtx_runtime


🟡 Warning · Design Architecture — Unguarded cross-package import of isaaclab_ov

The removed inline logic used only stdlib, so NewtonViewerRTX worked with newton + ovrtx alone. The replacement unconditionally imports isaaclab_ov.renderers, but isaaclab-visualizers declares only isaaclab in the lock metadata (and isaaclab itself declares nothing), so an install without isaaclab-ov now raises ImportError during viewer construction — and in launch_simulation (line 539) — for a purely best-effort log-noise fix. Guard both imports with try/except ImportError or declare the dependency.

- Update the stale 3-key visualizer_intent assertion for the new has_newton_rtx_visualizer key (Greptile). - Guard the new isaaclab_ov.renderers.prepare_ovrtx_runtime imports with try/except ImportError in both launch_simulation and NewtonViewerRTX, since neither isaaclab nor isaaclab_visualizers declares isaaclab_ov as a dependency and this is a best-effort log-noise mitigation, not something a newton_rtx launch should hard-depend on (isaaclab-review-bot).

kellyguo11 · 2026-09-08T23:39:46Z

run-ci

sim_launcher.py already imports OvPhysxCfg and OVRTXRendererCfg from isaaclab_ov unconditionally at module scope, so an install missing isaaclab_ov fails to import this module regardless of the guarded prepare_ovrtx_runtime call; the try/except suggested an optionality that did not exist here. The equivalent guard in NewtonViewerRTX.__init__ (isaaclab_visualizers, which does not depend on isaaclab_ov) is unaffected and stays as-is. Addresses huidongc's review feedback on PR isaac-sim#7653. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

kellyguo11 · 2026-09-10T01:59:44Z

run-ci

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

matthewtrepte · 2026-09-10T03:34:24Z

run-ci

isaaclab-bot · 2026-09-10T05:56:29Z

Backported to release/3.0.0 as 7192d5a.

## Summary - Fixes benign `[omni.rtx] FindAppliedAPIPrimDefinition(...) returned nothing` errors when `newton_rtx` is used after another backend has initialized USD. - Treats the current `newton_rtx` viewer and `OVRTXRendererCfg` as OVRTX runtime consumers during the existing config scan. - Rejects incompatible Isaac Sim / Kit combinations before loading OVRTX, then calls OVRTX's own `register_schema_paths()` once before physics or user code can initialize USD. - Removes the public Isaac Lab setup helper, duplicate path resolution, visualizer-to-`isaaclab_ov` dependency, and best-effort fallback. Registration failures now fail at the selected runtime boundary. - Removes the obsolete `NewtonViewerRTX` environment-path workaround; pinned OVRTX 0.4.1 resolves its native library and schema paths itself. The resulting PR is +21 LOC overall and -18 production LOC against its develop base. ## Root cause USD's plugin registry is initialized once per process. `ovrtx.register_schema_paths()` must therefore run before a physics backend or renderer first accesses USD. Previously, OVPhysX could initialize USD before Newton constructed `ViewerRTX`, leaving the `OmniRtx*API` schemas undiscoverable. OVRTX already owns native-library discovery and schema-path publication. Isaac Lab only needs to invoke that primitive at the launch composition root after validating that no incompatible Kit runtime was selected. ## Test plan - `uv run --frozen --extra test python -m pytest source/isaaclab/test/app/test_launch_simulation_require_kit.py -q` — 9 passed. - Related launcher and visualizer suites — 74 passed. - Verified a config-declared `newton_rtx` registers OVRTX before storage setup and user code. - Verified CLI `newton_rtx` with Isaac Sim PhysX is rejected before OVRTX is loaded. - With `LD_LIBRARY_PATH` and `OMNI_USD_PLUGINS_BASE_PATH` unset, verified `ovrtx.register_schema_paths()` publishes the USD plugin path and `ovrtx.Renderer()` constructs successfully. - Changed-file formatting, lint, spelling, RST, and changelog checks pass. ## Release backport - [x]  Backport this pull request to the active release branch after it merges into `develop` --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Octi Zhang <zhengyuz@nvidia.com> (cherry picked from commit eaf85e3)

matthewtrepte requested a review from a team September 8, 2026 21:20

matthewtrepte requested review from AntoineRichard, StafaH, fatimaanes, huidongc, kellyguo11, marcodiiga, ooctipus, pbarejko and rilei-nvidia as code owners September 8, 2026 21:20

github-actions Bot added the isaac-lab Related to Isaac Lab team label Sep 8, 2026

greptile-apps Bot reviewed Sep 8, 2026

View reviewed changes

isaaclab-review-bot Bot reviewed Sep 8, 2026

View reviewed changes

matthewtrepte requested a review from hujc7 as a code owner September 8, 2026 21:57

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 8, 2026

huidongc reviewed Sep 9, 2026

View reviewed changes

Comment thread source/isaaclab/isaaclab/app/sim_launcher.py Outdated

matthewtrepte and others added 4 commits September 9, 2026 20:17

Trim ovrtx-prep comment in launch_simulation

a5162e3

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Register OVRTX schemas after runtime validation

ee54694

Remove unrelated lockfile changes

5c23ced

huidongc approved these changes Sep 10, 2026

View reviewed changes

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 10, 2026

kellyguo11 approved these changes Sep 10, 2026

View reviewed changes

kellyguo11 enabled auto-merge (squash) September 10, 2026 02:00

Restore IsaacRtxRendererCfg guidance in OVRTX-vs-Kit error message

8813258

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 10, 2026

kellyguo11 merged commit eaf85e3 into isaac-sim:develop Sep 10, 2026
52 of 53 checks passed

Navigation Menu

Uh oh!

Fix benign OmniRtx*API schema errors with newton_rtx + ovphysx - #7653

Fix benign OmniRtx*API schema errors with newton_rtx + ovphysx#7653
kellyguo11 merged 7 commits into
isaac-sim:developfrom
matthewtrepte:mtrepte/ovrtx-applied-api-schema-warnings

matthewtrepte commented Sep 8, 2026 •

edited by ooctipus

Loading

Uh oh!

greptile-apps Bot commented Sep 8, 2026 •

edited

Loading

Summary

Diagram

Uh oh!

greptile-apps Bot Sep 8, 2026

Uh oh!

isaaclab-review-bot Bot left a comment

Uh oh!

isaaclab-review-bot Bot Sep 8, 2026

Uh oh!

kellyguo11 commented Sep 8, 2026

Uh oh!

Uh oh!

kellyguo11 commented Sep 10, 2026

Uh oh!

matthewtrepte commented Sep 10, 2026

Uh oh!

Uh oh!

isaaclab-bot Bot commented Sep 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

matthewtrepte commented Sep 8, 2026 • edited by ooctipus Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root cause

Test plan

Release backport

Uh oh!

greptile-apps Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 4/5

Findings

Summary

Diagram

Uh oh!

greptile-apps Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

Isaac Lab Review Bot

Uh oh!

isaaclab-review-bot Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

kellyguo11 commented Sep 8, 2026

Uh oh!

Uh oh!

kellyguo11 commented Sep 10, 2026

Uh oh!

matthewtrepte commented Sep 10, 2026

Uh oh!

Uh oh!

isaaclab-bot Bot commented Sep 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

matthewtrepte commented Sep 8, 2026 •

edited by ooctipus

Loading

greptile-apps Bot commented Sep 8, 2026 •

edited

Loading