Skip to content

Add Newton MPM surface rendering to the teapot demo - #7640

Open
maxkra15 wants to merge 16 commits into
isaac-sim:developfrom
maxkra15:mpm-surface-extraction
Open

Add Newton MPM surface rendering to the teapot demo#7640
maxkra15 wants to merge 16 commits into
isaac-sim:developfrom
maxkra15:mpm-surface-extraction

Conversation

@maxkra15

@maxkra15 maxkra15 commented Sep 8, 2026

Copy link
Copy Markdown

Description

Add GPU surface reconstruction for the Newton MPM teapot-fill demo so fluid can be rendered as a coherent translucent surface instead of only as individual particles.

The implementation follows Newton 1.6's reusable ParticleSurface workflow: reconstruction storage is allocated once, CUDA work is graph-captured when enabled, and only the latest dynamic mesh update is staged for the next viewer frame. No new dependency is introduced; the change uses the Newton 1.6 version already pinned on develop.

The demo now provides:

  • --fluid_render_mode particles|surface|both, defaulting to surface
  • reconstructed dynamic surfaces in Newton GL and Newton RTX
  • translucent water rendering, a closer initial camera, and less viscous water parameters
  • consistent particle color configuration for Newton GL and Newton RTX
  • concurrent Newton visualizer support
  • the standard Isaac Lab ground plane
  • optional kitchen-island and bowl presentation assets in Newton RTX
  • a user guide covering launch modes, integration lifecycle, tuning, and the complete reusable helper

To keep the viewer lifecycle encapsulated, NewtonVisualizer.log_mesh() stages mesh registrations and topology updates and submits them between the viewer's begin_frame() and end_frame() calls. It supports interactive, paused, and headless rendering paths while coalescing repeated updates by mesh name.

This also fixes two rendering issues exposed by the demo:

  • NewtonRTXVisualizer now honors its configured particle color for Newton's canonical particle batch without changing user-defined point clouds.
  • Newton-generated proxy-collider visual meshes and nested static colliders remain hidden when an imported rigid-body root already has separate visual geometry.

The Kit visualizer remains particle-only because this surface path targets Newton's GL/RTX viewer mesh API.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Documentation update

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Screenshots

No screenshot attached. The teapot-fill demo was exercised interactively with Newton GL, Newton RTX, and Kit during development.

Validation

  • uv run --no-sync pytest source/isaaclab_visualizers/test/test_newton_adapter.py source/isaaclab_newton/test/cloner/test_visual_shape_import.py -q — 69 passed
  • uv run --no-sync isaaclab -f — passed
  • uv run --no-sync python tools/changelog/cli.py check develop — passed
  • uv run --no-sync python scripts/demos/mpm/teapot_fill.py --help — passed
  • uv run --isolated --extra dev -- make -C docs current-docs — passed with warnings treated as errors
  • Manual teapot-fill playback with Newton GL, Newton RTX, and Kit

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab -f
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective and that the feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • My name already exists in CONTRIBUTORS.md

@maxkra15
maxkra15 requested a review from a team September 8, 2026 13:17
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Sep 8, 2026
@greptile-apps

greptile-apps Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds GPU-reconstructed MPM fluid surfaces to the teapot demo and introduces staged dynamic-mesh submission for Newton GL and RTX viewers. It also applies RTX particle coloring consistently and improves filtering of generated and nested collider visuals.

  • Adds selectable particle, surface, and combined fluid-rendering modes.
  • Reconstructs the fluid surface with Newton ParticleSurface, including CUDA graph capture.
  • Coalesces mesh updates by path and submits them within Newton viewer frame boundaries.
  • Extends collider-visibility and visualizer behavior tests.
  • The Newton GL paused lifecycle still does not publish staged mesh updates.

Confidence Score: 4/5

The PR appears safe to merge after addressing the non-blocking Newton GL paused-mesh publication gap.

Active, headless on-demand, and RTX paused rendering paths submit staged meshes correctly, but a one-time mesh registration or update made while Newton GL is paused remains queued until rendering resumes.

Files Needing Attention: source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py

Important Files Changed

Filename Overview
scripts/demos/mpm/teapot_fill.py Adds configurable particle/surface rendering, reusable surface reconstruction, presentation changes, and revised fluid parameters.
source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py Adds staged dynamic-mesh logging and RTX particle-color handling, but GL does not flush staged meshes while paused.
source/isaaclab_newton/isaaclab_newton/cloner/newton_clone_utils.py Hides generated proxy visuals and resolves nested static collider visibility from the nearest rigid-body owner.
source/isaaclab_visualizers/test/test_newton_adapter.py Covers mesh staging, coalescing, active-frame submission, and RTX particle colors, but omits the GL paused lifecycle.
source/isaaclab_newton/test/cloner/test_visual_shape_import.py Adds focused coverage for nested static colliders, standalone colliders, and generated proxy visuals.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[MPM simulation step] --> B[ParticleSurface extraction]
  B --> C[FluidSurfaceRenderer.log_mesh]
  C --> D[Per-visualizer pending mesh map]
  D --> E{Viewer lifecycle}
  E -->|Newton GL/RTX active frame| F[Begin frame]
  F --> G[Log state and staged mesh]
  G --> H[End frame]
  E -->|Newton RTX paused| I[Paused frame and staged mesh flush]
  E -->|Newton GL paused| J[UI update only]
Loading

Reviews (1): Last reviewed commit: "Support concurrent Newton teapot visuali..." | Re-trigger Greptile

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The staged Newton mesh API and teapot surface reconstruction are generally well integrated, but two compatibility paths need correction: the demo no longer configures the previously supported newton selector, and staged meshes are not published for models with no rigid bodies.

  • Design and architecture: Name-keyed mesh staging appropriately encapsulates Newton's begin_frame()/end_frame() lifecycle across interactive, paused, and headless rendering. However, the interactive step path returns when body_q is empty before flushing pending meshes, so particle-only or otherwise body-less models cannot publish staged geometry despite the documented next-frame contract.
  • API: log_mesh() closely follows the Newton viewer signature and coalesces updates as intended. The demo's visualizer configuration now recognizes only newton_gl and newton_rtx, dropping the prior newton alias even though the surface-renderer selection still recognizes it; the selector must be handled consistently to preserve existing behavior.
  • Implementation: The surface renderer reuses reconstruction storage, separates CUDA graph capture from physics capture, handles empty meshes, and stages updates through the visualizer API. The accepted issues are confined to selector handling and the empty-body early-return placement; the cloner's generated proxy-visual hiding rule is explicitly implemented and tested as broader behavior, so the proposed objection to that rule is not sufficiently supported as a defect.

Minor fixes needed. Posted 2 actionable findings inline.

Automated review; human maintainers own approval decisions.

Comment thread scripts/demos/mpm/teapot_fill.py
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Sep 8, 2026
@maxkra15

maxkra15 commented Sep 8, 2026

Copy link
Copy Markdown
Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 8, 2026
@maxkra15

maxkra15 commented Sep 8, 2026

Copy link
Copy Markdown
Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 8, 2026
@kellyguo11 kellyguo11 moved this to In review in Isaac Lab Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants