IsaacSim Newton Physics: Physics Raycast Sensor Phasing Through Objects

[KEYL-PH]

Description

Issue can be replicated from an example here:
https://docs.isaacsim.omniverse.nvidia.com/6.0.1/sensors/isaacsim_sensors_physics_raycast.html

This issue also affects sensor readings.

Newton Physics Raycast Sensor:

Physx Raycast Sensor:

Isaac Sim version

6.0.1

Operating System (OS)

Windows 11

GPU Name

Any Nvidia GPU

GPU Driver and CUDA versions

Driver: 595.95, CUDA13.2

Logs

No response

Additional information

No response

[PeterL-NV]

Hi, thanks for reporting the issue. The physics raycast sensor performs its casts through the PhysX scene query interface. When Newton is the active engine, the PhysX scene is never populated with your colliders so every ray misses.
In the meantime, you can either use PhysX or use an RTX lidar from isaacsim.sensors.experimental.rtx, which does not depend on the PhysX scene and works under Newton.

[KEYL-PH]

Is there a way to make the physx raycast sensors work in newton or are there any alternatives that replicates raycast sensors?
The RTX lidar can't do single point proximity sensing.

[PeterL-NV]

On RTX lidar single point proximity sensing, RTX lidar firing patterns are authorable: OmniLidar prims configure emitter states via OmniSensorGenericLidarCoreEmitterStateAPI, and the custom-profile docs list "emitter patterns" as a supported configuration area. Refer to content://Isaac Sim/Sensors/NVIDIA/Simple_Example_Solid_State.usda as a minimal starting point. A 1-beam or few-beam config is expressible. The beam count is just the length of the emitterState arrays, which must agree with four scalars:

  numberOfEmitters = numberOfChannels = numLines = 1
  numRaysPerLine = [1]
  emitterState:s001:azimuthDeg = [0]      # and elevationDeg, bank, channelId,
                                          # fireTimeNs, focalDistM, focalSlope,
                                          # horOffsetM, vertOffsetM, rangeId,
                                          # distanceCorrectionM, emitterPeakPowerW

For Raycast sensor support in Newton, currently there're contact/IMU/camera sensors but no ray sensor, so you will need to wait for Newton to add that support.

[KEYL-PH]

I have a very crude implementation of raycast sensors in isaacsim (Newton Physics). As of now, it is limited to targeted meshes rather than existing collision bodies. Also, it is laggy as hell once there are more robots and raycast sensors. I plan to incorporate this into the newton physics sensors if possible.

Newton_RayCast.zip

[KEYL-PH]

More improvements. Now, it detects all collision bodies, and the endpoints of every rays now detects complex mesh bodies instead of it being limited to square collision bodies. Performance isn't that good yet, but it is tolerable for my case. It is also able to ignore self-collisions and bodies without collision body properties.

Sample File:
Newton_Raycast_Complete.zip

[PeterL-NV]

Hi, Thanks for sharing your work! I am sure this will be helpful to others with similar use cases. Let us know if you need further help.

[KEYL-PH]

@PeterL-NV
I have 2 issues with my implementation:

1. Loading times become slow as there are more complex meshes.

• This happens because the code tries to generate its own mesh cache rather than using the pre-generated meshes by the scene.
• How or where do you find this pre-generated collision meshes. What implementation was used for the Physx raycast sensors?
• I have tried implementing this pre-generated collision meshes but it only worked for SDF mesh. It did not work for bounding sphere and bounding cube. I have not tried the other mesh implementations yet, so I don't know if they worked or not.

2. When there are more complex meshes, the performance of the simulation plumets as compared to the Physx counterpart.

• What optimization or approach was used to not greatly impact the performance for Physx sensors?