[Question] Is there a native method to sample point clouds at the surface of the mesh?

[ProNeverFake]

Question

I'm now developing a motion planning module in isaaclab. For this, I need a geometrical & spatial representation of the world (for collision check in/outside isaaclab). Is there a native method in isaaclab for this purpose? For example, sampling a number of points on the surface of a mesh in the USD stage (specified by providing the prim path), or getting a number of geometry primitives that can approximate a mesh in the USD stage?

[RandomOakForest]

Thank you for posting this. You can leverage several native features in Isaac Lab and Isaac Sim, especially through their integration with USD and PhysX. I'll move this post to our Discussions for follow up. Here’s a summary that may help.

Collision Geometry Representation in Isaac Lab & Isaac Sim

Collision Approximations:

• Isaac Sim natively supports several types of collision geometry approximations for meshes:
  • Convex Hull (default): Efficient and suitable for most rigid bodies.
  • Convex Decomposition: Breaks complex meshes into multiple convex parts for better accuracy.
  • Primitive Shapes: Box, sphere, and capsule approximations are available for simple or regular shapes.
  • SDF Meshes (Signed Distance Field): For high-fidelity collision, especially with complex or non-convex surfaces¹.

You can apply these approximations via the Omniverse UI or programmatically using Python. For example, to apply a convex hull or convex decomposition:

from pxr import UsdPhysics, PhysxSchema

prim = stage.GetPrimAtPath("/Path/To/Prim")
# Apply Convex Hull
UsdPhysics.CollisionAPI.Apply(prim)
PhysxSchema.PhysxCollisionAPI(prim).CreateCollisionApproximationAttr("convexHull")
# Or for convex decomposition
PhysxSchema.PhysxCollisionAPI(prim).CreateCollisionApproximationAttr("convexDecomposition")

This sets up the collision geometry that PhysX will use for simulation and collision queries¹.

Collision Visualization:

• You can visualize the actual collision geometry used in simulation:
  • In the viewport, select the eye icon → Show by type → Physics → Colliders.
  • For a solid mesh view, use Window → Simulation → Debug → Collision Mesh Debug Visualization¹.

Surface Sampling and Geometry Primitives

Surface Point Sampling:

• As of now, Isaac Lab and Isaac Sim do not provide a direct, built-in API to sample arbitrary points on the surface of a mesh via a simple function call in the Python API.
• However, you can access the mesh geometry data using the USD Python API (UsdGeom.Mesh) to extract vertices and faces, and then implement your own surface sampling logic. This involves:
  • Reading the mesh’s vertex positions and face indices.
  • Sampling points on the triangles (e.g., via barycentric coordinates).
• This approach is standard in robotics and graphics when native sampling utilities are not present.

Accessing Geometry Primitives:

• For collision purposes, you can programmatically retrieve or assign primitive colliders (box, sphere, capsule) to USD prims as described above.
• The bounding box of any USD prim can be computed using:

from pxr import UsdGeom
boundable = UsdGeom.Boundable(prim)
bbox = boundable.ComputeWorldBound(0, "default")

This gives you an axis-aligned bounding box which can be used for broad-phase collision checks¹.

Integration with Motion Planning

• For motion planning, you typically use the collision approximations (convex hulls, decompositions, primitives) for fast collision queries.
• For finer collision checks or sampling-based planners, you may need to extract mesh geometry and perform your own sampling or mesh processing, as described above.

Summary Table

Task	Native Support	Method/Workaround
Assigning collision geometry	Yes	Use Physics/PhysX APIs for convex hull, decomposition, primitives
Visualizing collision geometry	Yes	Physics debug visualization in UI
Surface point sampling on mesh	No direct API	Use USD API to extract mesh, implement sampling in Python
Getting bounding boxes/primitives	Yes	Use UsdGeom.Boundable.ComputeWorldBound

References

Footnotes

1. https://docs.omniverse.nvidia.com/isaacsim/latest/simulation_fundamentals.html ↩ ↩² ↩³ ↩⁴