isaac-patterns

uneezaismail's avatarfrom uneezaismail

NVIDIA Isaac Sim and Isaac ROS integration patterns for Physical AI applications.

⭐1stars🔀0forks📁View on GitHub🕐Updated Dec 8, 2025
[Architecture Design]

When & Why to Use This Skill

This Claude skill provides specialized integration patterns for NVIDIA Isaac Sim and Isaac ROS, designed to accelerate the development of Physical AI applications. It bridges the gap between simulation and reality by offering structured templates for USD scene organization, hardware-accelerated perception pipelines, and automated sim-to-real workflows, ensuring optimal performance on NVIDIA Jetson and RTX platforms.

Use Cases

  • Designing scalable USD-based simulation environments for humanoid and mobile robots with proper physics and sensor integration.
  • Implementing high-performance VSLAM and perception pipelines using Isaac ROS nodes for real-time navigation on Jetson Orin.
  • Developing sim-to-real strategies using Python-based domain randomization to improve the robustness of AI models in unpredictable real-world environments.
  • Optimizing hardware resource allocation and VRAM usage for complex, ray-traced robotic simulations.
nameisaac-patterns
descriptionNVIDIA Isaac Sim and Isaac ROS integration patterns for Physical AI applications.

Isaac Sim USD Scene Structure

Asset Organization (Universal Scene Description)

/World
├── /Environment
│   ├── /Ground_Plane (physics material: friction 0.8)
│   ├── /Obstacles (procedurally generated for domain randomization)
│   └── /Lighting
│       ├── /DomeLight (HDRI environment map)
│       └── /DistantLight (sun simulation)
├── /Robot
│   ├── /Humanoid (USD reference to robot asset)
│   ├── /Sensors
│   │   ├── /Camera_RGB (1920x1080, 60 FPS)
│   │   ├── /Camera_Depth (640x480, 30 FPS)
│   │   └── /Lidar (360°, 0.1° resolution)
│   └── /ActionGraph (OmniGraph for ROS 2 bridge)
└── /PhysicsScene (gravity: -9.81 m/s², time step: 1/60s)

Isaac ROS VSLAM Pipeline

Hardware-Accelerated Workflow

RealSense D435i → isaac_ros_visual_slam → nav2_map_server → nav2_planner → cmd_vel
                     ↓ (CUDA accelerated)
                  Pose Estimation (30-60 FPS on Jetson Orin)

Key Isaac ROS Nodes

  • visual_slam: isaac_ros_visual_slam (NOT ORB-SLAM2, NOT RTAB-Map)
  • stereo_matching: isaac_ros_ess (Enhanced Semi-Global Matching, GPU-based)
  • object_detection: isaac_ros_dnn_inference (TensorRT optimized)

Performance Expectations (Jetson Orin Nano)

  • VSLAM: 30-60 FPS (vs 5-10 FPS on CPU-based SLAM)
  • Object Detection: 30 FPS (YOLOv5 with TensorRT)
  • Depth Estimation: 30 FPS (ESS model)

Isaac Sim Python API Patterns

Create Scene Programmatically

import omni.isaac.core.utils.stage as stage_utils
from omni.isaac.core.objects import DynamicCuboid

# Add ground plane
stage_utils.add_ground_plane()

# Spawn robot
robot_prim_path = "/World/Robot"
stage_utils.add_reference_to_stage(
    usd_path="omniverse://localhost/NVIDIA/Assets/Isaac/2023.1.1/Unitree/unitree_g1.usd",
    prim_path=robot_prim_path
)

# Add camera sensor
from omni.isaac.sensor import Camera
camera = Camera(prim_path=f"{robot_prim_path}/camera", position=[0, 0, 1.5])
camera.initialize()

Domain Randomization for Sim-to-Real

from omni.isaac.core.utils.stage import randomize_shader_properties

# Randomize material properties every episode
randomize_shader_properties(
    prim_path="/World/Environment",
    parameters=["inputs:roughness", "inputs:metallic"],
    ranges=[(0.1, 0.9), (0.0, 1.0)]
)

RTX VRAM Allocation (Isaac Sim 4.x)

  • Base Sim: 4-6 GB
  • Robot Asset (Humanoid): 1-2 GB
  • Ray Tracing: 3-4 GB
  • Physics: 2-3 GB
  • Total: 10-15 GB → 12GB minimum, 24GB recommended