[Question] Deploying trained humanoid policy in Isaac Sim

[MrOCW]

Question

I have trained a G1 locomotion policy (1), with 13 action output by removing the ActuatorCfg in G1_CFG instead of the default 37 and modified the joints in ActionsCfg of velocity_env_cfg.py. This is to prevent the policy from controlling the arms as I need to control the arms for other purposes.
The training in Isaac Lab went well. Note that the observation size is still 37.

When attempting to deploy policy 1, i used the H1FlatTerrainPolicy in the Humanoid Examples as a template. Damping and Stiffness of all the joints were adjusted according to the env.yaml file. I printed self.robot.dof_names

1. in forward method, I modified self.action to be of length 37 before passing it into ArticulationAction
   Sample self.robot.dof_names & self.action:

['torso_joint', 'left_shoulder_pitch_joint', 'right_shoulder_pitch_joint', 'left_hip_pitch_joint', 'right_hip_pitch_joint', 'left_shoulder_roll_joint', 'right_shoulder_roll_joint', 'left_hip_roll_joint', 'right_hip_roll_joint', 'left_shoulder_yaw_joint', 'right_shoulder_yaw_joint', 'left_hip_yaw_joint', 'right_hip_yaw_joint', 'left_elbow_pitch_joint', 'right_elbow_pitch_joint', 'left_knee_joint', 'right_knee_joint', 'left_elbow_roll_joint', 'right_elbow_roll_joint', 'left_ankle_pitch_joint', 'right_ankle_pitch_joint', 'left_ankle_roll_joint', 'right_ankle_roll_joint', 'left_five_joint', 'left_three_joint', 'left_zero_joint', 'right_five_joint', 'right_three_joint', 'right_zero_joint', 'left_six_joint', 'left_four_joint', 'left_one_joint', 'right_six_joint', 'right_four_joint', 'right_one_joint', 'left_two_joint', 'right_two_joint']
[ 0.01355542  0.          0.          0.23275226 -0.2605567   0.
  0.          0.17375039  0.07851431  0.          0.         -0.0086574
 -0.18230547  0.          0.          0.48574701  0.82196361  0.
  0.          0.94566303  3.32378697 -0.11683509  1.14421988  0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.        ]

I get this error message:

2025-05-07 09:37:11 [8,712,897ms] [Error] [omni.kit.app._impl] [py stderr]: Traceback (most recent call last):
2025-05-07 09:37:11 [8,712,897ms] [Error] [omni.kit.app._impl] [py stderr]:   File "c:/isaacsim/exts/isaacsim.examples.interactive/isaacsim/examples/interactive/humanoid/humanoid_example.py", line 255, in on_physics_step
2025-05-07 09:37:11 [8,712,898ms] [Error] [omni.kit.app._impl] [py stderr]:     self.h1.initialize()
2025-05-07 09:37:11 [8,712,898ms] [Error] [omni.kit.app._impl] [py stderr]:   File "c:/isaacsim/exts/isaacsim.robot.policy.examples/isaacsim/robot/policy/examples/robots/g1.py", line 387, in initialize
2025-05-07 09:37:11 [8,712,898ms] [Error] [omni.kit.app._impl] [py stderr]:     return super().initialize(set_articulation_props=False)
2025-05-07 09:37:11 [8,712,898ms] [Error] [omni.kit.app._impl] [py stderr]:   File "c:/isaacsim/exts/isaacsim.robot.policy.examples/isaacsim/robot/policy/examples/controllers/policy_controller.py", line 109, in initialize
2025-05-07 09:37:11 [8,712,899ms] [Error] [omni.kit.app._impl] [py stderr]:     self.robot._articulation_view.set_gains(stiffness, damping)
2025-05-07 09:37:11 [8,712,899ms] [Error] [omni.kit.app._impl] [py stderr]:   File "c:/isaacsim/exts/isaacsim.core.prims/isaacsim/core/prims/impl/articulation.py", line 3035, in set_gains
2025-05-07 09:37:11 [8,712,900ms] [Error] [omni.kit.app._impl] [py stderr]:     stiffnesses = self._backend_utils.assign(
2025-05-07 09:37:11 [8,712,900ms] [Error] [omni.kit.app._impl] [py stderr]:   File "c:/isaacsim/exts/isaacsim.core.utils/isaacsim/core/utils/numpy/tensor.py", line 91, in assign
2025-05-07 09:37:11 [8,712,900ms] [Error] [omni.kit.app._impl] [py stderr]:     dst[tuple(indices)] = src
2025-05-07 09:37:11 [8,712,900ms] [Error] [omni.kit.app._impl] [py stderr]: ValueError: shape mismatch: value array of shape (13,) could not be broadcast to indexing result of shape (1,37)

The G1 falls immediately upon playing it is observed to still respond to commands.

2. Went to torso_link and excluded the arms from articulation. modified self.robot.get_joint_positions() and self.robot.get_joint_velocities() such that they match the order of self.robot_dof_names from point 1 and padded with 0s to length 37, since it is now 13 due to excluding joints from articulations
   Sample self.robot.dof_names & self.action:

['left_hip_pitch_joint', 'right_hip_pitch_joint', 'torso_joint', 'left_hip_roll_joint', 'right_hip_roll_joint', 'left_hip_yaw_joint', 'right_hip_yaw_joint', 'left_knee_joint', 'right_knee_joint', 'left_ankle_pitch_joint', 'right_ankle_pitch_joint', 'left_ankle_roll_joint', 'right_ankle_roll_joint']
[-0.11803413 -1.2521374  -0.12940435  0.04658721  0.52817    -0.24584377
 -0.43887097 -0.21444759  0.9849761   2.5686626  -1.6447524  -3.0439992
  0.9085994 ]

I assume this should be the order used in IsaacLab? Otherwise, where is the right place to check?

The error from (1) is gone but the outcome is the same.

I then attempted to train another policy (2) by using the g1_minimal.usd from isaac sim assets, and excluding the arms from articulation.
rsl_rl crashes midway during training regarding a torch.normal error requiring std>=0, and skrl fails to train, with NaN values in the rewards. The training doesnt work as the values can go up to inf

env.yaml:

sim:
  physics_prim_path: /physicsScene
  device: cuda:0
  dt: 0.005
  render_interval: 4
  gravity: !!python/tuple
  - 0.0
  - 0.0
  - -9.81
  enable_scene_query_support: false
  use_fabric: true
  physx:
    solver_type: 1
    min_position_iteration_count: 1
    max_position_iteration_count: 255
    min_velocity_iteration_count: 0
    max_velocity_iteration_count: 255
    enable_ccd: false
    enable_stabilization: true
    enable_enhanced_determinism: false
    bounce_threshold_velocity: 0.5
    friction_offset_threshold: 0.04
    friction_correlation_distance: 0.025
    gpu_max_rigid_contact_count: 8388608
    gpu_max_rigid_patch_count: 327680
    gpu_found_lost_pairs_capacity: 2097152
    gpu_found_lost_aggregate_pairs_capacity: 33554432
    gpu_total_aggregate_pairs_capacity: 2097152
    gpu_collision_stack_size: 67108864
    gpu_heap_capacity: 67108864
    gpu_temp_buffer_capacity: 16777216
    gpu_max_num_partitions: 8
    gpu_max_soft_body_contacts: 1048576
    gpu_max_particle_contacts: 1048576
  physics_material:
    func: isaaclab.sim.spawners.materials.physics_materials:spawn_rigid_body_material
    static_friction: 1.0
    dynamic_friction: 1.0
    restitution: 0.0
    improve_patch_friction: true
    friction_combine_mode: multiply
    restitution_combine_mode: multiply
    compliant_contact_stiffness: 0.0
    compliant_contact_damping: 0.0
  render:
    enable_translucency: null
    enable_reflections: null
    enable_global_illumination: null
    antialiasing_mode: null
    enable_dlssg: null
    enable_dl_denoiser: null
    dlss_mode: null
    enable_direct_lighting: null
    samples_per_pixel: null
    enable_shadows: null
    enable_ambient_occlusion: null
    carb_settings: null
    rendering_mode: null
ui_window_class_type: isaaclab.envs.ui.manager_based_rl_env_window:ManagerBasedRLEnvWindow
seed: 42
decimation: 4
scene:
  num_envs: 2048
  env_spacing: 2.5
  lazy_sensor_update: true
  replicate_physics: true
  filter_collisions: true
  robot:
    class_type: isaaclab.assets.articulation.articulation:Articulation
    prim_path: /World/envs/env_.*/Robot
    spawn:
      func: isaaclab.sim.spawners.from_files.from_files:spawn_from_usd
      visible: true
      semantic_tags: null
      copy_from_source: true
      mass_props: null
      deformable_props: null
      rigid_props:
        rigid_body_enabled: null
        kinematic_enabled: null
        disable_gravity: false
        linear_damping: 0.0
        angular_damping: 0.0
        max_linear_velocity: 1000.0
        max_angular_velocity: 1000.0
        max_depenetration_velocity: 1.0
        max_contact_impulse: null
        enable_gyroscopic_forces: null
        retain_accelerations: false
        solver_position_iteration_count: null
        solver_velocity_iteration_count: null
        sleep_threshold: null
        stabilization_threshold: null
      collision_props: null
      activate_contact_sensors: true
      scale: null
      articulation_props:
        articulation_enabled: null
        enabled_self_collisions: false
        solver_position_iteration_count: 8
        solver_velocity_iteration_count: 4
        sleep_threshold: null
        stabilization_threshold: null
        fix_root_link: null
      fixed_tendons_props: null
      joint_drive_props: null
      visual_material_path: material
      visual_material: null
      usd_path: http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/4.5/Isaac/IsaacLab/Robots/Unitree/G1/g1_minimal.usd
      variants: null
    init_state:
      pos: !!python/tuple
      - 0.0
      - 0.0
      - 0.74
      rot: !!python/tuple
      - 1.0
      - 0.0
      - 0.0
      - 0.0
      lin_vel: !!python/tuple
      - 0.0
      - 0.0
      - 0.0
      ang_vel: !!python/tuple
      - 0.0
      - 0.0
      - 0.0
      joint_pos:
        left_hip_yaw_joint: 0.0
        left_hip_roll_joint: 0
        left_hip_pitch_joint: -0.1
        left_knee_joint: 0.3
        left_ankle_pitch_joint: -0.2
        left_ankle_roll_joint: 0
        right_hip_yaw_joint: 0.0
        right_hip_roll_joint: 0
        right_hip_pitch_joint: -0.1
        right_knee_joint: 0.3
        right_ankle_pitch_joint: -0.2
        right_ankle_roll_joint: 0
        torso_joint: 0.0
      joint_vel:
        .*: 0.0
    collision_group: 0
    debug_vis: false
    soft_joint_pos_limit_factor: 0.9
    actuators:
      legs:
        class_type: isaaclab.actuators.actuator_pd:ImplicitActuator
        joint_names_expr:
        - .*_hip_yaw_joint
        - .*_hip_roll_joint
        - .*_hip_pitch_joint
        - .*_knee_joint
        - torso_joint
        effort_limit: 300
        velocity_limit: 100.0
        effort_limit_sim: null
        velocity_limit_sim: null
        stiffness:
          .*_hip_yaw_joint: 150.0
          .*_hip_roll_joint: 150.0
          .*_hip_pitch_joint: 200.0
          .*_knee_joint: 200.0
          torso_joint: 200.0
        damping:
          .*_hip_yaw_joint: 5.0
          .*_hip_roll_joint: 5.0
          .*_hip_pitch_joint: 5.0
          .*_knee_joint: 5.0
          torso_joint: 5.0
        armature:
          .*_hip_.*: 0.01
          .*_knee_joint: 0.01
          torso_joint: 0.01
        friction: null
      feet:
        class_type: isaaclab.actuators.actuator_pd:ImplicitActuator
        joint_names_expr:
        - .*_ankle_pitch_joint
        - .*_ankle_roll_joint
        effort_limit: 20
        velocity_limit: null
        effort_limit_sim: null
        velocity_limit_sim: null
        stiffness: 20.0
        damping: 2.0
        armature: 0.01
        friction: null

[RandomOakForest]

Thanks for posting this. It just looks like a bad shape in one your input arrays. To be on the same page, could you share a link to the examples you are drawing from? Also, please share your Isaac Lab and Isaac Sim versions. Thanks.

[MrOCW]

Isaac Lab (main: 9be0de5)
Isaac Sim 4.5.0
isaacsim/exts/isaacsim.examples.interactive/isaacsim/examples/interactive/humanoid/humanoid_example.py
please also look for H1FlatTerrainPolicy class in another file
I basically created and replaced H1FlatTerrainPolicy with a new G1FlatTerrainPolicy, which loads an rsl_rl jit policy trained in IsaacLab

Also,
I have managed to get a 13 size observation shape to train now by modifying joint_pos_rel and joint_vel_rel in observations.py from IsaacLab. In Isaac Sim, I then modified g1.usd by excluding the arms from articulations, and setting local rotation 1 of the joints to the values in self.default_pos. However, the deployment in IsaacSim is still not working as the G1 spazzes and falls over upon spawning.

Please refer to this video:
https://drive.google.com/file/d/1zlcT6mxDeIvWyQ45aZVvvfYLrSIPYxmf/view?usp=sharing

This is quite similar to an issue that i posted here:
https://forums.developer.nvidia.com/t/h1-control-policy-not-working-in-custom-environment/331071
In the forum post, I was not modifying directly from
isaacsim/exts/isaacsim.examples.interactive/isaacsim/examples/interactive/humanoid/humanoid_example.py

Hence, my guess is that the physics settings may be an issue here?

[MrOCW]

@RandomOakForest hi! any advise on this problem?

[RandomOakForest]

Thanks for following up. I'll move this post to our Discussions section for follow up. In the meantime, this is a dimensionality mismatch due to a likely incorrect handling of joint indices and action spaces between Isaac Lab training and Isaac Sim deployment. Here's a summary and possible next steps:

Core Issue

The error ValueError: shape mismatch: value array of shape (13,) could not be broadcast to indexing result of shape (1,37) indicates you're trying to apply actions/stiffness/damping for 13 joints to an articulation expecting 37 joints. This stems from:

1. Incomplete Articulation Configuration
   A modified action space to 13 joints in Isaac Lab but didn't properly exclude arm joints from the articulation in Isaac Sim. The robot USD file still contains 37 active joints.
2. Incorrect Gain Application
   When setting stiffness/damping through set_gains(), passing arrays sized for 13 joints without specifying target joint indices¹.

Critical Fixes Needed

1. Articulation Alignment

# In Isaac Sim deployment code:
self.robot._articulation_view.set_gains(
    stiffness_targets, 
    damping_targets,
    joint_indices=active_joint_indices  # Must specify which joints to modify
)

• Use get_joint_index() to verify joint order matches training²³
• Confirm excluded joints are removed from articulation in USD file⁴

2. Action Space Consistency

# When applying actions:
action = ArticulationAction(
    joint_positions=policy_output, 
    joint_indices=active_joint_indices  # Explicitly specify controlled joints
)
self.robot.apply_action(action)

• Action array length must match len(joint_indices)¹⁴
• Never pad actions to original 37-joint size

3. Physics Configuration
Ensure env.yaml matches between training/deployment for:

• Joint drive parameters (stiffness/damping)
• Physics solver iterations
• Simulation substeps

Recommended Debugging Steps

1. Print articulation metadata to verify active joints:

print(self.robot._articulation_view._metadata.joint_names)
print(self.robot._articulation_view._metadata.joint_indices)

2. Validate observation/action space alignment:

assert policy_output.shape[0] == len(active_joint_indices)

3. Use minimal USD (g1_minimal.usd) with arms properly excluded from articulation⁴.

This approach maintains consistency between training and deployment environments while properly handling joint indexing. The key is explicit specification of target joints in all articulation API calls rather than assuming default ordering²³¹.

References

Footnotes

1. https://isaac-sim.github.io/IsaacLab/main/source/tutorials/01_assets/run_articulation.html ↩ ↩² ↩³

2. https://docs.omniverse.nvidia.com/py/isaacsim/source/extensions/omni.isaac.core/docs/index.html ↩ ↩²

3. https://docs.isaacsim.omniverse.nvidia.com/4.5.0/robot_simulation/articulation_controller.html ↩ ↩²

4. https://forums.developer.nvidia.com/t/articulationaction-shape-mismatch/238196 ↩ ↩² ↩³

[MrOCW]

@RandomOakForest Thanks for the detailed response.

1. For active_joint_indices, I am obtaining them via:

self.indices = 
[self.robot.get_dof_index("left_hip_pitch_joint"),
 self.robot.get_dof_index("right_hip_pitch_joint"),
 self.robot.get_dof_index("left_hip_roll_joint"),
 self.robot.get_dof_index("right_hip_roll_joint"),
 self.robot.get_dof_index("left_hip_yaw_joint"),
 self.robot.get_dof_index("right_hip_yaw_joint"),
 self.robot.get_dof_index("left_knee_joint"),
 self.robot.get_dof_index("right_knee_joint"),
 self.robot.get_dof_index("left_ankle_pitch_joint"),
 self.robot.get_dof_index("right_ankle_pitch_joint"),
 self.robot.get_dof_index("left_ankle_roll_joint"),
 self.robot.get_dof_index("right_ankle_roll_joint")]

which gives the following:
[3, 4, 7, 8, 11, 12, 15, 16, 19, 20, 21, 22]

However, with reference to your debugging step to use
self.robot._articulation_view._metadata.joint_indices, the matching indices to the output of that, is:
[6, 8, 11, 12, 15, 16, 19, 20, 23, 24, 27, 28]
It has been observed that the output of the metadata also includes joints that are not actuated.

Using either of the above indices does not work.

for set_gains, it is performed in policy_controller.py in the initialize() function, which essentially sets all the joints's stiffness & damping found in env.yaml, so i assume the joint parameters have been set already?

Confirm excluded joints are removed from articulation in USD file

I have loaded the default g1.usd, and used the following code to extract the relevant joint observations:

        current_joint_pos = self.robot.get_joint_positions() # len 37
        current_joint_vel = self.robot.get_joint_velocities() # len 37
        obs[12:24] = np.array(current_joint_pos)[self.indices] - np.array(self.default_pos)[self.indices]
        obs[24:36] = np.array(current_joint_vel)[self.indices]

Please let me know if this is not the right way to do this. I require the arms to be part of articulations for external control.

For actions, I have implemented it as such:

action = ArticulationAction(joint_positions=np.array(self.default_pos)[self.indices] + (self.action * self._action_scale), joint_indices = self.indices)
self.robot.apply_action(action)

assert policy_output.shape[0] == len(active_joint_indices)

This has no problem too. I used self.action.shape[0] == len(self.indices)

3. the env.yaml file was exported from IsaacLab and no changes has been made to it. The initialization of FlatTerrainPolicy's self.load_policy reads from this env.yaml so I assume that there should be no difference between Sim and Lab's Joint/Physics/GPU properties?

At this current stage, there are no errors on shape mismatches thus far.