[Question] Question Regarding Sim2Real Transfer for IsaacLab UR3 Reach Task

[Holiclife-KTH]

I modified the IsaacLab reach task by replacing the UR10 with a UR3 and successfully trained a policy using RSL-RL.

After training, I attempted to deploy the resulting policy.onnx file on the real robot. However, the output of the policy is in the form of JointPositionAction, and when I apply it directly to the robot via the driver, the movement is either too jerky or not smooth at all.

Has anyone here successfully performed Sim2Real transfer in a similar setup? I would really appreciate any tips or insights regarding smoothing the control or improving the trajectory execution on the real hardware.

Thank you!

[RandomOakForest]

Thanks for posting this question. This is an excellent topic for our Discussions section. I will move this post to a discussion for the team to follow up. Thank you for your interest in Isaac Lab.

[louislelay]

Hey @Holiclife-KTH, I recently created a repository that might be helpful for you: https://github.com/louislelay/isaaclab_ur_reach_sim2real.

In it, I deployed a policy using ROS 2. Unfortunately, I wasn't able to test it on a real UR10, but it worked well in URSim. Hopefully, it can give you some useful ideas!

I'm also planning to release a similar repo for the Kinova Gen3 arm in the next few days, I'll share the link here once it's ready. For that one, I did test it on a real robot.

Just a heads-up: for both projects, I focused only on reach tasks, but I think they could still be good references for your setup.

[Holiclife-KTH]

Hi @louislelay,

I came across your repository: https://github.com/louislelay/isaaclab_ur_reach_sim2real.
Thank you for sharing the source code and for your helpful responses!!!

I understand that the command from the policy is sent to the robot using a JointTrajectory message.
However, I noticed there is always a gap between the target pose and the actual reached pose.
Additionally, the robot moves with some oscillation—it doesn’t move smoothly, unlike with velocity control or motion in simulation.

I also checked your other repository involving the reach task with the Gen3 robot.
I watched the video of the Gen3's motion, and I was impressed by how smoothly it moved.

Do you have any tips or methods for achieving smoother motion when using JointTrajectory messages?

[RandomOakForest]

Following up, the team has the following suggestions. First, assuming that you are:

• In sim, training a policy that outputs joint position targets, and that those targets are fulfilled either via joint drives or a custom impedance controller (that converts joint position targets to joint torques)
• In real, deploying the sim-trained policy, and fulfilling the targets via UR’s default controller

Then, it is likely that the drives/controller that you are using in sim is far less stiff than UR’s default controller in the real world. A few possible things to explore would be:

1. Making sure that the policy outputs in sim are as smooth as possible. One common way to do this is, in your reward function, minimize the difference between subsequent actions (often referred to as an action gradient penalty)
2. Try to model the UR controller more closely in sim by increasing the stiffness of the joint drives or custom impedance controller
3. In the real world, apply a low-pass filter to the policy outputs (e.g., via an exponential moving average)

Second, a good starting point for addressing the issues you mention would be to perform some quick visualization in simulation to assess whether the behavior also occurs there-- for example, by visualizing the policy outputs and the robot joint trajectory during a trained policy deployment in simulation. If the issue exists in simulation, it indicates that the RL task (reward function) does not adequately capture the jerkiness requirements. If there is no issue in simulation, then it is a sim-to-real gap, and you can follow (2) and (3) above.