AMP for Orbit?

[LuukvBree]

Last month, I started training the locomotion skill of the Mini Cheetah with RL. I have managed to get environments like IsaacGymEnvs and OmniIsaacGymEnvs working for the already implemented robots, but not yet with the Mini Cheetah. When I discovered Orbit, I began implementing the Mini Cheetah within it, and successfully, I can now train the locomotion skill using the rsl_rl train.py script.

Now, in IsaacGymEnvs, there's an example of training with Adversarial Motion Priors. Considering I believe that the preferred framework for future development involving robot learning for Isaac Sim is in Orbit, I was wondering if there are plans to implement an AMP example into the existing frameworks of Orbit, especially since the link to the roadmap is broken.

Thanks!

[Mayankm96]

Hi @LuukvBree ,

Glad to hear you got Mini-Cheetah working! We would be happy to see some videos and also include it in our suite of environments (if that is of interest/possible).

Unfortunately, AMP is not on our immediate roadmap right now. We want to move towards adding support for more manipulation-related things. I haven't used AMP, so I'm not sure what exact changes are needed. It would be great to understand this more so we can try to squeeze them into our development plans accordingly.

We are more than happy if you (or someone else) would like to initiate integrating AMP and contribute to the framework :)

[Robokan]

@Mayankm96

I just wanted to share a couple of new videos of my implementation of AMP running under IsaacLab. Happy to contribute it once I get it a little farther along. It comes with 30 minutes of retargeted motion capture for the go1 and soon go2.

Running in Isaac lab
https://youtu.be/bNTk4gmtJxo?si=gyUypBmCTjTFVv8Q

Deployed to the physical robot

https://youtu.be/7dkQAmGI-rw?si=3Haq6DsN4cO1Vnld

I'm currently capturing real-time data from the go1 to train the actuator network which should stabilize running. I will post more videos once I get that working.

@LuukvBr33 also has an implementation so it looks like there are lots of options here to get a good codebase for AMP/ASE.

[LuukvBr33]

Hi @Robokan,

It's great to hear that you're interested in connecting with me. Sharing my personal email here isn't ideal. I tried searching for you on LinkedIn but couldn't locate your profile. Could you connect with me there instead? Here's my LinkedIn profile: https://www.linkedin.com/in/luukvanbree/.

[Mayankm96]

@Robokan This looks really cool. We would definitely like to integrate it into Isaac Lab. @Dhoeller19 for visibility.

Regarding actuator net, is there a difference between Go1 and Go2 actuators? We had added the actuator model from this repository: https://gmargo11.github.io/walk-these-ways/

I thought what they have should have worked for sim-to-real with Isaac Lab too. If it isn't, then we should investigate a bit. What is your opinion?

[Robokan]

@Mayankm96 I am also using code from the walk-these-ways repository. Walk these ways used a low stiffness of only 20 which didn't work that great so I increased it to 30 and deployed mine using just the implicit actuator. From there I trained a new MLP from it running directly on the Go1. Walk these ways also used a lag buffer to handle the delay when talking with the actuators so I made a DelayedActuatorNetMLP class to handle that. Maybe supplying a few MLP for the actuators at different stiffness might be a good idea.

The Go1 and Go2 both have an additional gear reduction at the knees. The Go1 hip and thigh motors have a 27.7 torque but the knees are 35.5. The gear reduction also reduces the max speed of the knees.
The Go2 has a torque of 45 at the hips and thighs which is increased at the knees with the gear reduction. They don't specify the new torque but I am calculating it as a similar proportion to the Go1.

So the Go1 actuator config should look like this:

GO1_ACTUATOR_CFG = ActuatorNetMLPCfg(
joint_names_expr=["._hip_joint", "._thigh_joint", "._calf_joint"],
network_file=f"{ISAACLAB_NUCLEUS_DIR}/ActuatorNets/Unitree/unitree_go1.pt",
pos_scale=-1.0,
vel_scale=1.0,
torque_scale=1.0,
input_order="pos_vel",
input_idx=[0, 1, 2],
effort_limit={"._hip_joint": 23.7, "._thigh_joint": 23.7, "._calf_joint": 35.5}, # taken from spec sheet
velocity_limit=30.0, # taken from spec sheet
saturation_effort={"._hip_joint": 23.7, "._thigh_joint": 23.7, ".*_calf_joint": 35.5}, # same as effort limit
)

I put a bug report in as saturation_efforts does not currently support dictionaries.

[Marsak24]

@Robokan I have a question please, this MLP is in Isaac lab, I have trained my Go1 using it, then when I deployed the policy in Isaac sim, MLP actuator net is not defined. Is there a way to use the same model?
whenever I try to import packages or libraries from Isaac lab to Isaac sim, I got errors!
Thank you

[fan-ziqi]

I've developed the AMP code based on IsaacLab, referring to the AMP_for_hardware , but there were some problems and the robot could not walk normally. Here is my code:

https://github.com/fan-ziqi/robot_lab

If necessary, I can help merge this code into IsaacLab after it works properly.

[fan-ziqi]

The entire training process has been completed, but the trained model keeps twitching. I hope you can run my code to help me find the problem.

[fan-ziqi]

The training code for AMP has been perfectly implemented

https://github.com/fan-ziqi/robot_lab

[GiulioRomualdi]

Hi! We implemented a self-contained Python package for AMP based on RSL-RL. We are currently using it with isaaclab: https://github.com/ami-iit/amp-rsl-rl