Carter_V1 DifferentialController sim2real

[giangdao1402]

I'm currently setting up a reinforcement learning project for carter_v1, and I have implemented the following functions to control the mobile robot:

 
def _pre_physics_step(self, actions: torch.Tensor) -> None: 
   self.actions = actions.clone() 
   self.controls = torch.zeros((self.num_envs, 2), device=self.device) 
      for i in range(self.num_envs): 
          control = self._diff_controller.forward([actions[i][0].item(), actions[i][1].item()]) 
          self.controls[i] = control 
def _apply_action(self) -> None: 
   joint_ids = torch.arange(len(self._nova_dof_idx), dtype=torch.int32, device=self.device) 
   self.robot.set_joint_velocity_target(self.controls, joint_ids=joint_ids) 

I initialize the differential drive controller like this:

 self._diff_controller = DifferentialController( name="carter_v1_control", wheel_radius=0.24, wheel_base=0.54 ) 

I also define the action space to prevent the robot from going backward:

 action_space = gym.spaces.Box(low=0.0, high=1.0, shape=(2,), dtype=np.float32) 

During training, everything works fine — the robot does not go backward as expected. However, during inference, the robot occasionally moves backward, which I want to prevent.

Does anyone have suggestions or insights? I suspect I do not fully understand how the DifferentialController converts the action inputs into wheel velocities, especially how backward motion is still possible even with the action space constrained to non-negative values.

[RandomOakForest]

Thank you for posting this. It seems the issue arises because the DifferentialController's kinematic equations allow one wheel to rotate backward even with non-negative action inputs, depending on the ratio of linear and angular velocities. Also, training may be overfitting.

A couple of possible solutions are as follows:

1. Clamp Wheel Velocities

Modify the controller to enforce non-negative wheel speeds:

def forward(self, command):
    V, ω = command
    ω_R = (2 * V + ω * self.wheel_base) / (2 * self.wheel_radius)
    ω_L = (2 * V - ω * self.wheel_base) / (2 * self.wheel_radius)
    ω_R = max(ω_R, 0.0)  # Prevent negative velocities
    ω_L = max(ω_L, 0.0)
    return [ω_R, ω_L]

Trade-off: Limits turning radius but ensures no backward motion.

2. Dynamic Action Scaling

Constrain $\omega \leq \frac{2V}{b}$ to keep $\omega_L \geq 0$:

# In the RL environment
actions[:, 1] = actions[:, 1] * (2 * actions[:, 0] / self.wheel_base)

This ensures angular velocity scales with linear velocity, preventing $\omega_L < 0$. Also, this would be the recommended approach, as it directly addresses the kinematic constraint while preserving the ability to turn, albeit with a reduced angular velocity range at low linear speeds.