[Question] Is there a way to record the actor coordinates at each step and use them in the terrain curriculum?

[H-Hisamichi]

Hello everyone,

In the initial implementation of the terrain curriculum in Isaac Lab, the distance moved by the actor is computed using the norm between the starting point and the final position at the end of the episode.
However, this approach can be inaccurate when the actor moves in an irregular path.
To address this, I would like to record the actor's position at every learning steps and calculate the total traveled distance by summing the norms of the position differences between each step within an episode.
However, it seems that RecorderManager only allows access to position data after training is complete.
Is there a way to record actor positions at each step and access that data at the end of the episode?

Thanks in advance!

[RandomOakForest]

Thank you for starting this discussion. To accurately compute the total traveled distance for an actor in such way, probably the best is to implement a custom manager that caches positions during the episode and calculates the distance at episode end. Here's a step-by-step solution you may try:

1. Create a Custom Manager Class

Implement a manager to record positions at each step and compute distances at episode end:

import torch
from omni.isaac.lab.managers import ManagerTermBase

class TraveledDistanceRecorder(ManagerTermBase):
    def __init__(self, config, env):
        super().__init__(config, env)
        self._env = env
        self._num_envs = env.num_envs
        self.device = env.device
        
        # Buffers to store positions and distances
        self._position_cache = [[] for _ in range(self._num_envs)]
        self._episode_travel_dist = torch.zeros(self._num_envs, device=self.device)

    def reset(self, env_ids):
        # Reset cache and distance for specified environments
        for idx in env_ids:
            self._position_cache[idx] = []
            self._episode_travel_dist[idx] = 0.0

    def pre_physics_step(self):
        # Record positions at the start of each physics step
        current_pos, _ = self._env.actors.get_world_poses()
        for i in range(self._num_envs):
            self._position_cache[i].append(current_pos[i].clone())

    def post_episode(self):
        # Calculate total distance at episode end
        for env_idx in range(self._num_envs):
            if len(self._position_cache[env_idx]) < 2:
                continue
                
            positions = torch.stack(self._position_cache[env_idx])
            deltas = positions[1:] - positions[:-1]
            distances = torch.norm(deltas, dim=1)
            self._episode_travel_dist[env_idx] = torch.sum(distances)

    def get_episode_travel_dist(self, env_ids=None):
        # Retrieve distances for curriculum manager
        if env_ids is None:
            return self._episode_travel_dist.clone()
        return self._episode_travel_dist[env_ids].clone()

2. Integrate with the Environment

Add the manager to your environment setup:

class MyEnvironment(RLTaskEnv):
    def _setup_managers(self):
        super()._setup_managers()
        self.add_manager(
            "traveled_distance_recorder",
            TraveledDistanceRecorder, 
            cfg={}, 
            env=self
        )

3. Connect to Curriculum Manager

Access the distance data from the curriculum manager:

class TerrainCurriculumManager:
    def update_curriculum(self, env_ids):
        dist_recorder = self._env.managers["traveled_distance_recorder"]
        travel_distances = dist_recorder.get_episode_travel_dist(env_ids)
        
        # Use distances for curriculum updates
        self._adjust_difficulty(travel_distances)

Key Implementation Notes:

1. Position Caching:
   • _position_cache stores positions at each physics step.
   • pre_physics_step() captures positions before physics simulation.
2. Distance Calculation:
   • post_episode() computes the sum of Euclidean distances between consecutive positions.
   • Uses vectorized PyTorch operations for efficiency.
3. Data Flow:
   • Curriculum manager accesses data via get_episode_travel_dist().
   • Reset logic ensures clean state per episode.
4. Performance:
   • Minimal overhead: Only stores positions.
   • GPU-accelerated computations using PyTorch.

Usage Workflow:

1. During Episode:
   • Positions recorded at every physics step.
   • Data stored per-environment.
2. At Episode End:
   • Total distance computed for each environment.
   • Results cached in _episode_travel_dist.
3. Curriculum Update:
   • Terrain manager pulls distances.
   • Adjusts terrain difficulty based on traveled distance.

This solution bypasses RecorderManager limitations by:

• Capturing per-step data in real-time
• Computing distances on-device (GPU)
• Providing immediate access to curriculum parameters
• Maintaining compatibility with Isaac Lab's manager architecture

[H-Hisamichi]

Hello @RandomOakForest,

Thank you for your kind advice!
I will try implementing it in my learning environment right away.