[Question] It takes too much time when store camera data into EpisodeData

[wghou]

Question

I'm generating manipulation data with mimic_gen, i.e. https://isaac-sim.github.io/IsaacLab/main/source/overview/teleop_imitation.html# . And I try to store the camera data (images) into the hdf5 file. But it takes a huge amount of time to store camera observation data into EpisodeData, almost over 100ms in each step.

I review the source code of IsaacLab, as following (https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab/isaaclab/utils/datasets/episode_data.py, line 103-123)

sub_keys = key.split("/")
current_dataset_pointer = self._data
for sub_key_index in range(len(sub_keys)):
    if sub_key_index == len(sub_keys) - 1:
        # Add value to the final dict layer
        if sub_keys[sub_key_index] not in current_dataset_pointer:
            current_dataset_pointer[sub_keys[sub_key_index]] = value.unsqueeze(0).clone()
        else:
            current_dataset_pointer[sub_keys[sub_key_index]] = torch.cat(
                (current_dataset_pointer[sub_keys[sub_key_index]], value.unsqueeze(0))
            )
        break
    # key index
    if sub_keys[sub_key_index] not in current_dataset_pointer:
        current_dataset_pointer[sub_keys[sub_key_index]] = dict()
    current_dataset_pointer = current_dataset_pointer[sub_keys[sub_key_index]]

In each simulation step, the image data would be stored into a single tensor variable by calling tensor.cat(...). It would take too much time when the image size is large.

Does anyone has suggestions, good idea to solve this problem?

Thanks

[RandomOakForest]

Thank you for posting this. I'll move this post to our Discussions for follow up. In the meantime, here are a few things to try:

1. Preallocate Storage Space

Instead of concatenating tensors at each step, preallocate a fixed-size tensor for the entire episode:

# Preallocate at episode start
episode_length = 1000  # Estimated max steps
image_tensor = torch.empty((episode_length, height, width, channels), dtype=torch.uint8)

# During simulation
image_tensor[step_index] = current_frame

2. Use HDF5 Chunking and Compression

Modify HDF5 storage parameters for better performance:

with h5py.File("dataset.hdf5", "w") as f:
    dset = f.create_dataset("images", 
                            shape=(episode_length, height, width, channels),
                            chunks=(1, height, width, channels),  # Per-image chunk
                            compression="gzip", 
                            dtype="uint8")

Key optimizations:

• chunks=(1, ...) matches write pattern
• compression="gzip" reduces storage (3-5x smaller)
• dtype="uint8" for RGB images

3. Reduce Image Resolution

Where acceptable, lower camera resolution. Storage scales quadratically with resolution - 320x240 uses ~4x less space than 640x480.