IsaacLab icon indicating copy to clipboard operation
IsaacLab copied to clipboard

Adds enable_cpu_readback option for separation of sim and task device

Open kellyguo11 opened this issue 2 months ago • 0 comments

Description

Up until now, we have been working under the assumption of keeping consistency between simulation device and the task device. Data returned from simulation will always match the device simulation is running on, which in turn, should also be the environment device and training device.

Recent surface gripper interface allows for running simulation on the GPU, but does not support the full end-to-end GPU pipeline and instead requires data to be copied back onto the CPU for additional processing. To allow for this type of workflow, we have to relax our assumption and allow for having different devices for which simulation runs computation on and which data from simulation is returned on.

This means we can now mix and match between simulation, environment, and training device:

  • simulation device: GPU, enable_cpu_readback: True (simulation data returned on CPU), environment device: CPU, training device: optional GPU/CPU (default follows training config)
  • simulation device: GPU, enable_cpu_readback: False (previous behavior and still the default behavior), environment device: GPU, training device: optional GPU/CPU (default follows training config
  • simulation device: CPU, enable_cpu_readback: ignored, environment device: CPU, training device: optional GPU/CPU (default follows training config)

I'm not 100% convinced we should do this though, it requires a lot of additional code paths and checks for different devices that feels very error prone, and we likely won't need this flexibility in most cases...

Type of change

  • New feature (non-breaking change which adds functionality)
  • Documentation update

Checklist

  • [x] I have read and understood the contribution guidelines
  • [x] I have run the pre-commit checks with ./isaaclab.sh --format
  • [x] I have made corresponding changes to the documentation
  • [x] My changes generate no new warnings
  • [x] I have added tests that prove my fix is effective or that my feature works
  • [x] I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • [ ] I have added my name to the CONTRIBUTORS.md or my name already exists there

kellyguo11 avatar Nov 07 '25 22:11 kellyguo11