Using PoseExtractor in habitat-lab

[smorad]

❓ Questions and Help

I'm currently trying to pretrain a CNN using a PoseExtractor, then train an RL agent using said CNN in habitat-lab. Unfortunately, it seems like PoseExtractor doesn't have habitat-lab integration. Ideally, I'd be able to pass my nice habitat-lab configs specifying train/test split, device, sensor setup, agent height, etc to a PoseExtractor instance.

I was wondering if there was a way to do this.

[aclegg3]

Hey, @smorad. Yeah, this sounds like an additional feature. You'd probably want to fork the code and make a new tool for this. Instead of extracting poses from a Simulator instance in habitat-sim, I'd instantiate the Task or Env and then make calls to the underlying Simulator instance. Alternatively you could modify the PoseExtractor to take an existing instance of the Simulator object and then use the Task/Env to instantiate an episode from your dataset and pass it in.