ProcTHOR support and speed

[aishwaryap]

Hi all, I was excited to see the recent announcement of ProcTHOR and have been exploring it this week. I was wondering if there is a planned timeline for it to get into mainline. I have also noticed with some of my trials that a lot of steps (eg: create controller, initialize scene, take an action) seem much slower in the new ProcTHOR scenes compared to older AI2-THOR scenes. I have not properly benchmarked this yet but I was wondering if this issue has been noticed, whether it is expected and whether there is a plan to address it.

[mattdeitke]

Hi @aishwaryap,

Yep, I'm working on documenting everything and making it more useable now! Should have a bunch more documentation out within the next week or so :)

There are a bunch of speed hacks that we used to make things faster. In particular, I'd recommend using SetObjectFilter to not render all the objects at each time step:

# specifies which object ids should appear in the metadata
controller.step(action="SetObjectFilter", objectIds=["Apple1", "Apple2"])

# resets the object filter so all object ids appear in the metadata
controller.step(action="ResetObjectFilter")

In general, we found that the smallish houses (less than 3 rooms) are about as fast as using iTHOR or RoboTHOR, but larger houses (e.g., multi-family sized houses) to be slower, likely due to the many more objects and lights that are in those scenes.

Additionally, there is currently a beta function to speed up resetting the scenes, that makes it 25x faster than the current resets, and those in iTHOR and RoboTHOR, but it occasionally ended up hanging the controller. I'm still debugging this, but hope to have a solution for this out soon!

For creating the controller, it should only be slower once it first downloads the AI2-THOR unity build binary. After that, it can use the local download and should be as fast as before. ProcTHOR includes many new objects and assets that weren't in previous versions of AI2-THOR, so the build size is a bit bigger.

Hope that helps :) I'll update this thread when more documentation is out!

[aishwaryap]

Thanks this helps!

[pushkalkatara]

Hi @mattdeitke,

I have been using ai2thor==0+391b3fae4d4cc026f1522e5acf60953560235971 and also noticed that the simulator is comparatively slow. Is there a more recent binary with the features above to speed up?

Thanks for the amazing work on ProcTHOR!

[LWJ312]

Hi @aishwaryap,

Yep, I'm working on documenting everything and making it more useable now! Should have a bunch more documentation out within the next week or so :)

There are a bunch of speed hacks that we used to make things faster. In particular, I'd recommend using SetObjectFilter to not render all the objects at each time step:

# specifies which object ids should appear in the metadata
controller.step(action="SetObjectFilter", objectIds=["Apple1", "Apple2"])

# resets the object filter so all object ids appear in the metadata
controller.step(action="ResetObjectFilter")

In general, we found that the smallish houses (less than 3 rooms) are about as fast as using iTHOR or RoboTHOR, but larger houses (e.g., multi-family sized houses) to be slower, likely due to the many more objects and lights that are in those scenes.

Additionally, there is currently a beta function to speed up resetting the scenes, that makes it 25x faster than the current resets, and those in iTHOR and RoboTHOR, but it occasionally ended up hanging the controller. I'm still debugging this, but hope to have a solution for this out soon!

For creating the controller, it should only be slower once it first downloads the AI2-THOR unity build binary. After that, it can use the local download and should be as fast as before. ProcTHOR includes many new objects and assets that weren't in previous versions of AI2-THOR, so the build size is a bit bigger.

Hope that helps :) I'll update this thread when more documentation is out!

Hi @mattdeitke! I also found that the ProcTHOR (with ai2thor=5.0.0) runs too slow, especially in the larger rooms. I'm wondering about the progress of your mentioned 'speeding up beta function'? Could you please help me? Thanks a lot!!!