House model export to Unity

[HusterHope]

Hi AI2 Team, I want to put the generated house model into a virtual reality environment to interact with it. Is there a way to export the model to a Unity scene? Thanks a lot.

[Lucaweihs]

Hi @HusterHope, sorry for the delayed reply. @AlvaroHG would you know if this can be done easily?

[dendorferpatrick]

Any updates here?

[Inshsang]

@HusterHope Hello，have you export the json to unity successfully？I am wondering this too

[HusterHope]

@HusterHope Hello，have you export the json to unity successfully？I am wondering this too

@Inshsang Sorry for the late reply. After I raised this issue, I tried to use the Ai2THOR library (also this team's nice work), which had Unity projects that could be opened directly and partly met our needs.

[YxuanAr]

@HusterHope Hello，have you export the json to unity successfully？I am wondering this too

@Inshsang Sorry for the late reply. After I raised this issue, I tried to use the Ai2THOR library (also this team's nice work), which had Unity projects that could be opened directly and partly met our needs.

Sorry, could you help to show how to use ai2thor to visualize the temp.json file? I am really confused about it.

[Kybin-kb]

@HusterHope Hello，have you export the json to unity successfully？I am wondering this too

@Inshsang Sorry for the late reply. After I raised this issue, I tried to use the Ai2THOR library (also this team's nice work), which had Unity projects that could be opened directly and partly met our needs.

I have been trying for a long time without success, which is very important to me. May I ask how to visualize the generated JSON file using ai2thor? If possible, I would greatly appreciate it.

[Inshsang]

@YxuanAr @Kybin-kb Sorry for the late reply.You can refer to the following link to visualize the JSON representation of each room through the RGB image: https://ai2thor.allenai.org/ithor/documentation/environment-state, Additionally, there are many tools described in other documents, such as: https://ai2thor.allenai.org/robothor/documentation and https://ai2thor.allenai.org/manipulathor/documentation You can also refer to the related issue on GitHub for more information. They are generally compatible: https://github.com/allenai/ai2thor/issues/24

For example, you can initialize the JSON representation of a house using the following code:

import ai2thor.controller
from PIL import Image

controller = ai2thor.controller.Controller()
controller.start()

controller.reset('FloorPlan28')
event = controller.step(dict(action='Initialize', renderDepthImage=True, gridSize=0.25))
event = controller.step(dict(action='TeleportFull', renderDepthImage=True, x=-1.25, y=1.00, z=-1.5, rotation=90, horizon=30.0))
img = Image.fromarray(event.cv2img/ 20) # rescale image
print(event.cv2img.shape)
img.show(title='cv2img') 

Here, 'FloorPlan28' can be replaced by a JSON from ProcTHOR.