About different types control of multi-agent

[yaofeng11]

I wonder that if ai2thor support different types of control. I mean that one for human-control and another for randomly walk.Actually I run this two agent, the human-control agent just act the first action and then it appear no response.I don't know why. Can someone help me?

[mattdeitke]

Hi @yaofeng11, sorry for the delayed response!

We do support different agent types (such as the default iTHOR agent, ManipulaTHOR agent with an arm, and the LoCoBot for RoboTHOR), but currently the different agent types cannot be intermixed in a multi-agent setting.

The documentation for each agent type is here:

• agentMode="default": iTHOR Documentation
• agentMode="arm": ManipulaTHOR Documentation
• agentMode="locobot": RoboTHOR Documentation

[yaofeng11]

Thanks @mattdeitke . And as for the scene shown in Unity, Luca has showed an idea. cv2.imshow( "frames", cv2.cvtColor( np.concatenate([e0.frame, e1.frame], 1), cv2.COLOR_BGR2RGB ) ) I can't understand that for two random locobots, e0.frame and e1.frame work good, however when I use one locobot, and I mean I use np.concatenate([e1.frame, e1.third_party_camera_frames[0]],1) It didn't work.

[yaofeng11]

Dear Matt Deitke Because of the Spring Festival and covid 19 in Beijing, I have a rest at home. Thanks for your help with using ai2thor. Besides, I wonder that if I create two controllers. I mean that controller1 = Controller(agentCount=1) controller2 = Controller(agentCount=1) am I right? or it only supports one controller, and use controller(agentId) to distinguish different agents If I am right, for two controllers, controller1 is controlled by human. and controller2 is controlled by the correlation algorithm，it can be easy to achieve. So I don't know why this different control ways are not supported in multiagent?  

------------------ 原始邮件 ------------------ 发件人: "allenai/ai2thor" @.>; 发送时间: 2022年1月25日(星期二) 下午2:54 @.>; @.@.>; 主题: Re: [allenai/ai2thor] About different types control of multi-agent (Issue #972)

Hi @yaofeng11, sorry for the delayed response!

We do support different agent types (such as the default iTHOR agent, ManipulaTHOR agent with an arm, and the LoCoBot for RoboTHOR), but currently the different agent types cannot be intermixed in a multi-agent setting.

The documentation for each agent type is here:

agentMode="default": iTHOR Documentation

agentMode="arm": ManipulaTHOR Documentation

agentMode="locobot": RoboTHOR Documentation

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>

[mattdeitke]

Hi @yaofeng11,

Glad to see that you're using AI2-THOR during your time of rest! :)

But, if you create 2 controllers like this, it will instantiate AI2-THOR twice, resulting in two completely separate instances of it, where the controllers are independent of each other and each controller only has 1 agent.

[yaofeng11]

Hi @mattdeitke So when I instantiate AI2-THOR once, the only thing I can distinguish different robots is to use agentId. And in fact, when we talk about multi-agents, the control of agents sometimes will be different, which I mean in one controller, for two agents, agentId = 1 is independent of agentId = 2, thus we may use different algorithms to control agent(agentId = 1) and another agent(agentId = 2). I don't know that in multi-agents systems, what is the controllers' limits? As you say, all I used just LoCobot, not mixed default agent or manipula agent, just the ways to control LoCobot are different. Sorry, I just wonder if there exist any way to solve this problem.

[mattdeitke]

Hi @yaofeng11,

the only thing I can distinguish different robots is to use agentId.

Yes, this is correct.

And in fact, when we talk about multi-agents, the control of agents sometimes will be different, which I mean in one controller, for two agents, agentId = 1 is independent of agentId = 2, thus we may use different algorithms to control agent(agentId = 1) and another agent(agentId = 2).

This is also correct. agentId=1 and agentId=2 can be controlled independently.

I don't know that in multi-agents systems, what is the controllers' limits?

I believe everything you should be able to do with a single agent, you should also be able to do in the multi-agent setup. The main limitation is just that the agent types cannot be intermixed, so one could not control the LoCoBot at the same time that the Drone is being controlled.

[yaofeng11]

Hi @mattdeitke By the way, when I set controllers' agent mode is locobot. For example, I set robot A and robot B. When I try to interactive control robot A, will the platform modify the robot model by default? I mean lobobot A will automatically changed to default bot?