[Bug] [Relax VM] All of Arguments of SetInput will be Convert to The 1st Device

[Johnson9009]

https://github.com/apache/tvm/blob/299ef81f0f7e4bae93826141815dfc97c0ea0a42/src/runtime/relax_vm/vm.cc#L510-L531 https://github.com/apache/tvm/blob/299ef81f0f7e4bae93826141815dfc97c0ea0a42/src/runtime/relax_vm/vm.cc#L524

Why all of the arguments given to "SetInput" will be convert to the 1st device? Is it a typo?

I think it should be something below.

func_args[i] = ConvertArgToDevice(args[i], devices[i], allocators[i]);

[tqchen]

it should be the first device by default as intended, as the number of arguments and number of devices can differ

[Johnson9009]

@tqchen Thank for the explanation, after reading the relevant code deeply, I found the logic about devices of inputs in Relax is different with that of Relay.

In Relay, executable store the virtual device of each input, when VM initialization the real devices will be sorted and extended according to the virtual devices. So each input can get it's own device correctly and there won't any unnecessary data movement for each input.

I think Relax's currently logic will crush or do more unnecessary data movement, when device's of the multiple inputs need to be different？

By the way, currently I haven't any real case that must need devices of inputs are different, found this issue just because we are migrating our Relay to Relax, and reading these code.

[tqchen]

This is a good point, in theory we should be able to remove tihs logic even, for input, and require the input to be already on the right device, perhaps we can send a patch to fix this behavior