tvm
tvm copied to clipboard
[Bug][TVMC] Map is not supported by RPC
Hi, I'm attempting to run a model via RPC and TVMC but I'm running into an issue:
ValueError: Cannot pass argument 1, type Map is not supported by RPC
The TVMC command I'm using is similar to:
tvmc run --repeat=1 --print-top=5 --rpc-tracker=mytracker.com mymodel.tar
I seem to be able to get the model to run again when I revert #9889 which introduced get_input_info in the graph executor. Is this a case of RPC missing an implementation for Map, or should we do something in TVMC to handle get_input_info differently?
Full backtrace:
tvm.error.RPCError: Traceback (most recent call last):
8: TVMFuncCall
at ../src/runtime/c_runtime_api.cc:477
7: tvm::runtime::PackedFuncObj::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
at ../include/tvm/runtime/packed_func.h:1217
6: tvm::runtime::RPCWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
at ../src/runtime/rpc/rpc_module.cc:127
5: tvm::runtime::RPCClientSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)
at ../src/runtime/rpc/rpc_endpoint.cc:1009
4: tvm::runtime::RPCEndpoint::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)>)
at ../src/runtime/rpc/rpc_endpoint.cc:800
3: tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::function<void (tvm::runtime::TVMArgs)>)
at ../src/runtime/rpc/rpc_endpoint.cc:644
2: tvm::runtime::RPCEndpoint::EventHandler::HandleNextEvent(bool, bool, std::function<void (tvm::runtime::TVMArgs)>)
at ../src/runtime/rpc/rpc_endpoint.cc:135
1: tvm::runtime::RPCEndpoint::EventHandler::HandleProcessPacket(std::function<void (tvm::runtime::TVMArgs)>)
at ../src/runtime/rpc/rpc_endpoint.cc:315
0: tvm::runtime::RPCEndpoint::EventHandler::HandleReturn(tvm::runtime::RPCCode, std::function<void (tvm::runtime::TVMArgs)>)
at ../src/runtime/rpc/rpc_endpoint.cc:376
[bt] (8) libtvm_runtime.so(tvm::runtime::RPCEndpoint::EventHandler::HandleNextEvent(bool, bool, std::function<void (tvm::runtime::TVMArgs)>)+0x464) [0xffffacf33c54]
[bt] (7) libtvm_runtime.so(tvm::runtime::RPCSession::AsyncCallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::RPCCode, tvm::runtime::TVMArgs)>)+0x48) [0xffffacf45dd0]
[bt] (6) libtvm_runtime.so(tvm::runtime::LocalSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)+0x84) [0xffffacf38fac]
[bt] (5) libtvm_runtime.so(tvm::runtime::LocalSession::EncodeReturn(tvm::runtime::TVMRetValue, std::function<void (tvm::runtime::TVMArgs)> const&)+0x74) [0xffffacf38e1c]
[bt] (4) libtvm_runtime.so(+0x125ac4) [0xffffacf45ac4]
[bt] (3) libtvm_runtime.so(std::_Function_handler<void (tvm::runtime::RPCCode, tvm::runtime::TVMArgs), tvm::runtime::RPCEndpoint::EventHandler::HandleNormalCallFunc()::{lambda(tvm::runtime::RPCCode, tvm::runtime::TVMArgs)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::RPCCode&&, tvm::runtime::TVMArgs&&)+0x4c) [0xffffacf31aa4]
[bt] (2) libtvm_runtime.so(tvm::runtime::RPCEndpoint::EventHandler::ValidateArguments(TVMValue const*, int const*, int)+0x1ec) [0xffffacf311f4]
[bt] (1) tvm_rpc(tvm::runtime::detail::LogFatal::Entry::Finalize()+0x6c) [0x40fef4]
[bt] (0) libtvm_runtime.so(tvm::runtime::Backtrace[abi:cxx11]()+0x1c) [0xffffaceb6d84]
File "../src/runtime/rpc/rpc_endpoint.cc", line 376
RPCError: Error caught from RPC call:
ValueError: Cannot pass argument 1, type Map is not supported by RPC
cc @Mousius @gromero @leandron @leeexyz @masahi
Submitted #11757 to work around the issue for now.
This is the intended behavior of RPC for now to restrict the object that we can support (so we can support minimum cases like uTVM). Would be a good starting pt to document the related behavior