Select GPU device id other than 0 using C++

[spxcds]

I want to run bert on GPU with C++, and the default device is 0. Is there any way to select other device with C++ API?

[feifeibear]

Currently, no such API provided. We can schedule this feature in the future if you insist. Or you can make your own contributions by simply adding a SetCudaDeviceId API here. https://github.com/Tencent/TurboTransformers/blob/f2d66bc12f0b904328372f472f6379aba50007cc/turbo_transformers/core/config.cpp Add register a Python API here. https://github.com/Tencent/TurboTransformers/blob/f2d66bc12f0b904328372f472f6379aba50007cc/turbo_transformers/python/pybind.cpp

[zejunwang1]

在加载bert模型npz文件时： https://github.com/Tencent/TurboTransformers/blob/master/turbo_transformers/loaders/npz_load.h 设置core::NewDLPackTensorT里面的device_id为3：

core::Tensor LoadT(const std::string &name) { auto &array = view_[name]; std::vector<int64_t> shape; shape.resize(array.shape.size()); std::copy(array.shape.begin(), array.shape.end(), shape.begin()); core::Tensor tensor(core::NewDLPackTensorT<T>(shape, device_)); core::Copy(array.data<T>(), tensor.numel(), DLDeviceType::kDLCPU, tensor); return tensor; }

改为： core::Tensor tensor(core::NewDLPackTensorT<T>(shape, device_, id_));

bert进行推理时，发现device_id是3，但是程序还是在0卡上：

The embedding layer out_tensor device id: 3 The embedding layer embedding_table device id: 3

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | 0 20564 C ./src/corrector_test 1121MiB |

有能简单设置其他gpu device id的方法嘛