torch2trt
torch2trt copied to clipboard
Failed to run the demo on Jetson Nano
Dear Authors,
We have the following hardware and software configuration:
Hardware: Nvidia Jetson Nano 4GB Software: 1. JetPack 4.4 (L4T 32.4.3); 2. PyTorch 1.6.0; 3. TensorRT 7.1.3.0
However, we failed to run even the demo example as follows:
import torch from torch2trt import torch2trt from torchvision.models.alexnet import alexnet
create some regular pytorch model...
model = alexnet(pretrained=True).eval().cuda()
create example data
x = torch.ones((1, 3, 224, 224)).cuda()
convert to TensorRT feeding sample data as input
model_trt = torch2trt(model, [x])
y = model(x) y_trt = model_trt(x)
check the output against PyTorch
print(torch.max(torch.abs(y - y_trt)))
The system gives a hint of "killed". We notice that you have displayed some experimental results on Jetson Nano. Would you mind suggesting the software version, such as PyTorch or TensorRT?
Looking forward to hearing from you!
Best Regards, Qiang Wang
Any luck with this? I am struggling with the same issue... Thanks!
I would like to say that may be the issue of insufficient CPU main memory of the mobile device. I have successfully run it on Jetson TX2, which has 8 GB of main memory.
Any luck with this? I am struggling with the same issue... Thanks!
I have the same issue on Jetson Nano 4GB version. More precisely, It was 'Killed' on the line
model_trt = torch2trt(model, [x])
After enlarging the Linux swap space from 2GB(default) to 4GB, it ran successfully.
jetson nano 4GB swap space12GB i got this error:
[TensorRT] ERROR: 4: Tensor: output_0 trying to set to TensorLocation::kHOST but only kDEVICE is supported (only network inputs may be on host)
[TensorRT] WARNING: Tensor DataType is determined at build time for tensors not marked as input or output.
[TensorRT] ERROR: 4: [network.cpp::validate::2506] Error Code 4: Internal Error (Tensor: input_0 set to TensorLocation::kHOST but only kDEVICE is supported (only RNNv2 allows host input))
pytorch time: 0.0016689300537109375 ms
Traceback (most recent call last):
File "nvidia_torch2trt.py", line 23, in
code: import torch from torch2trt import torch2trt #from torchvision.models.alexnet import alexnet from pytorch2onnx import PPLCNet_x2_5 import time import tensorrt as trt
create some regular pytorch model...
model = PPLCNet_x2_5() model.load_state_dict(torch.load('lcnet_x2_5.pth', map_location='cpu'), strict=True) model.eval()
create example data
x = torch.ones((1, 3, 224, 224))
convert to TensorRT feeding sample data as input
model_trt = torch2trt(model, [x], fp16_mode=True, log_level=trt.Logger.INFO, strict_type_constraints=True)
for i in range(50): s = time.time() #y = model(x) e = time.time() print("pytorch time: {} ms".format((e-s)*1000.0)) y_trt = model_trt(x) print("tensorrt time: {} ms".format((time.time()-e)*1000.0))
I have the same issue on Jetson Nano 4GB version. More precisely, It was 'Killed' on the line
model_trt = torch2trt(model, [x])
After enlarging the Linux swap space from 2GB(default) to 4GB, it ran successfully.
I am using Jetson Xavier NX, and I added 20G swap. My process was still killed when converting the model. Only about 4G swap was used. I don't know why.
I have the same issue on Jetson Nano 4GB version. More precisely, It was 'Killed' on the line
model_trt = torch2trt(model, [x])
After enlarging the Linux swap space from 2GB(default) to 4GB, it ran successfully.
jetson tx2 enlarge swap to 4GB fix the error, thanks