gpu-manager Does it support NVIDIA TensorRT

Does it support NVIDIA TensorRT?

inside container cannot read tensorrt engine which generate from same deivce and cuda version ,shows

[TensorRT] WARNING: using an engine plan file across different models of devices is not recommended and is likely to affect performance or enven cause errors

inside container some onnx models cannot build tensorrt engine, report segmentation fault

any solution or suggestion?

Aug 05 '21 07:08 ivoxx

I had the same problem. It is normal to use the whole card, but not once it is fragmented.

Aug 05 '21 07:08 xwttzz

The gpu-manager version and log?

Aug 05 '21 07:08 mYmNeo

日志看起来是正常的。 manager的版本是1.1.4 @mYmNeo 方便加个微信号？

Aug 05 '21 07:08 xwttzz

Please follow the FAQ, provides the application container log

Aug 06 '21 01:08 mYmNeo

Aug 06 '21 01:08 ivoxx

this is the container log. @mYmNeo

Aug 06 '21 02:08 xwttzz

You need following the FAQ, set environment first, then run your application, the expected log should have /tmp/xxx pattern

Aug 09 '21 07:08 mYmNeo

I have followed the FAQ。 I can't find any useful logs in this directory。Which log in this directory do you want。 @mYmNeo

Aug 10 '21 03:08 xwttzz

I have followed the FAQ。 I can't find any useful logs in this directory。Which log in this directory do you want。 @mYmNeo

Export LOGGER_LEVEL environment
run your application
vcuda log will print on the screen

Aug 10 '21 03:08 mYmNeo

https://gist.github.com/xwttzz/1f4b3794a2fb19f430ebea828030d145 The above link shows all logs after VCUDA is enabled。 @mYmNeo

Aug 10 '21 06:08 xwttzz

https://gist.github.com/xwttzz/1f4b3794a2fb19f430ebea828030d145 The above link shows all logs after VCUDA is enabled。 @mYmNeo

The error shows you application missing library ImportError: libnvinfer.so.7: cannot open shared object file: No such file or directory, this is not VCUDA library

Aug 11 '21 01:08 mYmNeo

The confusing part is that when I set vcore to 100 it is ok, only when it is fragmented 。 Besides, it has been confirmed that LibnVinfer has been installed in the container. @mYmNeo

Aug 11 '21 05:08 xwttzz

How about re-pull the image thomassong/gpu-manager:1.1.5. We have fixed a recursive problem about vcuda.

Aug 11 '21 07:08 mYmNeo

I updated the latest log in the original address. [https://gist.github.com/xwttzz/1f4b3794a2fb19f430ebea828030d145] Libnvinfer was a historical log and it has nothing to do with this。 @mYmNeo

Aug 11 '21 07:08 xwttzz