multi-model-server getAvailableGpu() inaccurate on Jetson

getAvailableGpu() inaccurate on Jetson

Open ThomasDelteil opened this issue 6 years ago • 3 comments

Jetsons do not use nvidia-smi, hence inaccurate gpu count, 0 instead of one.

Feb 12 '19 22:02 ThomasDelteil

@ThomasDelteil : Thanks for pointing this out. Can you specify which AWS host did you test this on?

Feb 12 '19 22:02 vdantu

This a Jetson TX2 which is an edge device, it isn't running on AWS.

One work around is to create a executable nvidia-smi file in usr/local/bin that contains the following:

#!/bin/bash
echo "index"
echo "second line"

Feb 12 '19 22:02 ThomasDelteil

@ThomasDelteil pointed out how MXNet deals with this as follows:

https://github.com/apache/incubator-mxnet/blob/992c3c0dd90c0723de6934e826a49bad6569eeac/include/mxnet/base.h#L420

MMS could extract this logic and make it a standalone module that can be used by MMS.

Mar 15 '19 22:03 vdantu