How to use ssd-mobilenet 300x300 (torch version) provided by mlcommons for inference process

[lixiaolx]

I would like to ask, how to use the ssd-mobilenet model provided by MLcommons on pytorch and convert it into the corresponding jit model, can you provide a demo or link?

[pgmpablo157321]

@lixiaolx Checking the reference implementation, I can see that unfortunately this model is not implemented. Since it was removed in version 2.1, there is no incentive to fix it. There a few steps you can follow to run this benchmark, but you'll have to modify some code.

1. Clone the repository with its submodules:

git clone --recurse-submodules https://github.com/mlcommons/inference.git --depth 1

2. Install LoadGen

cd inference/loadgen
CFLAGS="-std=c++14" python setup.py develop
cd ..

3. Go to the vision/classification_and_detection folder and download the model and the dataset there. The dataset in its own folder. Download model:

cd vision/classification_and_detection
wget https://zenodo.org/record/3239977/files/ssd_mobilenet_v1.pytorch

Download dataset:

mkdir data
cd data
wget http://images.cocodataset.org/zips/val2017.zip
wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
unzip val2017.zip
unzip annotations_trainval2017.zip

4. Open the file vision/classification_and_detection/run_common.sh and add the following code in the pytorch section:

if [ $name == "ssd-mobilenet-pytorch" ] ; then
    model_path="$MODEL_DIR/ssd_mobilenet_v1.pytorch"
    profile=ssd-mobilenet-pytorch
fi

5. With this setup you should be able to run the benchmark from a jupyter notebook. The notebook vision/classification_and_detection/GettingStarted.ipynb contains most of the code for this, but the examples it has are very different. You can run the benchmark from a notebook in the folder vision/classification_and_detection with the following cells:

import os
root = os.getcwd()
os.environ['MODEL_DIR'] = root
os.environ['DATA_DIR'] = os.path.join(root, "data")
os.environ['EXTRA_OPS'] ="--time 600"

Inside of the jupyter notebook, you can run the performance benchmark:

!./run_local.sh pytorch ssd-mobilenet [device] --scenario [Scenario]

Inside of the jupyter notebook, you can compute the accuracy:

!./run_local.sh pytorch ssd-mobilenet [device] --accuracy

Where device is cpu or gpu