inference

inference copied to clipboard

[URGENT] RetinaNet mAP is 37.50% on main, 37.55% on r2.1 branch

10

According to the [README in the main branch](https://github.com/mlcommons/inference/tree/master/vision/classification_and_detection), the reference fp32 accuracy of RetinaNet is `37.5%` or `37.50%`. With this refence value, the 99% threshold for valid submissions is `37.50%...

psyhtest

Fix an unnecessary '.' at beginning of notes if hw_notes is empty

2

arjunsuresh

Map SSD-ResNet34 (now only in Open) to RetinaNet

4

psyhtest

add Dockerfile to rnnt

2

Change-Id: I0dcd0928ab3e298d01136a169eb8f95d7e7e169b

zeroorhero

Rename submission-checker.py to submission_checker.py

3

This rename can help in importing submission_checker in other python files

arjunsuresh

How to set batch size with Bert

1

Hi, I am trying to do inference with Bert. However, I don't find any API to set batch size in **run.py**. Could you pls. help me with that? Appreciate a...

JunxiChhen

Add bert-99 and bert-99.9 to the submission-checker model_mapping guess

2

Allows models prepended "bert-99" or "bert-99.9" to be mapped to their respective model category similarly to CV models i.e. bert-99_model-A, bert-99_model-B, etc. would be evaluated as bert-99 models. This was...

mgoin

Fix RetinaNet mAP=37.50%.

1

Fixes https://github.com/mlcommons/inference/issues/1208.

psyhtest

What is the intended way to get statistics from loadgen? Currently in the [mlperf mobile app](https://github.com/mlcommons/mobile_app_open) we use the `ReportLatencyResults` function to calculate performance from latency. However, recently this function...

d-uzlov

Move PerformanceResult/Summary to results.h/cc

3

Fixing an existing TODO in code.

saar-eliad