inference icon indicating copy to clipboard operation
inference copied to clipboard

Reference implementations of MLPerf™ inference benchmarks

Results 200 inference issues
Sort by recently updated
recently updated
newest added

If onnxruntime is supporting more than one execution provider, bert-onnxruntime reference implementation is failing with below error. ``` Loading ONNX model... Traceback (most recent call last): File "/media/arjun/T7_Touch/hp-envy/CM/repos/local/cache/abdb11453edc4744/inference/language/bert/run.py", line 120,...

TEST04 is only applicable to ResNet50. Is it still worthwhile?

Is TEST05 still worthwhile?

I would like to ask, how to use the ssd-mobilenet model provided by MLcommons on pytorch and convert it into the corresponding jit model, can you provide a demo or...

There is a slight difference between repo submission checker compared to UI submission checker. For example, 50GB limit is imposed only in UI submission checker. We need to sync both...

Inference v3.0

Hi, I'm looking for a speech to text benchmark for datacenter. I found that MLPerf still use RNN-T for the speech to text benchmark. However, there are many other better...

Hello There! I noticed that the power checker classifies the power warnings as an error. Here is the output of using the power checker, it passes and has no errors:...

Compliance TEST01 can fail for some systems due to non-determinism and an additional [non-determinism test](https://github.com/mlcommons/inference/tree/master/compliance/nvidia/TEST01#non-determinism) can let this pass. In case of such a scenario, it would be good to...

when I run cm run script --tags=generate-run-cmds,inference,_find-performance,_all-scenarios --model=bert-99 --implementation=reference --device=cuda --backend=onnxruntime --category=edge --division=open --quiet the error is /home/zhaohc/cm/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider,...

# Cross Posting From Review Committee on behalf of the original authors. According to [MLCommons policies, submission rules](https://github.com/mlcommons/policies/blob/master/submission_rules.adoc#inference-1) the structure of a submission should conform to: measurements/ / / __.json...