inference

inference copied to clipboard

CM script failed to run harness after docker done

2

Hi @arjunsuresh **I am running the Resnet50 benchmark with the command:** cm run script --tags=run-mlperf,inference,_find-performance,_full,_r4.1-dev \ --model=resnet50 \ --implementation=nvidia \ --framework=tensorrt \ --category=edge \ --scenario=Offline \ --execution_mode=test \ --device=cuda \...

Bob123Yang

We need to update the [system_requirements](https://github.com/mlcommons/inference/blob/docs/docs/system_requirements.yml) file for all the reference implementations and then use the same in the [inference docs](https://docs.mlcommons.org/inference/)

arjunsuresh

Error: Cannot perform a '--user' install. User site-packages are not visible in the virtual environment. CM error : portable CM script failed (name = build-mlperf-inference-server-nvidia, return code = 256)

9

 please help me out with this error.

shyambansal17

llama3: performance_sample_count update

1

https://github.com/mlcommons/inference/blob/2d2eb3081dcda64e766a28dc1b6fb9c40fb9eefa/tools/submission/submission_checker.py#L432C28-L432C32 should this be 8313 as per length of 405b official dataset?

viraatc

The following issues appear when running the LLM reference implementation. Multiple GPUs issue: ``` (VllmWorkerProcess pid=1795) ERROR 12-03 18:49:03 multiproc_worker_utils.py:231] Exception in worker VllmWorkerProcess while processing method init_device: Cannot re-initialize...

pgmpablo157321

The following issues appear when running the LLM reference implementation Dependencies in the docker container: ``` Collecting mistral-common>=1.4.4 (from mistral-common[opencv]>=1.4.4->vllm==0.6.3->-r requirements.txt (line 8)) Downloading mistral_common-1.5.0-py3-none-any.whl.metadata (4.6 kB) Downloading mistral_common-1.4.4-py3-none-any.whl.metadata (4.6...

pgmpablo157321

GNN/R-GAT Inference Submission disallows sorting of the nodes. Need a mechanism to prevent and detect sorting.

6

GNN/R-GAT Inference task force agreed that the sorting of the nodes is disallowed. Sorting of the nodes reduces number of unique nodes and artificially increases the throughput up to 6x....

ukurkure

mixtral-8x7b baseline reproduction : gsm8k : 72.00

1

Hi, I tried to reproduce your baseline on mixtral-8x7b, but the accuracy on gsm8k is 72.00 instead of 73.66. Can you reproduce it? Also, what is the version of transformers?

DehuaTang

All merged branch from #1934-#1931

2

Updated 1 pull request submission from previous pull requests (#1934, #1933, #1932, #1931), involving functionality of multi-gpu & multi-node for pytorch & migraphx backend. Submitted by SCC24 UCSD Team Zixian...

zixianwang2022

[v4.1 inference] Detected system did not match any known systems.

1

Hi, I'm facing some issues when i tried running the benchmark for 3d-unet. When i ran **make run RUN_ARGS="--benchmarks=3d-unet --scenarios=offline,server""** Got the errors, which is also the system i'm working...

loganwuw