ck

ck copied to clipboard

Mark most CM scripts with "prototype":True flag unless have a code-quality standard, proper tests and documentation.

gfursin

Some feedback from our users and MLCommons: - Convert all prints into logger calls with different levels of verbosity

gfursin

Running dlrm cpu inference ends up using resnet50

2

Ran with `cm run script "app mlperf reference inference _dlrm _cpu" --env.CM_RERUN`. From the attached log file, I saw pulling dlrm ` cm run script "get dlrm src"`, but when...

yuyantingzero

Missing whl nvmitten-0.1.3-cp38-cp38-linux_x86_64.whl when running bert-99 or rnnt

3

When trying to run bert-99 or rnnt inference. always fails with: /root/cm/bin/python3 -m pip install "/opt/nvmitten-0.1.3-cp38-cp38-linux_x86_64.whl" WARNING: Requirement '/opt/nvmitten-0.1.3-cp38-cp38-linux_x86_64.whl' looks like a filename, but the file does not exist ERROR:...

stbailey001

gfursin

Whenever CM script fails because of some wrapped code (such as MLPerf inference), add support in meta where to report an error

1

We got a feedback from CM users that when some tool/library fails when wrapped by a CM script, it can be useful to provide a link to a repository of...

gfursin

[W:onnxruntime:, graph.cc:3593 CleanUnusedInitializersAndNodeArgs] Removing initializer 'bert.pooler.dense.bias'. It is not used by any node and should be removed from the model.

1

When I run the command cm run script --tags=generate-run-cmds,inference,_find-performance,_all-scenarios --model=bert-99 --implementation=reference --device=cuda --backend=onnxruntime --category=edge --division=open --quiet There are some warnnings which I don't konw it matters. 2024-03-23 16:39:29.057780772 [W:onnxruntime:, graph.cc:3593...

KingICCrab

Why the results both on CPU and GPU are same?

2

When I run cmr "run mlperf inference generate-run-cmds _submission" --quiet --submitter="MLCommons" --hw_name=default --model=bert-99 --implementation=reference --backend=onnxruntime --device=cuda --scenario=Offline --adr.compiler.tags=gcc --target_qps=1 --category=edge --division=open default-reference-gpu-onnxruntime-v1.17.1-default_config +---------+----------+----------+--------+-----------------+---------------------------------+ | Model | Scenario | Accuracy |...

KingICCrab

fatal: unable to access 'https://github.com/mlcommons/ck/': GnuTLS recv error (-9): Error decoding the received TLS packet.

7

I want to reproduce nvidia-bert https://github.com/mlcommons/ck/blob/master/docs/mlperf/inference/bert/README_nvidia.md#build-nvidia-docker-container-from-31-inference-round when I run "cm docker script --tags=build,nvidia,inference,server", I encounter some problems. => ERROR [10/12] RUN cm pull repo mlcommons@ck 104.6s ------ > [10/12] RUN...

KingICCrab

/cm/bin/python3: No module named pip

4

When I run the command cm run script "app mlperf reference inference _bert-99 _offline _onnxruntime _cuda _fp32" Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.10/importlib/metadata/__init__.py", line...

KingICCrab