storage

storage copied to clipboard

Generating a Cosmoflow dataset is very slow

1

When I was generating the dataset for Cosmoflow, the generation was slow, and it took a minute interval to print the logs once. When I switch the data format to...

Linzsd

How to submit result?

1

Linzsd

Can I submit our benchmark result (v0.5 based) now?

2

https://mlcommons.org/benchmarks/storage/ shows us benchmark results from a couple of storage vendors, such as DDN/Weka. I just completed the test with our storage system and mlperf/storage v0.5 in CLOSED division. I'd...

lioupayphone

It would be useful to have a reproducible script-based method to configure (say) two targets (one compute and one storage) and run the ML Perf Storage benchmark across them. This...

sbates130272

mpirun: Error: unknown option "-o"

2

Linzsd

Workload resnet50: no IOs are generated at all with dali while benchmark reports it is running

1

Steps: - git clone -b v1.0-rc1 --recurse-submodules https://github.com/mlcommons/storage.git - pip3 install -r dlio_benchmark/requirements.txt - ./benchmark.sh datagen --workload resnet50 --accelerator-type h100 --num-parallel 8 --param dataset.num_files_train=1200 --param dataset.data_folder=/mnt/1/ifs/data/rosnet50_05_04_2024_x02 - ./benchmark.sh run --hosts...

alexander272272

mount: IP:/ifs on /mnt/1/ifs type nfs (rw,relatime,vers=3,rsize=131072,wsize=524288,namlen=255,hard,forcerdirplus,proto=rdma,nconnect=24,port=20049,timeo=600,retrans=2,sec=sys,mountaddr=IP,mountvers=3,mountproto=tcp,local_lock=none,addr=IP) h# cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.1 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.1 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy h# ./benchmark.sh run...

alexander272272

Can I use a GPU other than a100, h100?

1

I want to measure storage performance, but I want to use GPU model(NVIDIA GeForce RTX 2060) I currently have. Is it possible? First of all, I measured it with the...

fadu-dufresne

AU doesn't meet expectation

13

I am new, just had the storage benchmark run the 1st time ever and got this line: Training_au_meet_expectation = fail. My questions are: 1. Will this block me from submitting...

hanyunfan

can we enable direct_io flag when issue file system API from MLPerf storage test suite

1

currently, we notice, MLPerf storage still leverage page cache a lot, not stress storage enough. can we support one branch that issue direct IO for all places? I see there...

gaowayne