Can not create mlperf_storage_report.json
Following up on the concern raised in issue #177 , I noticed that although the Submission Guidelines have been updated, the mlperf_storage_report.json file still cannot be generated as expected. When executing:
mlpstorage reports reportgen --output-dir <output path> --results-dir <results path> --closed
only results.json and results.csv are produced. Additionally, the --output-dir parameter appears to have no effect—the files are forcibly written to the directory specified by --results-dir.
Therefore, I would like to suggest the following revision to the Submission Guidelines:
Original:
Final submission JSON summary file mlperf_storage_report.json. The JSON file must be generated using the mlpstorage reportgen script. The mlpstorage reportgen command must be run on the rank0 machine in order to collect the correct set of files for the submission.
Suggested revision:
Final submission JSON summary file results.json. The JSON file must be generated using the mlpstorage reports reportgen script. The mlpstorage reports reportgen command must be run on the rank0 machine in order to collect the correct set of files for the submission.
Alternatively, if mlperf_storage_report.json is still intended to be part of the final submission, the mlpstorage reports Python code may need to be updated accordingly to ensure it generates this file.
Additionally, please consider removing the section "Structure the output as shown in this example", as the linked example still refers to generating the report using benchmark.sh, which no longer exists and may cause confusion.