Wes Vaske
Wes Vaske
**Please acknowledge the following before creating a ticket** - [x ] I have read the GitHub issues section of [REPORTING-BUGS](../blob/master/REPORTING-BUGS). **Description of the bug:** When I run a test against...
Currently a data format (npc, tfrecord, etc) maps to a single sample type (image, text, etc). We will need a method for the various container formats to support different sample...
Checkpointing is a critical piece of AI/ML and there are frequently systems deployed solely to hold checkpoints. I have a basic PoC for running a checkpoint here: https://github.com/wvaske/checkpoint_bench/blob/main/do_checkpoints.py What features...
Benchmark script is getting migrated from bash to python for better integration with results checking scripts. - Update to latest version of DLIO - Started updating rules document - Separate...
After the 1.0 submission we found that usability of the benchmark can be greatly improved. This issue will track the 'sub-issues' we intend to address for the 2.0 release. Please...
One of the metrics reported with results submissions is the standard deviation of the primary metric. The rules currently imply a requirement for consistency of runs but we don't have...