ck
ck copied to clipboard
mlcommons
Collective Mind (CM) is a small, modular, cross-platform and decentralized workflow automation framework with a human-friendly interface and reusable automation recipes to make it easier to build, run...
In CK package release versions can be handled by variations as done in the below package for opencv. I believe the same can be used to replace versions in CK....
As suggested by our users, we started working on an out-of-the-box support to run MLPerf inference benchmarks: https://github.com/ctuning/ck/blob/master/docs/mlperf-automation/README.md .
If we recorded experiments during "ck benchmark program --record" and we used "variations" to install related CK packages, then when we try to reply recorded experiment, CK will not install...
I aggregated ideas from the old tickets to improve CK. Some are still relevant but not urgent:
* Add detection of wrong/broken repositories during "ck pull repo" (https://github.com/ctuning/ck/issues/77) At the moment, there can be a strange behavior when pulling non-existent GitHub repository (see dividiti/ck-caffe#107). The reason is...
Ongoing work: https://github.com/ctuning/ck/blob/master/docs/mlperf-automation/inference/workflow.md
I would like to substitute travis/appveyour tests with more rigorous testing of CK and all related components using different ML/SW/HW stacks (including MLPerf benchmark). As a base, we can use...
Add CK package for [Imagenette](https://github.com/fastai/imagenette) to our [automation recipes](https://github.com/ctuning/ai/tree/main/package) and test with [MLPerf inference workflow for ONNX/TVM](https://github.com/octoml/mlops/tree/main/program/mlperf-inference-bench-image-classification-onnx-cpu). CC: @guschmue - Guenther suggested to test this dataset as a potential substitution...
1) When building TVM on MacOS, dynamic libraries have a `.dylib` extension, but for checking `end_full_path_universal` in the `CK / local / venv / mlperf-inference / CK / ck-mlops /...
