Grigori Fursin

[MLPerf project] modularize MLPerf inference benchmark and automate submission

1

# Motivation This project aims at decomposing MLPerf inference benchmarking into a database of reusable, portable, customizable and deterministic scripts with a unified CLI, common Python API and extensible JSON/YAML...

[CK2/CM project] implement modular ML benchmarking based on loadgen and CK2(CM)

1

We want to develop a CK2(CM) script to enable modular ML benchmarking across diverse ML models, data sets, engines, libraries and platforms (available as CM scripts too) based on [C++...

[CK2/CM scripts] add system deps to install-python-src and install-python-venv

1

Add either complete system deps (get-sys-utils-cm) or minimal for python (new script) when building python from sources on UBUNTU and MacOS. We can skip this dependency with --skip-system-deps flag (already...

Whenever we release the CK2 version, we should add the branch protection as suggested by @morphine00: To add branch protection, go to Settings, Branches, "Add branch protection rule," type in...

[CK2/CM scripts] discuss and unify how to reinstall/rebuild packages

1

Installation of packages often requires multiple steps: * clone/download sources * configure * build * install If it fails, we want to be able to restart such CM scripts from...

We need to add logging of the main [CK2/CM access function](https://github.com/mlcommons/ck/blob/master/ck2/cmind/core.py#L132) and other core functions. Maybe we should add logging in these 2 places: * https://github.com/mlcommons/ck/blob/master/ck2/cmind/core.py#L163 (to record i var)...