algorithmic-efficiency
algorithmic-efficiency copied to clipboard
mlcommons
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Fixes https://github.com/mlcommons/algorithmic-efficiency/issues/758 and https://github.com/mlcommons/algorithmic-efficiency/issues/719 These two lines of code are sufficient to enable many extrapolation algorithms, including the weight averaging variants such as stochastic weight averaging, exponential moving average, schedule-free...
## Description First of all if this feature is already supported then please consider this as a question. I'm trying to reproduce some results of existing algorithms such as SGD,...
## Description On `imagenet_resnet` workload, I encounter the following warning when running with DDP and `pytorch` framework. > /u/najroldi/miniconda3/envs/alpe/lib/python3.8/site-packages/torch/autograd/init.py:251: UserWarning: Grad strides do not match bucket view strides. This may...
Running the example command provided in the readme [here](https://github.com/mlcommons/algorithmic-efficiency/tree/5b4914ff18f2bb28a01c5669285b6a001ea84111?tab=readme-ov-file#getting-started). ```BASH python3 submission_runner.py \ --framework=pytorch \ --workload=mnist \ --experiment_dir=$HOME/experiments \ --experiment_name=my_first_experiment \ --submission_path=reference_algorithms/paper_baselines/adamw/pytorch/submission.py \ --tuning_search_space=reference_algorithms/paper_baselines/adamw/tuning_search_space.json ``` (after switching ...adamw/**jax**/submission.py to ...adamw/**pytorch**/submission.py)...
In submission_runner.py, if we are in the self-tuning rules, the hyperparameters argument to train_once will always be None. Then in this code snippet ``` dropout_rate = None aux_dropout_rate = None...
AlgoPerf submitter team reports that they are no longer able to reproduce the NAdam baseline results in PyTorch using the current repo in PyTorch on the ImageNet workloads (both ResNet...
**tl;dr**: We should let the submission know if an evaluation is going to happen at the current step or not. ## Description Currently, there is no easy way for the...
## Feature request: allow users to skip eval on train and test Evaluating on the training and test sets is time-consuming and not necessary for self-reporting results. We should add...
### Workload LM1B #### Task Transformer language model training, resembling https://github.com/google/flax/tree/main/examples/lm1b . #### Dataset LM1B dataset. https://www.tensorflow.org/datasets/catalog/lm1b #### Model decoder-only Transformer, encoder+decoder based Transformer. #### Reference Implementation https://github.com/google/flax/tree/main/examples/lm1b --- ###...
