algorithmic-efficiency
algorithmic-efficiency copied to clipboard
mlcommons
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
For this PR, I added two assertions to validate the timing and evaluation consistency at the end of the training loop: 1) Duration consistency check: The total duration of training...
BN Fixes
There are some subtle issues with how BatchNorm is handled in the PyTorch version of the code. Currently, `workload.model_fn` has an `update_batch_norm` parameter, which in theory should allow the submission...
Our current API has 2 dropout related limitations: 1. Currently, in the external tuning ruleset we read the dropout value from the hparam config and pass it to the model...
Add technical documentation for default behavior in both rulesets.
## Description Currently, `update_params` has no up-to-date information about the elapsed time since start. My motivation for adding this feature is to simplify the implementation of a _time-based learning rate...
Updated the CONTRIBUTING readme to include detailed steps for installing and configuring the Google Cloud Ops Agent. This will help users set up monitoring for their VM logs.
See https://github.com/google-research/big_vision/issues/109, fix suggested by @yeqingli in https://github.com/tensorflow/models/pull/11219#pullrequestreview-2355525720. In short, the original implementation of the contrast() transform which is copied 4-5+ times is broken: What is meant to be the...
## Purpose The goal of this PR is to allow model parameter and optimizer state sharding, and also to migrate the JAX code from using jax.pmap to using jax.jit. ##...
I added some code to the ogbg workload according to the migration guide helpfully provided by Ahmed. Unfortunately, there seems to be some bugs that I still hope to fix.
A test is added to evaluate timing consistency in the MNIST training workload using PyTorch and JAX. It ensures that the total reported training time matches the sum of submission,...