optimum-benchmark icon indicating copy to clipboard operation
optimum-benchmark copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Add t4 for llm perf leaderboard

Open baptistecolle opened this issue 7 months ago • 0 comments

Summary

This PR adds Nvidia T4 to the LLM-Perf Leaderboard:

Fixes

  • Fixed the trust_remote_code issue that was broken for the CI/CD pipeline.

Features

  • Added a new machine configuration (T4) to run the LLM performance benchmarking code.
  • Updated the A100 configuration to use the new runs-on definition (without tags).

Related PRs and Discussions

Workflow Trigger Changes

One thing that I am unsure about is that I modified the trigger for the workflow. To reduce unnecessary compute, you can manually trigger the workflow, and it is triggered with each new release of the repo (releases).

I think this could be better. What do you think, @IlyasMoutawwakil?

baptistecolle avatar Aug 01 '24 08:08 baptistecolle