burn icon indicating copy to clipboard operation
burn copied to clipboard

Add ResNet benchmark

Open laggui opened this issue 10 months ago • 4 comments

Related Issues/PRs

Closes #1521

Changes

Added ResNet-50 inference in backend-comparison

  • New build script to fetch the resnet-burn implementation and apply a diff patch to be compatible with the current version of Burn on main
  • ResNetBenchmark executes a forward pass on a random input w/ shape [1, 3, 224, 224]
  • Two minor typos fixed

Note: doesn't include autodiff but could be added if required, similar to the current gelu benchmark

Testing

Ran the benchmark with multiple backends

cargo run --release --bin burnbench -- run --benches resnet50 --backends tch-gpu tch-cpu wgpu wgpu-fusion ndarray

Note: it doesn't run on candle because AdaptiveAvgPool2d in the model uses padding which is not supported.

laggui avatar Mar 26 '24 18:03 laggui

Hmm CI failed but the failing test is unrelated to this PR

/edit: looking for feedback on the benchmark implementation but let's wait until we optimize some stuff before we actually merge it.

This will simply be a helpful tool to evaluate the improvements along the way and once we're satisfied with the results we can merge.

laggui avatar Mar 26 '24 18:03 laggui

LGTM so far, great work!

syl20bnr avatar Mar 26 '24 19:03 syl20bnr

Codecov Report

Attention: Patch coverage is 98.33333% with 1 line in your changes missing coverage. Please review.

Project coverage is 85.93%. Comparing base (6f0e61a) to head (98311f2). Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
backend-comparison/build.rs 98.33% 1 Missing :warning:
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1534   +/-   ##
=======================================
  Coverage   85.92%   85.93%           
=======================================
  Files         750      751    +1     
  Lines       94357    94417   +60     
=======================================
+ Hits        81076    81135   +59     
- Misses      13281    13282    +1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Mar 26 '24 19:03 codecov[bot]

This PR has been marked as stale because it has not been updated for over a month

github-actions[bot] avatar Apr 28 '24 12:04 github-actions[bot]

This PR has been marked as stale because it has not been updated for over a month

github-actions[bot] avatar Aug 27 '24 12:08 github-actions[bot]