beam icon indicating copy to clipboard operation
beam copied to clipboard

Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs

Open github-actions[bot] opened this issue 1 year ago • 0 comments

Performance change found in the test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU. Test link - https://github.com/apache/beam/blob/42d0a6e3564d8b9c5d912428a6de18fb22a13ac1/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy#L151 Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7


timestamp: Tue May 21 07:03:32 2024, metric_value: 157167.31
timestamp: Mon May 20 06:50:24 2024, metric_value: 156648.68
timestamp: Sun May 19 06:46:25 2024, metric_value: 154421.54 <---- Anomaly
timestamp: Mon Mar  4 06:58:11 2024, metric_value: 204363.73
timestamp: Sat Mar  2 06:45:32 2024, metric_value: 212875.61
timestamp: Fri Mar  1 06:58:29 2024, metric_value: 219374.20
timestamp: Thu Feb 29 06:49:29 2024, metric_value: 230385.92
timestamp: Mon Feb 26 06:50:12 2024, metric_value: 223331.64
timestamp: Sun Feb 25 06:47:31 2024, metric_value: 221115.22
timestamp: Sat Feb 24 06:45:36 2024, metric_value: 225314.40
timestamp: Fri Feb 23 06:53:41 2024, metric_value: 217522.32
timestamp: Wed Feb 21 07:12:25 2024, metric_value: 219607.02
timestamp: Tue Feb 20 06:53:46 2024, metric_value: 213568.59

github-actions[bot] avatar May 21 '24 22:05 github-actions[bot]