beam icon indicating copy to clipboard operation
beam copied to clipboard

Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_inference_batch_latency_micro_secs:mean_inference_batch_latency_micro_secs

Open github-actions[bot] opened this issue 1 year ago • 0 comments

Performance change found in the test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_inference_batch_latency_micro_secs for the metric: mean_inference_batch_latency_micro_secs.

For more information on how to triage the alerts, please look at Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU. Test link - https://github.com/apache/beam/blob/42d0a6e3564d8b9c5d912428a6de18fb22a13ac1/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy#L151). Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?from=now-90d&to=now&viewPanel=2


timestamp: Tue May 21 07:03:32 2024, metric_value: 1251828.36
timestamp: Mon May 20 06:50:24 2024, metric_value: 1097567.28
timestamp: Sun May 19 06:46:25 2024, metric_value: 901021.34 <---- Anomaly
timestamp: Mon Mar  4 06:58:11 2024, metric_value: 4702155.36
timestamp: Sat Mar  2 06:45:32 2024, metric_value: 3264010.21
timestamp: Fri Mar  1 06:58:29 2024, metric_value: 2738479.76
timestamp: Thu Feb 29 06:49:29 2024, metric_value: 4662914.06
timestamp: Mon Feb 26 06:50:12 2024, metric_value: 3928341.12
timestamp: Sun Feb 25 06:47:31 2024, metric_value: 3650458.48
timestamp: Sat Feb 24 06:45:36 2024, metric_value: 3540819.03
timestamp: Fri Feb 23 06:53:41 2024, metric_value: 4560157.85
timestamp: Wed Feb 21 07:12:25 2024, metric_value: 4883101.24
timestamp: Tue Feb 20 06:53:46 2024, metric_value: 5398728.80

github-actions[bot] avatar May 21 '24 22:05 github-actions[bot]