dnceng icon indicating copy to clipboard operation
dnceng copied to clipboard

Production - [Alerting] Android devices disconnected

Open dotnet-eng-status[bot] opened this issue 7 months ago • 44 comments

:broken_heart: Metric state changed to alerting

Machine DNCENGWIN-020 in windows.11.amd64.android.open has been disabled Machine DNCENGWIN-096 in windows.11.amd64.android.open has been disabled Machine DNCENGWIN-133 in windows.11.amd64.android.open has been disabled Machine DNCENGWIN-143 in windows.11.amd64.android.open has been disabled

The machines were disabled because it was reporting a failure for several consecutive reports.

Investigation incident for 'DNCENGWIN-020' has been opened: https://portal.microsofticm.com/imp/v5/incidents/details/629187415/summary?os=28131&tm=74522&ttl=Broken%20System%20or%20service%20-%20Android%20phones%20disconnected&ci=1&sev=3&is=28131&itm=&ic=&sti=&spi=&csi=false&it=LiveSite&env=&dc=&cp=557076049&tag=&ir=

Machines have been re-eabled, waiting to see if the persistent alerts resolve: Machine DNCENGWIN-020 in windows.11.amd64.android.open has been enabled Machine DNCENGWIN-096 in windows.11.amd64.android.open has been enabled Machine DNCENGWIN-133 in windows.11.amd64.android.open has been enabled Machine DNCENGWIN-143 in windows.11.amd64.android.open has been enabled

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 83

Go to rule

@dotnet/dnceng, @dotnet/prodconsvcs, please investigate

Automation information below, do not change

Grafana-Automated-Alert-Id-35f560112f7a4bfabf9fd69bc1bd76fa

dotnet-eng-status[bot] avatar Apr 26 '25 22:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 83

Go to rule

dotnet-eng-status[bot] avatar Apr 27 '25 10:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 83

Go to rule

dotnet-eng-status[bot] avatar Apr 27 '25 22:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 83

Go to rule

dotnet-eng-status[bot] avatar Apr 28 '25 10:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar Apr 28 '25 22:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar Apr 29 '25 10:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar Apr 29 '25 22:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar Apr 30 '25 10:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar Apr 30 '25 22:04 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar May 01 '25 10:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar May 01 '25 22:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar May 02 '25 11:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar May 02 '25 23:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar May 03 '25 11:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar May 03 '25 23:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100

Go to rule

dotnet-eng-status[bot] avatar May 04 '25 11:05 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

Go to rule

dotnet-eng-status[bot] avatar May 04 '25 17:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 04 '25 22:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 05 '25 10:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 05 '25 22:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 06 '25 10:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 88
  • FailureRate {Machine=DNCENGWIN-143} 90

Go to rule

dotnet-eng-status[bot] avatar May 06 '25 22:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 88
  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 07 '25 10:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 84
  • FailureRate {Machine=DNCENGWIN-143} 90

Go to rule

dotnet-eng-status[bot] avatar May 07 '25 22:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 85
  • FailureRate {Machine=DNCENGWIN-133} 100
  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 08 '25 10:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100
  • FailureRate {Machine=DNCENGWIN-133} 90
  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 08 '25 23:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100
  • FailureRate {Machine=DNCENGWIN-133} 100
  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 09 '25 11:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 87
  • FailureRate {Machine=DNCENGWIN-133} 91
  • FailureRate {Machine=DNCENGWIN-143} 81

Go to rule

dotnet-eng-status[bot] avatar May 09 '25 23:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 87
  • FailureRate {Machine=DNCENGWIN-133} 91
  • FailureRate {Machine=DNCENGWIN-143} 83

Go to rule

dotnet-eng-status[bot] avatar May 10 '25 11:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100
  • FailureRate {Machine=DNCENGWIN-096} 100
  • FailureRate {Machine=DNCENGWIN-133} 100
  • FailureRate {Machine=DNCENGWIN-143} 83

Go to rule

dotnet-eng-status[bot] avatar May 10 '25 23:05 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

  • FailureRate {Machine=DNCENGWIN-020} 100
  • FailureRate {Machine=DNCENGWIN-096} 100
  • FailureRate {Machine=DNCENGWIN-133} 100
  • FailureRate {Machine=DNCENGWIN-143} 100

Go to rule

dotnet-eng-status[bot] avatar May 11 '25 11:05 dotnet-eng-status[bot]