arcade Production - [Alerting] Servicing Builds Running in R&D Pools Alert

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 372

Metric Graph

Go to rule

@dotnet/dnceng, please investigate

Automation information below, do not change

Grafana-Automated-Alert-Id-a5641c4a6d8e4e499f1710aa8386d81b

Aug 31 '22 21:08 dotnet-eng-status[bot]

This is basically all the 7.0 branches. @MattGal

Aug 31 '22 22:08 michellemcdaniel

Yes I've already been dealing with the same alert in staging; this won't be cleared until the 7.0 branches take the 7.0 arcade update once. Feel free to assign to me.

Sep 01 '22 01:09 MattGal

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Sep 04 '22 18:09 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 7

Metric Graph

Go to rule

Sep 04 '22 20:09 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Sep 18 '22 06:09 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 5

Metric Graph

Go to rule

Sep 19 '22 10:09 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Sep 26 '22 00:09 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 4

Metric Graph

Go to rule

Sep 26 '22 19:09 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Oct 02 '22 00:10 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 4

Metric Graph

Go to rule

Oct 03 '22 15:10 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Oct 08 '22 20:10 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 22

Metric Graph

Go to rule

Oct 11 '22 00:10 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Oct 17 '22 07:10 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 4

Metric Graph

Go to rule

Oct 17 '22 09:10 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Oct 23 '22 08:10 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 4

Metric Graph

Go to rule

Oct 24 '22 00:10 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Oct 26 '22 14:10 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 5

Metric Graph

Go to rule

Oct 26 '22 23:10 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Oct 27 '22 19:10 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 10

Go to rule

Oct 27 '22 22:10 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Oct 29 '22 23:10 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 2

Metric Graph

Go to rule

Oct 31 '22 11:10 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Nov 01 '22 08:11 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 2

Metric Graph

Go to rule

Nov 02 '22 01:11 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Nov 06 '22 00:11 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 2

Metric Graph

Go to rule

Nov 06 '22 01:11 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Nov 06 '22 12:11 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 2

Metric Graph

Go to rule

Nov 06 '22 13:11 dotnet-eng-status[bot]

:green_heart: Metric state changed to ok

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

Metric Graph

Go to rule

Nov 07 '22 00:11 dotnet-eng-status[bot]

:broken_heart: Metric state changed to alerting

Please run the following query to identify the delinquent pipelines:

TimelineBuilds
| where QueueTime > ago(1d)
| where SourceBranch has "release"
| join kind=inner (TimelineRecords
                    | where Order != 0
                    | where strlen(Path) == 11
                    | where WorkerName has "NetCore1ESPool"
                    | where not(WorkerName has "-Svc")
                    | summarize arg_min(WorkerName, *) by RecordId, BuildId ) on BuildId

count_ 2

Metric Graph

Go to rule

Nov 07 '22 01:11 dotnet-eng-status[bot]

arcade arcade copied to clipboard

Production - [Alerting] Servicing Builds Running in R&D Pools Alert

arcade
arcade copied to clipboard