runtime icon indicating copy to clipboard operation
runtime copied to clipboard

Long running test: SendMailAsync_CanBeCanceled_CancellationToken

Open karelz opened this issue 3 years ago • 2 comments

Long running test:

  • SendMailAsync_CanBeCanceled_CancellationToken - 'System.Net.Mail.Tests.SmtpClientTest.SendMailAsync_CanBeCanceled_CancellationToken'

Regression on 7/18

  • Test disabled on 8/4 for Mono only in PR #73340
  • Test disabled on 8/5 for all Unix platforms in PR #73452

Failures 6/6-8/5 (incl. PRs):

  • x64 is default architecture
  • CoreCLR is default runtime
Date Run Notes
8/5 1x PR Centos 8
8/3 1x Official run Debian 10 - Mono
8/2 2x PR Debian 10 - Mono
OpenSUSE 15.2
~8/2~ ~1x PR~ ~Windows 8.1~ - Unrelated infra problem
8/1 1x PR Centos 8
7/30 1x PR OSX 12.00 - Mono
7/29 1x PR Debian 10 - Mono
~7/29~ ~1x PR~ ~Windows 10 - Windows Nano 1809 - amd64~ - Unrelated infra timeout
7/28 1x Official run Debian 10 - amd64 - Mono
7/27 3x (1x Official run) 2x Centos 8
Debian 10 - Mono
7/25 1x Official run Centos 7
7/22 2x (1x Official run) 2x openSUSE 15.2
7/21 1x PR Debian 10 - Mono
7/20 1x PR Alpine 314 - Arm32
7/18 1x Official run Fedora 34
let timeouts = (friendlyNamePrefix : string, includePR : bool) {
cluster('engsrvprod.kusto.windows.net').database('engineeringdata').WorkItems
| where Status == "Timeout"
| where FriendlyName startswith friendlyNamePrefix
//| where Finished > datetime('2021-09-07')
| distinct JobId, WorkItemId, Name, FriendlyName, ConsoleUri
| join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
    | where ((Branch == 'refs/heads/main') or (Branch == 'refs/heads/master') or (includePR and (Source startswith "pr/")) or (Branch startswith 'refs/heads/release/6.0'))
    | where Type startswith "test/functional/cli/"
        and not(Properties contains "runtime-staging")
    | summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName) by JobId
| project-rename JobType = Type) on JobId
| extend PropertiesJson = parse_json(Properties)
| extend OS = replace_regex(tostring(PropertiesJson.operatingSystem), @'\((.*)\).*|([^\(].*)', @'\1\2')
| extend Runtime = iif(PropertiesJson.runtimeFlavor == "mono", "Mono", iif(PropertiesJson.DefinitionName contains "coreclr", "CoreCLR", ""))
| extend TargetBranch = extractjson("$.['System.PullRequest.TargetBranch']", Properties)
| extend Architecture = PropertiesJson.architecture
| extend Scenario = iif(isempty(PropertiesJson.scenario), "--", PropertiesJson.scenario)
| project-away PropertiesJson
| order by Finished desc
};
timeouts('System.Net.Mail.Functional.Tests', true);

karelz avatar Aug 05 '22 09:08 karelz

Tagging subscribers to this area: @dotnet/ncl See info in area-owners.md if you want to be subscribed.

Issue Details

Long running test:

  • SendMailAsync_CanBeCanceled_CancellationToken - 'System.Net.Mail.Tests.SmtpClientTest.SendMailAsync_CanBeCanceled_CancellationToken'

Regression on 7/18 Disabled on 8/4 in PR #73340

Failures 6/6-8/5 (incl. PRs):

Date Run Notes
8/1 1x PR Ubuntu 16.04 amd64
7/30 1x PR OSX 12.00 amd64
7/29 1x PR Ubuntu 18.04 amd64
~7/29~ ~1x PR~ ~Windows10~ - Helix infra timeout
7/28 1x Official run Ubuntu 18.04 amd64
7/27 3x (1x Official run) 2x Ubuntu 16.04 amd64 + 1x Ubuntu 18.04 amd64
7/25 1x Official run Ubuntu 16.04 amd64
7/22 2x (1x Official run) 2x Ubuntu 16.04 amd64
7/21 1x PR Ubuntu 18.04 amd64
7/20 1x PR Ubuntu 18.04 arm
7/18 1x Official run Ubuntu 16.04 amd64
let timeouts = (includePR : bool) {
cluster('engsrvprod.kusto.windows.net').database('engineeringdata').WorkItems
| where Status == "Timeout"
| where FriendlyName startswith "System.Net.Mail.Functional.Tests"
//| where Finished > datetime('2021-09-07')
| distinct JobId, WorkItemId, Name, FriendlyName, ConsoleUri
| join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
    | where ((Branch == 'refs/heads/main') or (Branch == 'refs/heads/master') or (includePR and (Source startswith "pr/")) or (Branch startswith 'refs/heads/release/6.0'))
    | where Type startswith "test/functional/cli/"
        and not(Properties contains "runtime-staging")
    | summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName) by JobId
| project-rename JobType = Type) on JobId
| order by Finished desc
};
timeouts(true);
Author: karelz
Assignees: -
Labels:

area-System.Net, disabled-test, os-linux, os-mac-os-x

Milestone: 7.0.0

msftbot[bot] avatar Aug 05 '22 09:08 msftbot[bot]

Test disabled in PR #73452

karelz avatar Aug 07 '22 14:08 karelz

Update: We are re-enabling the test in main via PR #74545 with changes that will hopefully give us more info on the root cause.

karelz avatar Aug 25 '22 16:08 karelz

@wfurt please check Kusto and if there is no new failure in last 5 days (since the test was re-enabled), let's close it as done (for 8.0). Thanks!

karelz avatar Aug 30 '22 16:08 karelz

Happened once on main in last two weeks https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-1ec4a131ebe140708d/System.Net.Mail.Functional.Tests/1/console.bd6b838c.log?helixlogtype=result

wfurt avatar Aug 31 '22 17:08 wfurt

More hits updated in the top post

karelz avatar Sep 12 '22 22:09 karelz

@rzikm can you please check if this was fixed by #76361 or if we still see failures? It is 18 days so it should be probably long enough period to close this if we don't see occurrences.

wfurt avatar Nov 08 '22 18:11 wfurt

No hits (of either assert or long running test) in last 2 weeks, so I think it is safe to close this as fixed.

rzikm avatar Nov 09 '22 09:11 rzikm