[wasm][AOT] emcc : error - received SIGKILL (-9)

[pavelsavara]

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=653707

Error Blob

{
  "BuildRetry": false,
  "ErrorPattern": ["emcc", "SIGKILL"],
  "ExcludeConsoleLog": false
}

Reproduction Steps

AOT wasm Microsoft.Extensions.Options.SourceGeneration.Unit.Tests

Log

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=653707 Error message validated: [emcc SIGKILL] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 4/25/2024 7:29:32 AM UTC

Report

Build	Definition	Test	Pull Request
717715	dotnet/runtime	WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution
716370	dotnet/runtime	WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution	dotnet/runtime#103844
713254	dotnet/runtime	WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution	dotnet/runtime#103617
707058	dotnet/runtime	normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
706174	dotnet/runtime	WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution	dotnet/runtime#103392
702836	dotnet/runtime	normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
697906	dotnet/runtime	WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution	dotnet/runtime#102967

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	1	7

[ghost]

Tagging subscribers to 'arch-wasm': @lewing See info in area-owners.md if you want to be subscribed.

Issue Details

---

Error Blob

{
  "BuildRetry": false,
  "ErrorPattern": "emcc.*SIGKILL",
  "ExcludeConsoleLog": true
}

Reproduction Steps

AOT wasm Microsoft.Extensions.Options.SourceGeneration.Unit.Tests

Log

Author:	pavelsavara
Assignees:	-
Labels:	arch-wasm, blocking-clean-ci, area-Codegen-AOT-mono, wasm-aot-test, Known Build Error, os-browser
Milestone:	8.0.0

[lewing]

Is this it hitting the ulimit of the container?

[radical]

[ 7455.672962] [ 2825] 1000 2825 158937 41558 684032 0 0 python3 
[ 7455.672964] [ 4160] 102 4160 67335 453 180224 0 0 rsyslogd 
[ 7455.672966] [ 4181] 0 4181 455864 16951 634880 0 0 amacoreagent 
[ 7455.672968] [ 4453] 0 4453 298556 3977 249856 0 0 auoms 
[ 7455.672969] [ 4495] 0 4495 215902 6240 204800 0 0 auomscollect 
[ 7455.672971] [ 4750] 102 4750 191707 14176 622592 0 0 mdsd 
[ 7455.672973] [ 4967] 0 4967 334103 3030 253952 0 0 azsecd 
[ 7455.672975] [ 5046] 0 5046 352600 4193 270336 0 0 azsecmond 
[ 7455.672977] [ 32457] 0 32457 317321 3577 258048 0 0 azsecd 
[ 7455.672979] [ 2226] 0 2226 376486 334278 2924544 0 0 clamscan 
[ 7455.672981] [ 22067] 0 22067 180189 824 114688 0 -998 containerd-shim 
[ 7455.672983] [ 22089] 1000 22089 1159 16 57344 0 0 helix_docker_wo 
[ 7455.672985] [ 22156] 1000 22156 1159 17 53248 0 0 execute.sh 
[ 7455.672987] [ 22212] 1000 22212 5047 87 81920 0 0 bash 
[ 7455.672989] [ 22215] 1000 22215 68517493 67031 1163264 0 0 dotnet 
[ 7455.672990] [ 23320] 1000 23320 1159 17 61440 0 0 sh 
[ 7455.672992] [ 23321] 1000 23321 34332 22818 335872 0 0 python3 
[ 7455.672994] [ 23345] 1000 23345 1420573 1386396 11264000 0 0 wasm-opt 
[ 7455.672995] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/docker/a2a4080691301bb74e3b282eab0b6f638e9d6f1227e62967b7fd46654706e554,task=wasm-opt,pid=23345,uid=1000 
[ 7455.673018] Out of memory: Killed process 23345 (wasm-opt) total-vm:5682292kB, anon-rss:5545584kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:11000kB oom_score_adj:0

[lewing]

lets disable these tests in aot for now

[lewing]

closting as a duplicate of https://github.com/dotnet/dnceng/issues/450

[pavelsavara]

I see this again https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-99833-merge-af0760c542d34f95a8/normal-System.Runtime.Tests/1/console.f849dad3.log?helixlogtype=result

[lewing]

Analysis claims there are recent failures but the links are not updated?

[pavelsavara]

we run most of those problematic cases only in runtime-wasm pipeline. Which needs to be triggered manually. I updated the filter