runtime icon indicating copy to clipboard operation
runtime copied to clipboard

[apple][TvOS] Timeout when building aot-instances.dll for System.Numerics.Tensors.Tests

Open matouskozak opened this issue 1 year ago • 11 comments

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=701353 Build error leg or test failing: System.Numerics.Tensors.Tests Affected CI: tvos_arm64_release_allsubsets_mono (runtime-extra-platforms) Range of commits: https://github.com/dotnet/runtime/compare/a3bf53fc...39ecbe0e

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: Result validation: :warning: Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: :mag_right:" line. Validation performed at: 9/4/2024 4:16:18 PM UTC

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0

matouskozak avatar Jun 19 '24 10:06 matouskozak

Tagging subscribers to 'os-tvos': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

Tagging subscribers to this area: @directhex, @matouskozak See info in area-owners.md if you want to be subscribed.

It looks like it could be related to changes in https://github.com/dotnet/runtime/pull/103005. @akoeplinger do you know what needs to be done so that the test dll can be signed?

matouskozak avatar Jun 19 '24 10:06 matouskozak

There is also an ILStrip error which might be the root cause of the issue /tmp/helix/working/B82A0A2B/p/build/apple/AppleBuild.targets(262,5): error : ILStrip failed for /tmp/helix/working/B82A0A2B/w/A9580976/e/publish/System.Numerics.Tensors.dll: Object reference not set to an instance of an object.

matouskozak avatar Jun 29 '24 06:06 matouskozak

@matouskozak Is this still happening- it's has not been hit in the last week per the automated tracking....

vitek-karas avatar Jul 09 '24 13:07 vitek-karas

@matouskozak Is this still happening- it's has not been hit in the last week per the automated tracking....

Yes, still failing, not sure why the failure message changed. Example of recent failing build https://dev.azure.com/dnceng-public/public/_build/results?buildId=732443&view=logs&j=acac6bf3-3edd-5022-4e45-3362a02520b1&t=1de8bf71-8fe5-5d5c-4360-8c2eb159d012. The test is timing out currently ERROR: WORKLOAD TIMED OUT - Killing user command.. thus doesn't have any useful logs to match against.

matouskozak avatar Jul 09 '24 14:07 matouskozak

removing blocking-clean-ci as it has not failed in 30 days

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0

jeffschwMSFT avatar Aug 22 '24 16:08 jeffschwMSFT

removing blocking-clean-ci as it has not failed in 30 days

24-Hour Hit Count 7-Day Hit Count 1-Month Count 0 0 0

It's a mobile failure and we don't have a good way to match it for Build Analysis yet. It is still blocking clean extra-platforms CI: image

matouskozak avatar Aug 22 '24 17:08 matouskozak

This failure is actually much more widespread than just Tensors. I'll edit the json in the description so all the failures are captured.

Here is an example:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-107334-merge-c1d33defa49442c09e/System.Runtime.Tests/1/console.a3d21fde.log?helixlogtype=result

No embedded provisioning profile found at /tmp/helix/working/B06108DF/w/B36209B2/e/System.Runtime.Tests.app/embedded.mobileprovision! Failed to sign the app!

It is affecting at least ios-arm64 and tvos-arm64.

carlossanlop avatar Sep 04 '24 15:09 carlossanlop

Tagging subscribers to 'os-ios': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

This failure is actually much more widespread than just Tensors. I'll edit the json in the description so all the failures are captured.

Here is an example:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-107334-merge-c1d33defa49442c09e/System.Runtime.Tests/1/console.a3d21fde.log?helixlogtype=result

No embedded provisioning profile found at /tmp/helix/working/B06108DF/w/B36209B2/e/System.Runtime.Tests.app/embedded.mobileprovision! Failed to sign the app!

It is affecting at least ios-arm64 and tvos-arm64.

@carlossanlop the embedded.mobileprovision! Failed to sign the app! is actually a generic failure message.

The actual failure actually is error MSB3030: Could not copy the file "/tmp/helix/working/BE410A2F/p/build/microsoft.netcore.app.runtime.ios-arm64/runtimes/ios-arm64/native/System.Runtime.Serialization.Formatters.pdb" because it was not found. and is tracked in https://github.com/dotnet/runtime/issues/107309 (not sure why it didn't get matched).

I'll change this back because we actually have a timeout issue when running dedup on Tensors dll.

matouskozak avatar Sep 04 '24 15:09 matouskozak

resolved in https://github.com/dotnet/runtime/pull/104097

matouskozak avatar Dec 27 '24 08:12 matouskozak