opentelemetry-dotnet icon indicating copy to clipboard operation
opentelemetry-dotnet copied to clipboard

Published 20 hours ago • verified publisher icon open-telemetry

API Check CI reliability

Open cijothomas opened this issue 3 years ago • 2 comments

Fails quite often: Example: https://pipelines.actions.githubusercontent.com/qUmkdRTOExOOUcipfw2r74VakBMGn99kKwI1fYYZyOCfUDlpFN/_apis/pipelines/1/runs/53945/signedlogcontent/2?urlExpires=2022-02-10T19%3A56%3A18.4295005Z&urlSigningMethod=HMACV1&urlSignature=FmvRNYPK%2B53JnQ3Ii0dU72ekKuzMNGqn%2Btsxw%2BImNWg%3D

2022-02-10T19:07:04.4043985Z CheckAPICompatibility: true 2022-02-10T19:07:04.4044253Z DOTNET_MULTILEVEL_LOOKUP: 1 2022-02-10T19:07:04.4044529Z DOTNET_ROOT: C:\Users\runneradmin\AppData\Local\Microsoft\dotnet 2022-02-10T19:07:04.4044820Z ##[endgroup] 2022-02-10T19:07:05.1622185Z Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET 2022-02-10T19:07:05.1629900Z Copyright (C) Microsoft Corporation. All rights reserved. 2022-02-10T19:07:05.1630434Z 2022-02-10T19:07:08.5244268Z Working directory for previous package versions not found, creating... 2022-02-10T19:07:08.6203888Z Working directory for previous package versions not found, creating... 2022-02-10T19:07:08.7768214Z Retrieving OpenTelemetry @1.1.0 for compatibility check 2022-02-10T19:07:08.7815848Z New-Item : An item with the specified name D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\LastMajorVersionBinaries 2022-02-10T19:07:08.7848826Z already exists. 2022-02-10T19:07:08.7888309Z At D:\a\opentelemetry-dotnet\opentelemetry-dotnet\build\PreBuild.ps1:7 char:5 2022-02-10T19:07:08.7895910Z + New-Item -Path $workDir -ItemType "directory" | Out-Null 2022-02-10T19:07:08.7942195Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-02-10T19:07:08.7948715Z + CategoryInfo : ResourceExists: (D:\a\openteleme...VersionBinaries:String) [New-Item], IOException 2022-02-10T19:07:08.8264845Z + FullyQualifiedErrorId : DirectoryExist,Microsoft.PowerShell.Commands.NewItemCommand 2022-02-10T19:07:08.8278338Z Retrieving OpenTelemetry @1.1.0 for compatibility check 2022-02-10T19:07:08.8282139Z
2022-02-10T19:07:09.7086572Z Invoke-WebRequest : The process cannot access the file 2022-02-10T19:07:09.7090164Z 'D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\LastMajorVersionBinaries\OpenTelemetry.1.1.0.zip' because it is 2022-02-10T19:07:09.7093559Z being used by another process. 2022-02-10T19:07:09.7096556Z At D:\a\opentelemetry-dotnet\opentelemetry-dotnet\build\PreBuild.ps1:17 char:5 2022-02-10T19:07:09.7099773Z + Invoke-WebRequest -Uri https://www.nuget.org/api/v2/package/$pack ... 2022-02-10T19:07:09.7102554Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-02-10T19:07:09.7105299Z + CategoryInfo : NotSpecified: (:) [Invoke-WebRequest], IOException 2022-02-10T19:07:09.7375896Z + FullyQualifiedErrorId : System.IO.IOException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand 2022-02-10T19:07:09.7444843Z
2022-02-10T19:07:10.0815956Z New-Item : An item with the specified name 2022-02-10T19:07:10.0873578Z D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\LastMajorVersionBinaries\OpenTelemetry\1.1.0 already exists. 2022-02-10T19:07:10.0902256Z At C:\program files\powershell\7\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:325 char:32 2022-02-10T19:07:10.0909439Z + ... eatedItem = New-Item -Path $DestinationPath -ItemType Directory -Conf ... 2022-02-10T19:07:10.0910356Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-02-10T19:07:10.0911064Z + CategoryInfo : ResourceExists: (D:\a\openteleme...Telemetry\1.1.0:String) [New-Item], IOException 2022-02-10T19:07:10.0911948Z + FullyQualifiedErrorId : DirectoryExist,Microsoft.PowerShell.Commands.NewItemCommand 2022-02-10T19:07:10.0912637Z
2022-02-10T19:07:10.2051549Z D:\a\opentelemetry-dotnet\opentelemetry-dotnet\build\Common.prod.props(20,5): error MSB3073: The command "powershell -ExecutionPolicy Unrestricted -File "D:\a\opentelemetry-dotnet\opentelemetry-dotnet\build\PreBuild.ps1" -package OpenTelemetry -version "1.1.0"" exited with code 1. [D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\OpenTelemetry\OpenTelemetry.csproj] 2022-02-10T19:07:11.8604801Z Retrieving OpenTelemetry.Exporter.Zipkin @1.1.0 for compatibility check 2022-02-10T19:07:11.8655545Z Retrieving OpenTelemetry.Api @1.1.0 for compatibility check 2022-02-10T19:07:18.7843170Z Retrieving OpenTelemetry.Exporter.Jaeger @1.1.0 for compatibility check 2022-02-10T19:07:22.2945746Z Retrieving OpenTelemetry.Exporter.OpenTelemetryProtocol @1.1.0 for compatibility check 2022-02-10T19:07:26.7342995Z OpenTelemetry.Api -> D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\OpenTelemetry.Api\bin\Release\net461\OpenTelemetry.Api.dll 2022-02-10T19:07:33.3701675Z Retrieving OpenTelemetry.Exporter.Console @1.1.0 for compatibility check 2022-02-10T19:07:41.3577617Z OpenTelemetry -> D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\OpenTelemetry\bin\Release\net461\OpenTelemetry.dll 2022-02-10T19:07:42.9345991Z Retrieving OpenTelemetry.Exporter.Prometheus @1.1.0 for compatibility check 2022-02-10T19:07:45.7387491Z OpenTelemetry.Api -> D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\OpenTelemetry.Api\bin\Release\netstandard2.0\OpenTelemetry.Api.dll 2022-02-10T19:07:46.1343591Z Retrieving OpenTelemetry.Exporter.InMemory @1.1.0 for compatibility check 2022-02-10T19:07:49.1177499Z Invoke-WebRequest : BlobNotFoundThe specified blob does not exist. RequestId:e2cb7479-e01e-0083-5fb1-1e4329000000 2022-02-10T19:07:49.1205115Z Time:2022-02-10T19:07:43.3934515Z 2022-02-10T19:07:49.1227862Z At D:\a\opentelemetry-dotnet\opentelemetry-dotnet\build\PreBuild.ps1:17 char:5 2022-02-10T19:07:49.1238252Z + Invoke-WebRequest -Uri https://www.nuget.org/api/v2/package/$pack ... 2022-02-10T19:07:49.1254978Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-02-10T19:07:49.1488228Z + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc 2022-02-10T19:07:49.1525702Z eption 2022-02-10T19:07:49.1529507Z + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand 2022-02-10T19:07:49.1531475Z Expand-Archive : The path '..\LastMajorVersionBinaries\OpenTelemetry.Exporter.Prometheus.1.1.0.zip' either does not 2022-02-10T19:07:49.1551859Z exist or is not a valid file system path. 2022-02-10T19:07:49.1612106Z At D:\a\opentelemetry-dotnet\opentelemetry-dotnet\build\PreBuild.ps1:25 char:5 2022-02-10T19:07:49.1643826Z + Expand-Archive -LiteralPath "$workDir$package.$version.zip" -Des ... 2022-02-10T19:07:49.1667252Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-02-10T19:07:49.1747358Z + CategoryInfo : InvalidArgument: (..\LastMajorVer...theus.1.1.0.zip:String) [Expand-Archive], InvalidOpe 2022-02-10T19:07:49.1791832Z rationException 2022-02-10T19:07:49.1797650Z + FullyQualifiedErrorId : ArchiveCmdletPathNotFound,Expand-Archive

cijothomas avatar Feb 10 '22 19:02 cijothomas

Workflow definition: https://github.com/open-telemetry/opentelemetry-dotnet/blob/00af7817fdba22b2a5948e574c6769098003f682/.github/workflows/apicompatibility.yml#L13

Common.prod.props: https://github.com/open-telemetry/opentelemetry-dotnet/blob/252437d1ea37bb1a9f960a2941ab922ea5cc23d2/build/Common.prod.props#L19-L21

I see three different errors in the log:

  • New-Item

    New-Item : An item with the specified name D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\LastMajorVersionBinaries already exists.

    https://github.com/open-telemetry/opentelemetry-dotnet/blob/8ca6574599cf412d732bac3fb46c515c746f15b8/build/PreBuild.ps1#L3-L8

    I think the Common.prop.props is getting run for multiple projects and hitting a race condition. I think this can be mitigated with a try/catch, log to output, and continue.

  • Invoke-WebRequest

    Invoke-WebRequest : The process cannot access the file 'D:\a\opentelemetry-dotnet\opentelemetry-dotnet\src\LastMajorVersionBinaries\OpenTelemetry.1.1.0.zip' because it is being used by another process.

    https://github.com/open-telemetry/opentelemetry-dotnet/blob/8ca6574599cf412d732bac3fb46c515c746f15b8/build/PreBuild.ps1#L10-L18

    Is it possible that two projects would download the same package?

  • Expand-Archive

    Expand-Archive : The path '..\LastMajorVersionBinaries\OpenTelemetry.Exporter.Prometheus.1.1.0.zip' either does not exist or is not a valid file system path.

    https://github.com/open-telemetry/opentelemetry-dotnet/blob/8ca6574599cf412d732bac3fb46c515c746f15b8/build/PreBuild.ps1#L19-L26

    May be able to mitigate this by moving the Expand-Archive into the same block as Inovke-WebRequest.

TimothyMothra avatar Mar 11 '22 22:03 TimothyMothra

Need to move away from this to the recommended way: https://github.com/open-telemetry/opentelemetry-dotnet/issues/2081

cijothomas avatar May 10 '22 19:05 cijothomas