sdk icon indicating copy to clipboard operation
sdk copied to clipboard

GenerateDepsFile: The process cannot access the file '...\MyProject.deps.json' because it is being used by another process.

Open LordMike opened this issue 5 years ago • 19 comments

UPDATE: While writing this, rather long, post -- I've managed to produce a small reproduction example, see bottom

We've recently upgraded our builds to run on faster hardware with more cpu cores, and are now ~100% of the time seeing the below error in a number of repositories. The error here is from Windows, but our CI servers run on linux and have the same error (see logs below).

We run the following commandlines:

"C:\Program Files\dotnet\dotnet.exe" restore C:\Project\MyProject.sln "C:\Program Files\dotnet\dotnet.exe" build C:\Project\MyProject.sln --configuration Debug --framework netstandard2.0 --no-restore

C:\Program Files\dotnet\sdk\2.2.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(129,5): error MSB4018

The "GenerateDepsFile" task failed unexpectedly. [C:\Project\MyProject.csproj]
System.IO.IOException: The process cannot access the file 'C:\Project\bin\Debug\netstandard2.0\MyProject.deps.json' because it is being used by another process. [C:\Project\MyProject.csproj]
   at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle) [C:\Project\MyProject.csproj]
   at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options) [C:\Project\MyProject.csproj]
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [C:\Project\MyProject.csproj]
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize) [C:\Project\MyProject.csproj]
   at System.IO.File.Create(String path) [C:\Project\MyProject.csproj]
   at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [C:\Project\MyProject.csproj]
   at Microsoft.NET.Build.Tasks.TaskBase.Execute() [C:\Project\MyProject.csproj]
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [C:\Project\MyProject.csproj]
   at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [C:\Project\MyProject.csproj]

Observations:

  • This fails on my local Windows desktop roughly 33% of the time. Rerunning the build (no cleanup) always succeeds (seems to be a race condition, so HW speeds play a role). On our Linux CI servers, it fails 100% of the time currently
  • The deps file in question is always the same file for a given repository / builds. The file is for a shared library between multiple projects.
  • A plain dotnet build MyProject.sln -f netstandard2.0 (no build script) also exhibits this issue.
  • A dotnet build MyProject.sln seemingly always succeeds (locally and on CI server), at least for three consecutive attempts right now.
  • All the projects in the MyProject.sln have netstandard2.0 - but a number of them have more targets (ie. netcoreapp)
  • ~We have more projects, f.ex. for tests, but they're in MyProject-Tests.sln. These projects only target netcoreapp2.x - we've split the solutions both for speed when developing and to be able to build artifacts without compiling the tests (since we can't build certain projects from the solution only ... that's another issue entirely)~ (not relevant)
  • Limiting the build to one process (msbuild: /m:1) seems to "solve" the problem.
  • At work (inaccessible currently), I performed some Procmon captures to see the difference between successful and failing runs
    • It turns out, that two dotnet.exe processes attempt to create the .deps.json files
    • When succeeding, there is a clear seperation between two CreateFile() calls, where the first creates the file, and the second call (from the second PID) finds the file already there and does nothing (not even reads it)
    • When failing, the two processes make calls overlapping each other, leading to both of them discovering the file as missing, and both trying to create it (one obviously failing, producing the stacktrace above) -- classical concurrency issue.

So.

  • It should be that when two projects reference a third, shared, project .. that this shared one is built exactly once .. right?.. How come two attempts are made at writing the same .deps.json files?
  • How come this seemingly works fine, when building for all target frameworks?

Versions:

  • Local, Windows 10 x64, dotnet 2.2.101
  • Linux CI, docker, dotnet 2.2.103 (we build our own docker images using the dockerfile from the base microsoft/dotnet:2-sdk images)

Reproduction

Project: ci-stresstest-master.zip

In it, I have a solution with 5 projects, all empty (no actual code), but with a targeting setup that mimicks a very small subset of our projects. I've then created four seperate CI builds that did the following:

  • log (succeeds) dotnet restore ci-stresstest.sln + dotnet build ci-stresstest.sln --no-restore
  • log (succeeds) dotnet build ci-stresstest.sln
  • log (fails) dotnet restore ci-stresstest.sln + dotnet build ci-stresstest.sln -f netstandard2.0 --no-restore
  • log (fails) dotnet build ci-stresstest.sln -f netstandard2.0
Notes on repro.
  • We run a vanilla docker setup for CI, except for one thing:
    • We mount a shared directory for nuget packages, to be able to share downloaded packages between builds

See also #2076

LordMike avatar Feb 03 '19 21:02 LordMike

While trying to reproduce a failure on Windows with Procmon, I'm met with this. I've noticed sporadic errors like this before, but never enough of them to make an impression. Given the above, I think there might be a lot of concurrency issues in msbuild/dotnet build ..

Other locking/access errors I've seen:

  • A copy file to output directory (from library to referencing project) which is retried a few seconds later (this was "just" a warning) (from memory --no pictures / logs)

The process cannot access the file '...\Shared02.dll' because it is being used by another process.

  • Cleaning the workspace (git clean -xdf) between builds produces this roughly once every two builds.

image

Log

PS C:\Users\MichaelBisbjerg\Git\ci-stresstest> dotnet build -f netstandard2.0
Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\Program01.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\TopLib02.csproj...
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\obj\TopLib02.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\TopLib01.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Shared01.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Shared02.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\obj\Program01.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Shared01.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\obj\Program01.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Shared02.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\obj\TopLib02.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\TopLib01.csproj.nuget.g.targets.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\TopLib02.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\Program01.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj.
  Shared01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\bin\Debug\netstandard2.0\Shared01.dll
CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj]
CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj]
  Shared02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\bin\Debug\netstandard2.0\Shared02.dll
  TopLib02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\bin\Debug\netstandard2.0\TopLib02.dll
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  Program01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\bin\Debug\netstandard2.0\Program01.dll

Build FAILED.

CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj]
CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj]
    0 Warning(s)
    2 Error(s)

Time Elapsed 00:00:01.30

Could not read state file "obj\..\TopLib01.csprojAssemblyReference.cache". The process cannot access the file '..\TopLib01.csprojAssemblyReference.cache' because it is being used by another process.

  • Building the solution without cleanup between builds, this happened 8 out of 17 times I built it.

image

Log

PS C:\Users\MichaelBisbjerg\Git\ci-stresstest> dotnet build -f netstandard2.0
Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 25,24 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\TopLib02.csproj.
  Restore completed in 25,22 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj.
  Restore completed in 25,19 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj.
  Restore completed in 25,24 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\Program01.csproj.
  Restore completed in 25,19 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj.
  Shared02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\bin\Debug\netstandard2.0\Shared02.dll
  Shared02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\bin\Debug\netstandard2.0\Shared02.dll
  Shared01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\bin\Debug\netstandard2.0\Shared01.dll
C:\Program Files\dotnet\sdk\2.2.101\Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3088: Could not read state file "obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache". The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache' because it is being used by another process. [C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj]
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  TopLib02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\bin\Debug\netstandard2.0\TopLib02.dll
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  Shared01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\bin\Debug\netstandard2.0\Shared01.dll
  Program01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\bin\Debug\netstandard2.0\Program01.dll

Build succeeded.

C:\Program Files\dotnet\sdk\2.2.101\Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3088: Could not read state file "obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache". The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache' because it is being used by another process. [C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj]
    1 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.41

LordMike avatar Feb 03 '19 21:02 LordMike

I have the same problem. Any solutions?

The core problem here is that you're specifying a TargetFramework at the solution level. As you note, it works without that, while building all of the TargetFrameworks specified at the individual project level.

We hope to improve the experience around this -- at least providing an informative error, but maybe also doing something smarter. But at the moment, I'm not quite sure what that would be.

Can you describe why you're specifying a TargetFramework and what you expect to happen in that build, as distinct from the no-TF-specified solution build?

rainersigwald avatar Feb 05 '19 18:02 rainersigwald

For our current use case, we wanted to optimize our builds a bit, because we're targeting two frameworks to allow our developers to test the modules they're working (they have a small program.cs, and a shared library to actually run code -- only available to them when in netcoreapp2.x).

So our builds on the CI would be building for both coreapp and netstandard. Targeting just netstandard (since everything in the slution supports that anyways) did this and shaved a few seconds of our builds.

We've workarounded by not targeting the specific framework, since we found that it was just a few seconds..

However. I'm not at all sure why this is an issue.. All of the projects in question target netstandard2.0..

LordMike avatar Feb 05 '19 18:02 LordMike

Hi,

I've a case where there is no specifying of TargetFramework at the solution level where the dotnet build behaves similar in some cases where TargetFramework at the project level refers to netstandard2.0.

This behavior has just emerged without any notice in dotnet core 2.0. I've also tried to upgrade to dotnet core 2.1, but in vein.

In my case, there seems to be a race condition from my hardware setup, due to the dotnet build leaves few spawned dotnet processes.

Thank you.

Regards Henrik

HenrikBach1 avatar Feb 20 '19 06:02 HenrikBach1

Note that the spawned processes could be the Dotnet Build Server. You can try shutting them down by running dotnet build-server shutdown. If its that, they shouldn't be a problem (they're meant to speed up future builds). I have an anecdotal feeling that we're worse of for having the build server, but so far it's just that.

LordMike avatar Feb 20 '19 09:02 LordMike

Two out of three of them shut down with the dotnet build-server shutdown command:

myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ ps
  PID TTY          TIME CMD
   31 pts/1    00:00:00 bash
10834 pts/1    00:00:04 dotnet
10850 pts/1    00:00:04 dotnet
10866 pts/1    00:00:06 dotnet
11617 pts/1    00:00:00 ps
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ dotnet build-server shutdown
Shutting down MSBuild server...
Shutting down VB/C# compiler server...
VB/C# compiler server shut down successfully.
MSBuild server shut down successfully.
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ ps
  PID TTY          TIME CMD
   31 pts/1    00:00:00 bash
10866 pts/1    00:00:07 dotnet
11668 pts/1    00:00:00 ps
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ ps 10866
  PID TTY      STAT   TIME COMMAND
10866 pts/1    SLl    0:07 /usr/share/dotnet/dotnet /usr/share/dotnet/sdk/2.2.102/MSBuild.dll /usr/share/dotnet/sdk/2.2.102/MSBuild.dll /nologo /nodemode:1 /nodeReuse:true
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ 

HenrikBach1 avatar Feb 20 '19 10:02 HenrikBach1

That does seem like one of the build processes from the parallel build (the /nodeReuse:true option). That, I have no good answer for.

LordMike avatar Feb 20 '19 12:02 LordMike

Ok.

Problem solved by provisioning a new virtual machine.

HenrikBach1 avatar Feb 22 '19 04:02 HenrikBach1

We've run into another similar (but not the exact same) race condition:

/usr/share/dotnet/sdk/2.2.103/Microsoft.Common.CurrentVersion.targets(2110,5):
error MSB3248: Parameter "AssemblyFiles" has invalid value
	"/build/Engine/Server/bin/Debug/netcoreapp2.2/Hydra.Engine.Server.dll".
The process cannot access the file
	'/build/Engine/Server/bin/Debug/netcoreapp2.2/Hydra.Engine.Server.dll'
because it is being used by another process. [/build/Server/Map/Hydra.Map.csproj]

I'm not sure if this is caused by a bug in the build system or if there is somehow a conflict in our projects that could be resolved by fixing the project definition. We use standard project layouts with simple SDK style csproj projects.

In the latter case, it would be amazing if the build system could detect the sources of these conflicts and warn or error on them.

anttirt avatar Mar 25 '19 10:03 anttirt

The core problem here is that you're specifying a TargetFramework at the solution level. As you note, it works without that, while building all of the TargetFrameworks specified at the individual project level.

I am getting this error if I simply call dotnet build Main.sln without specifying the target framework. So I can't believe in this "core problem" explanation.

victor-yarema avatar Jun 13 '19 09:06 victor-yarema

Currently I am thinking about workaround to simply change build script part from something like

dotnet build Main.sln &&
echo 'Success'

To something like

{
  dotnet build Main.sln ||
  dotnet build Main.sln
} &&
echo 'Success'

There can be more tries if needed. I already did that trick with MSBuild in year 2013 or 2014 when we were unable to get reliable build of website project (ASP.NET). We had 4 tries back then because 2 tries were not enough. P.S. The problem with website project was different.

victor-yarema avatar Jun 13 '19 10:06 victor-yarema

@victor-yarema Can you please file a new issue for that? Please tag me. Because you're not specifying global properties at the solution level, you're seeing a different problem.

If you can attach a binary log of your build (a successful build log is fine), hopefully we can figure out what's going on. Please read the warnings about the information contained in the log. If you cannot upload such a log, please file the bug anyway. I can suggest additional debugging techniques.

rainersigwald avatar Jun 13 '19 14:06 rainersigwald

FWIW, I've experienced similar issue with GenerateDepsFile phase using .net sdk 3.1.301 - in my case one way to mitigate the issue is to explicitly build the project in question first (using dotnet build) before building projects that depend on it.

theimowski avatar Jul 15 '20 13:07 theimowski

  1. I see builds fail with p5 because of an issue similar to this
  2. our pattern is to start several processes with
  3. dotnet tool restore
  4. then dotnet ef database update --no-build --verbose --context AppDbContext --project "SomeProjectThatMigratesSchemaLocally.csproj"
  5. then dotnet run --no-restore --project ./src/SomeProject.Startup *> console.log
  6. then execute an e2e test to see all is good
  7. we have custom snappy linux azure vm agents with azure yaml pipelines
  8. don't have any lead as to root cause

mercer avatar Dec 31 '20 16:12 mercer

Encountered this build-error on a project of ours. We build our .Net Core solutions using a FAKE-script and that script failed while building a solution. A manual build of the solution with dotnet build worked fine.

Turns out FAKE's FAKE.DotNet.MSBuild module adds the parameter -maxcpucount to the command dotnet.exe build as a parameter without a value.

From the build-log we see the maxcpucount parameter being used:

[...]
C:\Program Files\dotnet\sdk\5.0.104\MSBuild.dll /nologo - \
distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,C:\Program \
Files\dotnet\sdk\5.0.104\dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,C:\Program \
Files\dotnet\sdk\5.0.104\dotnet.dll -maxcpucount -property:Configuration=Release -target:Restore -verbosity:m /bl: [...]  
Determining projects to restore...
[...]

From the documentation of MSBuild about when no default value of maxcpucount is provided:

If you include this switch without specifying a value, MSBuild will use up to the number of processors in the computer

Which explains our issue of getting errors when building via FAKE but not via manual dotnet build.

Our fix was to add MaxCpuCount = Some(Some 1) to our script's MSBuildParams to ensure no parallelization and thus preventing the read-locks on the files during a build.

HenrikWM avatar May 04 '21 09:05 HenrikWM

I was bugged by this problem for a while. Took ages to spot the cause and resulted in having to build things twice if it failed. It turned out that one projects that referenced the project that always reported the error only had TargetFramework netstandard2.0, whilst all the others were both net5.0 & netstandard2.0. Once I switched that project to <TargetFrameworks>net5.0;netstandard2.0</TargetFrameworks> the problem went away.

Ian1971 avatar Aug 13 '21 09:08 Ian1971

I'm currently facing this issue with the latest net6.0 sdk in the following build scenario:

  • Solution with one "netstandard2.1;MonoAndroid10.0" (MSBuild.Sdk.Extras/3.0.22) and two "netstandard2.1" (Microsoft.NET.Sdk) projects
  • The "netstandard2.1;MonoAndroid10.0" project references both netstandard-only projects
  • Build using dotnet build xxx.sln /p:Configuration=Debug --framework netstandard2.1

The error occurs (on the first netstandard-only project) in ~30% of all builds.

Chris-D-Turk avatar May 01 '22 17:05 Chris-D-Turk

I've started facing this issue on CI (TeamCity) after introducing Source Generators to my project.

06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Apache.Ignite.csproj (in 282 ms).
06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj (in 288 ms).
06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/Apache.Ignite.Tests.csproj (in 501 ms).
06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Benchmarks/Apache.Ignite.Benchmarks.csproj (in 528 ms).
06:48:21     Apache.Ignite.Internal.Generators -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018: System.IO.IOException: The process cannot access the file '/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.deps.json' because it is being used by another process. [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Win32.SafeHandles.SafeFileHandle.Init(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at System.IO.File.Create(String path) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFile(String depsFilePath) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:23     Apache.Ignite -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/bin/Debug/netstandard2.1/Apache.Ignite.dll
06:48:26     Apache.Ignite.Tests -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/bin/Debug/netcoreapp3.1/Apache.Ignite.Tests.dll
06:48:26     Apache.Ignite.Benchmarks -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Benchmarks/bin/Debug/netcoreapp3.1/Apache.Ignite.Benchmarks.dll
06:48:26   
06:48:26   Build FAILED.
06:48:26 

The workaround for me was to add dotnet restore step before dotnet build. Sometimes there is still some file access issue during dotnet build, but it gets retried and the build passes:

All projects are up-to-date for restore.
[05:55:24 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=488&linesState=488)    /usr/share/dotnet/sdk/6.0.100/Microsoft.Common.CurrentVersion.targets(4635,5): warning MSB3026: Could not copy "obj/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll" to "bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll". Beginning retry 1 in 1000ms. The process cannot access the file '/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/obj/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll' because it is being used by another process.
[05:55:24 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=489&linesState=489)      Apache.Ignite.Internal.Generators -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll
[05:55:25 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=490&linesState=490)      Apache.Ignite.Internal.Generators -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll
[05:55:27 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=491&linesState=491)      Apache.Ignite -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/bin/Debug/netstandard2.1/Apache.Ignite.dll
[05:55:30 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=492&linesState=492)      Apache.Ignite.Tests -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/bin/Debug/netcoreapp3.1/Apache.Ignite.Tests.dll

So clearly there is some race condition with file access.

ptupitsyn avatar Sep 21 '22 06:09 ptupitsyn