Dotnet tool command doesn't respect nuget packages path

[uladz-zubrycki]

trafficstars

Description

It's possible to change the path for nuget packages globally by setting NUGET_PACKAGES environment variable (there're other means of achieving it check the documentation for details) . Once that done dotnet tool fails to start local tool. I guess it happens as tool was actually restored to nonstandard path, dotnet tool is not able to find it there and consistently asks to run dotnet tool restore.

Steps

1. Set NUGET_PACKAGES environment variable to some directory other than default %userprofile%/.nuget/packages
2. Initialize tool manifest by running dotnet new tool-manifest
3. Install some tool, e.g dotnet tool install paket
4. Try to run tool, e.g dotnet paket init

Expected result

Tool is invoked (which is paket init in current case)

Actual result

Error message saying Run "dotnet tool restore" to make the "paket" command available.

Running restore as suggested doesn't help. But everything works as expected after I manually move tool from the directory it was restored to to the %userprofile%/.nuget/packages.

Haven't checked whether the same is actual for global tool installation, but would guess it is.

Environment

.NET SDK (reflecting any global.json):
 Version:   5.0.101
 Commit:    d05174dc5a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.101\

Host (useful for support):
  Version: 5.0.1
  Commit:  b02e13abab

.NET SDKs installed:
  3.1.404 [C:\Program Files\dotnet\sdk]
  5.0.101 [C:\Program Files\dotnet\sdk]

[cfrob]

There seems to be another quirk to this issue: After moving the locally installed tool to the default global-packages folder (as described by @v-zubritsky above), trying to restore the tool locally again using dotnet tool restore won't even install it locally anymore.

This aspect renders our current workaround for this issue useless: We simply called to tool by running dotnet giving it the full path to the installed tool-assembly, e.g. dotnet <packages_folder>/trx2junit/1.3.1/tools/netcoreapp3.0/any/trx2junit.dll This doesn't work reliably as the effect of dotnet tool restore depends on the state of the default global-packages folder.

[wli3]

Cause: Local tools assuming there is only one nuget global cache, and it does not change (or it will start from scratch to populate the cache). It is also assumed by nuget. So local tools does not work well when changing the folder location on the fly.

[manfred-brands]

This seems to work for us. dotnet is a client/server type application. Make sure you shutdown the server with dotnet build-server shutdown after you change the environment variable. After that dotnet tool restore and dotnet tool run will use the new location.

[rmarinho]

I m also facing this issue, isn't no fix for this ? basically we need to just keep nuget packages on the default folder?

[manfred-brands]

I stand corrected. dotnet tool restore installs in the new place, but dotnet tool run doesn't look there.

[brantburnett]

I have also encountered this problem. I had to copy the restored tool from the custom path to the ~/.nuget/packages directory before dotnet tool run would work.

Note that I encountered this problem using the new Dev Drive feature in Windows, which recommends redirecting NUGET_PACKAGES to the Dev Drive for improved performance in the documentation. This seems like a significant gap in the new high performance development experience being touted.

https://learn.microsoft.com/en-us/windows/dev-drive/#storing-package-cache-on-dev-drive

[xqiu]

Same here, my C drive does not have any more spaces so that I have to move the nuget packages to another drive, then boom, dotnet ef stopped working! I have to use mklink /J to make a directory junction from %userprofile%/.nuget to my new directory to make it working.

[MichalMucek]

It's the same for me as in the issue description but with different tools. I have to handle it manually by moving the .NET tool packages to the service account's user folder.

[kevkov]

As a workaround the shim files in $HOME/.dotnet/toolResolverCache can be edited to fix the location of the tool. This way you don't need two nuget package dirs.

[Akarinnnnn]

Same here, my C drive does not have any more spaces so that I have to move the nuget packages to another drive, then boom, dotnet ef stopped working! I have to use mklink /J to make a directory junction from %userprofile%/.nuget to my new directory to make it working.

Make a junction from$env:USERPROFILE\\.nuget\\packages to $env:NUGET_PACKAGES also works.

[baronfel]

The fix we should go for here is to change the way we write and/or interpret the toolResolverCache entries for each tool. It may be helpful to look at a resolver cache entry in detail:

{
    "Version": "6.2.3",
    "TargetFramework": "net8.0",
    "RuntimeIdentifier": "any",
    "Name": "fantomas",
    "Runner": "dotnet",
    "PathToExecutable": "C:\\Users\\<username>\\.nuget\\packages\\fantomas\\6.2.3\\tools/net6.0/any/fantomas.dll"
}

This tool hardcodes the path at install-time. We should do one of the following:

• write a relative path (i.e. within the package's directory) instead of an absolute path
• ignore the PathToExecutable entirely for local tools and use the NuGet APIs at tool-run-time to compute the path to the dll to invoke
• keep the current absolute path behavior, but align the behavior of dotnet tool restore and dotnet tool run so that a dotnet tool restore properly updates the absolute paths in the tool resolver cache and run is able to use that to look up the packages
• remove the nuget package path from the PathToExecutable and write it as a new property, and then ensure that if the nuget package path changes that a tool restore operation is requested of the user to update the entries

My personal preference is option 1 or 2, but all 3 would solve the user-facing problem.

[tudor-turcu]

Any news about this? It would help a lot also the adoption of DevDrive: https://learn.microsoft.com/en-us/windows/dev-drive/

[CaringDev]

My personal preference is option 1 or 2, but all 3 would solve the user-facing problem.

As a user I'd prefer option 1, as that would be the least surprising / magic and help me debug potential issues. Second would be option 4

[SPIKEYPUP]

This tool hardcodes the path at install-time. We should do one of the following:

• write a relative path (i.e. within the package's directory) instead of an absolute path
• ignore the PathToExecutable entirely for local tools and use the NuGet APIs at tool-run-time to compute the path to the dll to invoke
• keep the current absolute path behavior, but align the behavior of dotnet tool restore and dotnet tool run so that a dotnet tool restore properly updates the absolute paths in the tool resolver cache and run is able to use that to look up the packages
• remove the nuget package path from the PathToExecutable and write it as a new property, and then ensure that if the nuget package path changes that a tool restore operation is requested of the user to update the entries

Aloha! I'm working towards implementing a DevDrive for my environment, but it seems like this might still be an open issue at this time.

I wanted to throw my vote in for the second option: using the Nuget APIs at runtime to compute the path to the dll.

The use of the Nuget APIs at runtime seems to be more ideal, offering: reliability, flexibility and transparency for the User while offering a more robust and configurable solution in the long run; especially for those Users that find themselves in highly dynamic or heavily customized envionments etc.

This fix would eliminate wasted time on troubleshooting/diagnosing or time spent performing a workaround implementation for Users that have modified their package cache location and are unaware of this issue. It would also streamline DevDrive setup and undoubtedly help the adoption of DevDrive as well.

Thanks for your consideration... 😺 Have a great day!

[mortenp-elkjop]

Same DevDrive scenario, same issue. Any ETA for the fix?

[bbrandt]

Same here, my C drive does not have any more spaces so that I have to move the nuget packages to another drive, then boom, dotnet ef stopped working! I have to use mklink /J to make a directory junction from %userprofile%/.nuget to my new directory to make it working.

Make a junction from$env:USERPROFILE\\.nuget\\packages to $env:NUGET_PACKAGES also works.

This is the workaround I used. After defining your NUGET_PACKAGES environment variable, from a new admin PowerShell terminal:

New-Item -ItemType Junction -Path ($env:USERPROFILE + "\\.nuget\\packages") -Target $env:NUGET_PACKAGES

[GF-Huang]

4 years pass, man. What can I say.