winget-cli
winget-cli copied to clipboard
microsoft
`PATH` variable eventually becomes too long when installing many CLI apps
Description of the new feature / enhancement
Summary
Currently, many CLI packages tend to modify the user PATH variable to make its binaries available in the shell. This is totally fine. However, as the user installs more CLI apps[^1], the PATH variable becomes so crowded that it exceeds Windows' default 2048-character limit.
[^1]: As is typically the case for many developer workflows.
Additional Context
Normally, this is not an issue because some programs just sweep the limit under the rug. However, I report this issue now because I did encounter an issue with Goneovim where it always crashes on startup. A "little" investigation with the --debug logs led me to the conclusion that the nvim executable could not be found in the PATH.
This baffled me because I was sure I had it in there. Sure enough, the Neovim's bin folder was in the PATH. It seemingly turns out that either Goneovim or Qt (i.e., the UI framework on which Goneovim is built) mishandles PATH variables whose lengths are greater than 2048. I have not sufficiently investigated where the issue is, but once I shortened the PATH, my problem was resolved.
Now, I know that this is technically an issue with Goneovim and Qt, but I figured to tackle the root of the issue instead: the excessively long PATH variable (hence filing this issue).
Proposed technical implementation details
Problem
When Winget installs CLI apps, some packages modify the PATH variable such that Winget appends the following entry: %LOCALAPPDATA%\Microsoft\WinGet\Packages\<package>_Microsoft.Winget.Source_8wekyb3d8bbwe. Even without substituting the %LOCALAPPDATA% variable, observe that the entry is already quite verbose for a PATH entry.
Now multiply this by the number of CLI apps that need to be installed. The bytes quickly add up!
Prior Art of a Possible Solution
Other package managers such as Chocolatey and Scoop work around this issue by automatically generating thin "shims" in a common bin-like folder (i.e., C:\ProgramData\chocolatey\bin and %USERPROFILE%\scoop\shims, respectively). Then, this folder is the only one added to the PATH variable.
Pros: we only add one entry to the PATH. New installations only need to generate a shim into the bin-like folder for it to be made available to the shell.
Cons: shim synchronization may be an issue. There are more points of failure in the indirection, which may introduce more bugs. In my experience with Chocolatey and Scoop, however, these have never been an issue.
For a bit of background - This "path cramming" only happens when winget can't create symlinks. This happens when developer mode is disabled and the cli is run without admin permissions. When developer mode is enabled, only one directory is added to PATH and all CLI packages are symlinked in through that folder, similar to the shims you suggested
This happens when developer mode is disabled and the cli is run without admin permissions.
Interesting. Why is this feature not the default, by the way? Is it currently experimental?
This happens when developer mode is disabled and the cli is run without admin permissions.
Interesting. Why is this feature not the default, by the way? Is it currently experimental?
It is the default, but symlink creation fails due to permissioning issues if developer mode is disabled and winget is run in user-space. Here is the original PR - https://github.com/microsoft/winget-cli/pull/2369#issuecomment-1235645108
If you either enable developer mode or run winget using an administrator terminal, then the symlinks get created normally
That's a little unfortunate. I would've preferred a non-"Developer Mode" way of activating it due to the security implications of "instal[ling] apps from any source, including loose files". But if that is the direction Winget intends to follow (perhaps due to OS limitations), then that can be arranged on my part.
That's a little unfortunate. I would've preferred a non-"Developer Mode" way of activating it due to the security implications of "instal[ling] apps from any source, including loose files".
As @Trenly mentioned, there is a non-developer mode way; You can get the same behavior if you run the winget command from an admin terminal.
Yes, this is fair enough. It's not often that I install new CLI apps anyway.
As a resolution to this issue, though, where may be the place to contribute to the documentation of this behavior?
Not entirely sure what would be the best place. I guess updating the portable spec and highlighting these points in the install flow may be a good starting point?
https://github.com/microsoft/winget-cli/blob/master/doc/specs/%23182%20-%20Support%20for%20installation%20of%20portable%20standalone%20apps.md#install
I'm open to suggestions for documentation.
I could see the "install" command page and the troubleshooting page as options.
Which do you think you would have looked into, or do you think there is a better way to make the information "more" discoverable?
Which do you think you would have looked into, or do you think there is a better way to make the information "more" discoverable?
My first instinct was definitely checking out if the winget install --help page mentioned any CLI flag for preferring shims and symlinks over the fallback PATH modification. The keyword "portable" and "symlinks" admittedly never crossed my mind, so it's definitely worth mentioning that in the --help page.
As for the troubleshooting page, I certainly wouldn't have considered visiting it because I can't say that this behavior is a "bug", per se. Rather, it feels more like a limitation than a bug.
So in summary, I believe the best visibility for this is in the winget install --help page. The install command page in the Microsoft Docs will also be a great place to add it because it will be easier for search engines to pick up on keywords like "portable", "symlinks", "PATH variable", "2048", and "administrator". I also found that linking to the PR #2369 was invaluable to my understanding of the reasoning behind the fallback behavior.
Problem: Running as administrator in order to create the portable app symlink does not work if the user is a 'standard user'. For a 'standard user' executing as administrator installs the portable app in the USERPROFILE of the adminstrative account used to do the install. That is obviously not accessible to the standard user.
Is there a workaround for this that does not involve developer mode that I am missing?
