I am facing a bizarre issue with the build. Two copies of the repo, each at commit 6096b5e, one builds and the other fails.

[AltitudeApps]

I have some minor contributions I'd like to make, and towards that goal I'm trying to get in a state where I can reliably build the solution. I am having difficulties that I cannot explain, and I am at a loss to even speculate as to the reasons.

I cloned the repo about March 31st, that repo is at commit 6096b5e. It builds fine, with no obvious errors, and searching for "-- FAILED" in the build output finds nothing. I made my quite minor proof-of-concept modifications, rebuilt it and it ran fine and as expected.

I then had the idea to re-clone the repo to get things into a more up-to-date state, and redo my changes with the aim of preparing a pull request. But I could not get it to build at all.

I was glad that I kept my previous version of the tree intact, because at least that is still working.

I decided to investigate by creating a new worktree for the repo, and checking out commit 6096b5e into that folder. I then attempted to "Rebuild all".

Quite a few failures, indicating missing dependencies/references. I will of course include the full build output, because I don't know exactly which details will turn out to be important.

I did create a script which did git ls-files and verified that the files in both trees were identical using Get-FileHash, out an abundance of caution. But this only counts for the source files, it turns out that there are differences in some folders when I compare with Beyond Compare.

Most concerning is that the vcpkg_installed folders contain libraries that are not identical.

[AltitudeApps]

Here is the full build output. (I will try putting it into a gist instead.)

https://gist.github.com/AltitudeApps/acbd04e649eb23f506e654c1c8d2c820

Things start to go south here: PowerShellConfigurationSetProcessorFactory.cs(17,46,17,65): error CS0234: The type or namespace name 'SetProcessorFactory' does not exist in the namespace 'Microsoft.Management.Configuration' (are you missing an assembly reference?)

13:15:45:613	17>------ Rebuild All started: Project: Microsoft.Management.Configuration.Processor, Configuration: Debug Any CPU ------
13:15:45:623	18>------ Rebuild All started: Project: Microsoft.Management.Configuration.Projection, Configuration: Debug Any CPU ------
13:15:48:343	11>Generating Code...
13:16:09:480	11>Compiling...
13:16:09:548	11>ManifestDeserializer_1_9.cpp
13:16:19:713	16>ManifestValidation.cpp
13:16:27:572	18>0 IID calculations/fetches patched
13:16:27:712	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(17,46,17,65): error CS0234: The type or namespace name 'SetProcessorFactory' does not exist in the namespace 'Microsoft.Management.Configuration' (are you missing an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Helpers\TypeHelpers.cs(15,46,15,65): error CS0234: The type or namespace name 'SetProcessorFactory' does not exist in the namespace 'Microsoft.Management.Configuration' (are you missing an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Helpers\TypeHelpers.cs(161,23,161,55): error CS0246: The type or namespace name 'PwshConfigurationProcessorPolicy' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Helpers\TypeHelpers.cs(180,103,180,135): error CS0246: The type or namespace name 'PwshConfigurationProcessorPolicy' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Helpers\TypeHelpers.cs(199,23,199,57): error CS0246: The type or namespace name 'PwshConfigurationProcessorLocation' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Helpers\TypeHelpers.cs(216,107,216,141): error CS0246: The type or namespace name 'PwshConfigurationProcessorLocation' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(28,200,28,247): error CS0246: The type or namespace name 'IPwshConfigurationSetProcessorFactoryProperties' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(177,42,177,89): error CS0246: The type or namespace name 'IPwshConfigurationSetProcessorFactoryProperties' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(207,44,207,91): error CS0246: The type or namespace name 'IPwshConfigurationSetProcessorFactoryProperties' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(177,9,177,41): error CS0246: The type or namespace name 'PwshConfigurationProcessorPolicy' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(177,42,177,89): error CS0538: 'IPwshConfigurationSetProcessorFactoryProperties' in explicit interface declaration is not an interface
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(207,9,207,43): error CS0246: The type or namespace name 'PwshConfigurationProcessorLocation' could not be found (are you missing a using directive or an assembly reference?)
13:16:27:727	17>D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e\src\Microsoft.Management.Configuration.Processor\Public\PowerShellConfigurationSetProcessorFactory.cs(207,44,207,91): error CS0538: 'IPwshConfigurationSetProcessorFactoryProperties' in explicit interface declaration is not an interface
13:16:27:727	17>Done building project "Microsoft.Management.Configuration.Processor.csproj" -- FAILED.

[denelon]

Did you use the WinGet Configuration file to set up your environment? (I don't think this would have any bearing, but I'm asking out of curiosity on how you set your environment up).

Have you tried a "clean" and "rebuild"?

[AltitudeApps]

yes, I set up the environment using the instructions on the project page, to use the:

To run the configuration, use winget configure .config/configuration.winget from the project root so relative paths resolve correctly.

command, in both folders. I'm not sure why or why not I should do that, because I thought that the winget configure thing was somehow global to the machine. But, the text says "from the project root so relative paths resolve correctly.", so that says to me that it has something to do with the folder it is invoked from.

Is it even possible to have two copies of the source tree at the same time? I am so unfamiliar with the tooling at this early stage I can't tell what's reasonable to expect.

And I also did the

Run vcpkg integrate install from the Developer Command Prompt / Developer PowerShell for VS 2022. in both folders.

I have done a full clean and rebuild many times.

I think I had it actually work once; I did do it at the suggestion that was posted here in another issue, or maybe one of the discussions I started. And I think that's how I got the tree that is actually building and functioning to get in a happy state. I "think" that's what happened, but I'm not sure.

I'll fire off a 'clean and rebuild all" right now, since I'm going to bed anyway.

[AltitudeApps]

same inexplicable result.

[denelon]

The "relative" path part is specifically to make sure the .vsconfig file is captured from the project's relative path. It should just be adding workloads and other components necessary to Visual Studio to be able to build/run the project.

[JohnMcPMS]

Is it even possible to have two copies of the source tree at the same time?

Yes, I have 3 on this machine that I work with and build simultaneously.

Your git root ("D:\a\aa.git.local.repos\github.com\AltitudeApps\winget-cli-repo\worktrees\wt-for-commit-6096b5e") is quite long; it makes me wonder if there is some tooling that does not handle long paths (over MAX_PATH, aka 255). How does this compare to the length of the working directory?

I would also suggest collecting a binlog and inspecting or posting that.

[AltitudeApps]

plausible.

I'm not sure what you mean by 'working directory'? That path is the working directory, as I understand the term. In other words, the src/ folder is right in that folder.

Here is the maximum path length in the folder where things are working:

Here is the maximum path length in the folder where things are broken (wt-for-commit-6096b5e):

They're both really long already. But it might be a "straw that broke the camel's back" situation.

I will create a junction to that folder at the root, and open the project from there to see if there's any change.

Wait... will that even work? I was looking in some of the generated .json files, and they had hard coded full paths in them. Will those get cleared out with a "clean" command?

I will also configure the collection of a binlog, as suggested.

[denelon]

I just did a quick search, and this is what the AI told me. I don't know if that's absolutely correct, but it might be worth trying a shorter path.

I don't think I've hit this problem myself, but I use a DevDrive "D:" myself with a folder called GitHub, and I keep all my repositories there.:

[AltitudeApps]

Thank you, yes. I am pretty sure I have used that registry tweak on all of my computers, just because I like to have my paths read like English. (So I can have a chance of remembering what's actually in them.)

I am in the middle of a rebuild using a shorter path, by making a directory junction at D:\wt-for-commit-6096b5e.

I am already past the point where it was failing previously. It will take some more time until the full rebuild completes. But it's looking good for the 'paths are too long' theory.

If the build succeeds, I will do another "rebuild all" on the long path that fails, and generate some binlogs which might tell us why it was failing with a path that was just 8 characters longer than the other path, which was itself seemingly "too long" to begin with.

[AltitudeApps]

Happy:

Suspicious, lol!

I intend to do a rebuild of only the 'Microsoft.Management.Configuration.Processor' project and collect some binlogs, and find out exactly which part of the toolchain is unhappy with long paths.

[AltitudeApps]

OK, I have a little bit of binlog info now. Struggling to get it working as expected, because depending on how I open Visual Studio, the environment variables get set a little differently. (I'll get this worked out.)

But I did find the following, but it's only telling me what I already know:

It says that Microsoft.Management.Configuration.Processor.xml does not exist, which isn't surprising since the path is so long. But it appears in this log as an output file? I was expecting that a missing or defective input file would be the cause of the trouble. (Obviously, I only have a tenuous understanding of what I'm looking at.)

[AltitudeApps]

OK, I've done some more searching and reading. Not finished fully comprehending in whole the material, but according to https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation , the maximum path length is somewhere around 32kb-1 for some of the unicode variants of the functions in the Windows API, and it also mentions that that the Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" option has existed since about 2016. (Not sure if these are directly related or not.)

I've also found some entertaining and educational threads discussing the topic, some going back more than a decade:

https://stackoverflow.com/questions/139964/msbuild-directory-structure-limit-workarounds https://stackoverflow.com/questions/69851021/how-to-pass-through-the-260-characters-path-limit https://old.reddit.com/r/cpp/comments/zdndig/msvc_compiler_cant_find_source_file_in_path_that/

... and some pretty good kvetching in this thread:

https://old.reddit.com/r/cpp/comments/zdndig/msvc_compiler_cant_find_source_file_in_path_that/

The workaround seems to be the obvious one: create a junction on a shorter path which points to the too-long project folder. (Only obvious if you know that this is possible, of course.) This suggestion comes up repeatedly.

I have no idea how often this limitation crops up in the tools typically involved in MSBuild.exe, csc.exe, vspkg, etc., so I guess I'll halt my search for a root cause at this point. The problem could pop up anywhere, and would pop up again if another build tool was added somewhere in a new project being added to the solution.

I am guessing that either:

1. Most of the teams responsible for the build tools have simply acquiesced to the problem, especially since the workaround is so simple.
2. or, there were attempts to modify the code to "opt-in" to the long path support, but this introduced weird edge cases that turned out to be worse than simply putting up with the limitation,
3. or, juggling an arbitrary number of path values, each up to 32kb in size, was deemed to be excessive.

So, I guess I'll leave it there for now.

I may experiment with creating a project to include in the solution, which would be a dependency for all the other projects, which would simply have as a pre-build step, and check to see how long the current path was, and issue a warning if it were longer than, say 200 characters or so.

[denelon]

There are some other folks who have been looking at the "long tail" problems with long path support. I'll pass this thread along to them as another solid datapoint on the value of finding all those "weird edge cases".

[AltitudeApps]

Thank you. You might also mention this additional bit of feedback:

As this problem becomes more and more rare, it trends further towards the "unexpected and inexplicable" category. It seems obvious to me now, but I had zero idea I should consider path length problems two weeks ago.

Had this happened to me 10 or 15 years ago? It would have been something I would have had to deal with on a semi-regular basis. Or maybe not? I've become accustomed to using quite long and verbose paths over the last ten years, 20 years ago it wouldn't even have occurred to me to try it at all. I would have already been punished when attempting it, thus therefore been trained not to do it.

I guess we're spoiled by the fact that very long paths work almost all of the time, but this puts us in the position of being completely baffled when something goes wrong as a consequence, especially when there is no obvious error message being surfaced.

Hopefully a 99% + five nines solution can be achieved, but it is highly desirable that good error messages (or even just a process panic) can be emitted in the cases where any edge cases slip by.