BuildXL fails with vcxproj and ProjectReferences

[wjk]

When I point BuildXL at an MSBuild C++ project that contains a <ProjectReference> element, it fails with the following error message (line-wrapped for readability):

D:\Projects\BuildXL Demo\config.dsc(3,9): error DX11401: An error occurred while parsing
MsBuild spec files: Errors found during target prediction. Project 'D:\Projects\BuildXL
Demo\bxl.demo\bxl.demo.vcxproj' is not specifying its project reference protocol. For more
details see https://github.com/Microsoft/msbuild/blob/master/documentation/specs/static-graph.md

As I understand it, this is because the vcxproj infrastructure lacks the content in Microsoft.Managed.targets that provides information required by BuildXL. Unfortunately, I do not know enough about BuildXL to provide the correct metadata myself.

[dannyvv]

Hi @wjk,

Unfortunately all devs that work on the MsBuild integration our out of office. I have added them as assignees for then they are back.

[wjk]

@dannyvv I have a slightly off-topic question for you. How should I submit ideas for improvement regarding vcxproj such that the team at MS will actually review them? I have found quite a few pain points over my years of using it. I have posted them in several areas, including several MSVC-related GitHub repos as well as VS Developer Community, and I have never received a response anywhere. Other developers have also complained about these issues, and they have not received a response either. (For example, take this Developer Community thread. The only response from MS it received was a boilerplate "comment received" message that every new issue there gets.) If you could help me route my comments to the approrpiate team, I would greatly appreciate it. Thanks so much!

[jbayardo]

In the meantime, you might want to try to set useLegacyProjectIsolation to true within the MSBuild resolver configuration. See here for precise comment.

[wjk]

@jbayardo Unfortunately, setting useLegacyProjectIsolation does not seem to do anything. The build still fails with the same error.

[cdmihai]

@wjk this section describes how to let static graph know what targets projects call each other with: https://github.com/Microsoft/msbuild/blob/master/documentation/specs/static-graph.md#inferring-which-targets-to-run-for-a-project-within-the-graph

As you've noticed, we've added the managed sdk specification in Microsoft.Managed.targets, because traditionally the managed sdk shipped parts of itself with msbuild. But vcxproj is pretty isolated, we don't have a story for them yet. Probably we'll have to send a PR to the vcxproj team to add the target reference protocol, and setup a CI to prevent them from regressing.

[hayhurst-zz]

Any update on this? @wjk did you get anywhere implementing this yourself. @cdmihai Is it something that is scheduled to happen?

[smera]

@cdmihai, if this is not implemented as part of the official SDK yet, maybe specifying the target protocol at the repo level is an option? I'm not super familiar with this, but if I remember correctly that was possible. In that case maybe we can help @wjk to come up with a definition for it that can be injected for his particular repository.

[cdmihai]

@cdmihai, if this is not implemented as part of the official SDK yet, maybe specifying the target protocol at the repo level is an option? I'm not super familiar with this, but if I remember correctly that was possible. In that case maybe we can help @wjk to come up with a definition for it that can be injected for his particular repository.

The official c++ SDK most likely does not have the target protocol specified.

Couple of options:

• To begin with, AFAIK BuildXL had an option to turn off setting /isolate on msbuild.exe calls (@smera what was the cmdline? If it exists). That should turn off any isolation errors you are getting so you can explore the rest of the potential build issues without caring about /isolate errors. You might still want to add a generic <ProjectReferenceTargets Include="Build" Targets="Build" /> protocol to at least have the target Build inferred on every node (or whatever the target name the c++ sdk uses to build projects).
• As @smera said, you can specify the target protocol for the c++ sdk yourself for the whole repo. A good place to put it in would be a directory.build.props file at the root of the repo. To get started, you should probably read the docs on the target protocol and take a look at how the managed sdk specifies it. To reverse engineer the target calling patterns you can use the msbuild binary log viewer to see which targets projects call on their dependencies.

[smera]

This knob allows you to turn off static graph project isolation and just rely on the 'do not build my parent' honor code, which is respected but many SDKs (but no engine enforcements are in place).

[hayhurst-zz]

This:

• To begin with, AFAIK BuildXL had an option to turn off setting /isolate on msbuild.exe calls (@smera what was the cmdline? If it exists). That should turn off any isolation errors you are getting so you can explore the rest of the potential build issues without caring about /isolate errors. You might still want to add a generic <ProjectReferenceTargets Include="Build" Targets="Build" /> protocol to at least have the target Build inferred on every node (or whatever the target name the c++ sdk uses to build projects).

Worked for me, thanks for your help.

I've hit another (unrelated?) issue, error DX0500 writing to output files that already exist when building projects referenced by the module being built.

Disallowed file accesses performed by: C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\MSBuild.exe
 W  C:\svn\c0\code\bin\Debug_Win32\GDXTest.lib
 W  C:\svn\c0\code\obj\Debug_Win32\GDXTest\GDXTest.tlog\GDXTest.lastbuildstate
 W  C:\svn\c0\code\obj\Debug_Win32\GDXTest\GDXTest.tlog\lib.command.1.tlog

W  = Write

Wondering if this is something to do with building referenced vcxproj files or something to do with the location of the output directories relative to the other files.

.bc file here: C:\svn\c0\code
module being build: C:\svn\c0\code\LUSAS\LusasWin.vcxproj output: C:\svn\c0\code\bin\Debug_Win32\

[smera]

By default BuildXL expects writes to happen under the repo root. This is just a default policy to block a build from writing into random places of your disk. You can declare additional places where outputs are expected to happen using the 'additionalOutputDirectories' in the main BuildXL configuration file.

[hayhurst-zz]

I did try that in LusasWin.bc

config({
    resolvers: [
        {
            kind: "MsBuild",
            root: d`.`,
            moduleName: "LusasWin",
            msBuildSearchLocations: [d`C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\`],
            fileNameEntryPoints: [r`LUSAS\LusasWin.vcxproj`],
            useLegacyProjectIsolation: true,
            additionalOutputDirectories: [d`C:\SVN\C0\code\bin\Debug_Win32\`,d`C:\SVN\C0\code\obj\Debug_Win32\`],
        }
    ]
});

Tried plenty of variations of additionalOutputDirectories but it didn't change the output.

[smera]

There should be additional info about the violation in the main bxl log file that explain the underlying reason with more detail. E.g. if those files already exist, bxl will also block the write. If you don't actually care about those files (in the sense of being part of the outputs of your build, or if no relevant decision is made based on its presence) you can also untrack them and bxl will allow the rewrite.