Improved build output when building project in multiple global properties

[TheRealPiotrP]

in https://github.com/dotnet/cli we use task batching on to enable parallel test execution. When tests fail, the summary data printed is fairly undiagnosable, telling us only that the test runner project failed N times. Even in verbose mode the output is not providing sufficient context to understand what actually went wrong. It would be nice if the output summary included both a listing of the instance ID’s [e.g. 2:{instance ID} in verbose output] and a summary of the properties passed in to each instance.

Actual Output:

"/Users/piotrp/code/github/piotrpMSFT/cli/build.proj" (default target) (1) ->
       "/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj" (default target) (2:18) ->
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1008: Only one project can be specified. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1008: Only one project can be specified. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]
         MSBUILD : error MSB1008: Only one project can be specified. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj]

Improved Output:

"/Users/piotrp/code/github/piotrpMSFT/cli/build.proj" (default target) (1) ->
       "/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj" (default target) (2:18) ->
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:16) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:17) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:21) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:16) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:12) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:17) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:19) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:22) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:11) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1008: Only one project can be specified. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:16) ->`
         MSBUILD : error MSB1008: Only one project can be specified. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1009: Project file does not exist. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`
         MSBUILD : error MSB1008: Only one project can be specified. [/Users/piotrp/code/github/piotrpMSFT/cli/build/test/RunTest.proj] `(default target) (2:18) ->`

Batch Summary
----------------

ID     Input Properties
---  ------------------
16    ProjectToTest={SomePath1}
        OtherProp=Debug
17    ProjectToTest={SomePath2}
        OtherProp=Debug
18    ProjectToTest={SomePath3}
        OtherProp=Debug
19    ProjectToTest={SomePath4}
        OtherProp=Debug
20    ProjectToTest={SomePath5}
        OtherProp=Debug
21    ProjectToTest={SomePath6}
        OtherProp=Debug
22    ProjectToTest={SomePath7}
        OtherProp=Debug

[danmoseley]

Task batching does not run the tasks concurrently, only serially. Either this isn't giving you the concurrent execution you're expecting, or you're not actually batching the MSBuild task -- you're passing all the projects/tasks to it at once. The only way to get any kind of parallelism in MSBuild is to give the MSBuild task multiple projects at once.

[rainersigwald]

From the text I infer that the issue is really about understanding which of potentially multiple configurations (global property sets) of a specific project a message is coming from. That is indeed hard to chase down. Updated the title to reflect that.

[TheRealPiotrP]

And since vocabulary is hard when it gets into the depths of batching... here is what I'm doing: https://github.com/dotnet/cli/blob/rel/1.0.0/build/Microsoft.DotNet.Cli.Test.targets#L48-L51

[rainersigwald]

That's not batching--the giveaway is that there's nothing in the form %(ItemName.MetadataName). That's just a parallel MSBuild call, more or less like a Parallel.Foreach--batching is more like a regular serial foreach of the element in which the batching takes place.

(This is definitely confusing!)

[TheRealPiotrP]

hehe, ok. I think we've been calling that Task Batching in our team. Task vs. Target batching. Though I do think I appreciate the delta you're drawing... let me know if you come up with a name for it :D

ParallelForEaching?

[rainersigwald]

Team triage: We'd like to improve this. But people do grep through logs, so it's a breaking change.

Tentative plan:

1. Improve console/text logging behind an opt-in flag (environment variable?)
2. Ship that for 16.x
3. Change the default for 17.0/next breaking change.

[livarcocc]

This needs design. Let's work on that first.

[rainersigwald]

https://github.com/dotnet/sdk/pull/12030 needs to be brought back to get this done.

[rainersigwald]

https://github.com/dotnet/sdk/pull/12030 is long complete and we're doing additional work on this for the -tl experience, so I'm going to close this.