source-build: support building runtime using non-portable runtime packages.

[tmds]

trafficstars

Currently source-build performs a 'runtime-portable' build that produces 'linux-{arch}' packages that are used when building target runtime (non-portable).

With this change, we can use the non-portable packages that are produced by a previous (non-portable) 'runtime' build. This helps eliminate the 'runtime-portable' build.

cc @omajid @MichaelSimons @crummel @ViktorHofer @ericstj @jkotas

[dotnet-issue-labeler[bot]]

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

[tmds]

This isn't working fully.

For some reason ILCompiler.csproj and crossgen2.csproj go look for Microsoft.AspNetCore.App.Runtime.<rid>.

  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj : error NU1102: Unable to find package Microsoft.AspNetCore.App.Runtime.linux-x64 with version (= 7.0.0-rc.2.22452.11) [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj : error NU1102:   - Found 1 version(s) in previously-source-built [ Nearest version: 7.0.0-rc.1.22404.6 ] [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj : error NU1102:   - Found 0 version(s) in source-built [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj : error NU1102:   - Found 0 version(s) in prebuilt [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj : error NU1102:   - Found 0 version(s) in reference-packages [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/crossgen2/crossgen2.csproj : error NU1102: Unable to find package Microsoft.AspNetCore.App.Runtime.linux-x64 with version (= 7.0.0-rc.2.22452.11) [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/crossgen2/crossgen2.csproj : error NU1102:   - Found 1 version(s) in previously-source-built [ Nearest version: 7.0.0-rc.1.22404.6 ] [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/crossgen2/crossgen2.csproj : error NU1102:   - Found 0 version(s) in source-built [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/crossgen2/crossgen2.csproj : error NU1102:   - Found 0 version(s) in prebuilt [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]
  /home/tmds/tarball/src/runtime/artifacts/source-build/self/src/src/coreclr/tools/aot/crossgen2/crossgen2.csproj : error NU1102:   - Found 0 version(s) in reference-packages [/home/tmds/tarball/src/runtime/artifacts/source-build/self/src/Build.proj]

I don't understand where this reference comes from, but probably we don't actually need this? @ericstj @ViktorHofer how can I handle this?

[tmds]

For some reason ILCompiler.csproj and crossgen2.csproj go look for Microsoft.AspNetCore.App.Runtime..

Apparently the reference happens as soon as you add <RuntimeIdentifier> based on the KnownFrameworkReferences. The fix is to remove the reference for Microsoft.AspNetCore.App.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure See info in area-owners.md if you want to be subscribed.

Issue Details

---

Currently source-build performs a 'runtime-portable' build that produces 'linux-{arch}' packages that are used when building target runtime (non-portable).

With this change, we can use the non-portable packages that are produced by a previous (non-portable) 'runtime' build. This helps eliminate the 'runtime-portable' build.

cc @omajid @MichaelSimons @crummel @ViktorHofer @ericstj @jkotas

Author:	tmds
Assignees:	-
Labels:	area-Infrastructure, community-contribution
Milestone:	-

[ghost]

Tagging subscribers to this area: @hoyosjs See info in area-owners.md if you want to be subscribed.

Issue Details

---

Currently source-build performs a 'runtime-portable' build that produces 'linux-{arch}' packages that are used when building target runtime (non-portable).

With this change, we can use the non-portable packages that are produced by a previous (non-portable) 'runtime' build. This helps eliminate the 'runtime-portable' build.

cc @omajid @MichaelSimons @crummel @ViktorHofer @ericstj @jkotas

Author:	tmds
Assignees:	-
Labels:	area-Infrastructure-coreclr, community-contribution
Milestone:	-

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure See info in area-owners.md if you want to be subscribed.

Issue Details

---

Currently source-build performs a 'runtime-portable' build that produces 'linux-{arch}' packages that are used when building target runtime (non-portable).

With this change, we can use the non-portable packages that are produced by a previous (non-portable) 'runtime' build. This helps eliminate the 'runtime-portable' build.

cc @omajid @MichaelSimons @crummel @ViktorHofer @ericstj @jkotas

Author:	tmds
Assignees:	-
Labels:	area-Infrastructure, community-contribution
Milestone:	-

[tmds]

@ViktorHofer @jkotas I took a step back and wrote up an issue that describes the goal: https://github.com/dotnet/source-build/issues/3027. And made a PR that includes these changes as a patch to help validate it: https://github.com/dotnet/installer/pull/14549.

@ViktorHofer I will address your feedback this week.

Besides this, we also need https://github.com/dotnet/runtime/issues/74721#issuecomment-1246787435.

[ViktorHofer]

@jkotas who would be the right person to review the ILCompiler and Crossgen2 changes?

[jkotas]

@jkotas who would be the right person to review the ILCompiler and Crossgen2 changes?

I can take a look or route as necessary once the feedback is addressed.

[tmds]

CI failed because of an issue with the dnceng nuget feed.

@ViktorHofer can you trigger CI, and also take another look at this PR?

[ViktorHofer]

Closing and reopening usually does the trick and retriggers all the legs.

[tmds]

@ViktorHofer since this has Condition=" '$(DotNetBuildFromSource)' == 'true' " and the integration test in https://github.com/dotnet/installer/pull/14549 is passing, I think we can consider CI green on this.

[ViktorHofer]

Added a style nit commit. Also, to retrigger CI as there were more than 20 failing legs.

[ViktorHofer]

Thanks a lot, Tom!

[tmds]

@ViktorHofer can you initiate backports to 7.0 for this and the other 2 related PRs: https://github.com/dotnet/runtime/pull/74504, https://github.com/dotnet/runtime/pull/76068. Can the bot handle these?

[ViktorHofer]

We just merged some of these into release/6.0. Ideally, we would have backported these changes at the same into release/6.0 and release/7.0 as we now need another set of approvals.

That said, yes, I will cherry-pick them together into a single PR and add the servicing template and ask for an approval.

[tmds]

Sorry for the overhead, and thank you for taking care of it!