sdk icon indicating copy to clipboard operation
sdk copied to clipboard

`slnx` support in the `dotnet` CLI

Open baronfel opened this issue 1 year ago • 23 comments

The dotnet CLI should support the new slnx format for building and in the existing solution management commands. It should also help interested users migrate to the new format.

The new format

slnx is an XML-based format that simplifies the current sln file format. When released, it will have an open-source parser, so tools like MSBuild and the dotnet CLI can consistently operate on the format. The new format is intended to reduce common customer pains like merge conflicts and readability, but not to drastically change the experience of working with solutions.

dotnet experiences using solutions

There are three primary ways that the dotnet CLI interacts with solution files today

  • building solutions
    • dotnet build myapp.sln
  • adding or removing projects and other files from solutions
    • dotnet sln myapp.sln add src/migrations
  • creating new solutions
    • dotnet new solution -n lodestone

Each of these should be made to work with the new format to some degree. In addition, a fourth new capability should be added:

  • migrating from sln to slnx
    • dotnet sln <sln file> migrate [-o <slnx file name>]

Building solutions

This should be mostly transparent to the dotnet CLI. Much like solutions today, building a solution involves passing the path to the solution file to the MSBuild engine, which has the sole responsibility of converting the build configurations into a 'metaproject' - a kind of MSBuild representation that MSBuild can actually execute - and then building the requested targets on that metaproject.

The same process would hold with slnx - the CLI would forward along the slnx file provided (if any) and MSBuild itself would translate that file into a metaproject and execute that metaproject. Very few changes should be required in the CLI codebase to support this. MSBuild's tracking issue for this is https://github.com/dotnet/msbuild/issues/10266.

Manipulating solution content

The CLI has several commands that allow for adding and removing projects in a solution, as well as listing the existing projects. All of these commands should work with slnx as well. This is the area that will require the most investment. The CLI will need to

  • learn that slnx files are valid inputs to these commands
  • provide some kind of alternative implementation for these commands that works on slnx files
  • pivot the implementation used based on the file type provided

These commands allow for selection of the solution file. If invoked in a location where multiple potential solution files are present, the command should error and prompt the user to choose one of the possible solutions.

We'll also need to invest in test coverage to make sure we have parity between our sln support and slnx support.

Creating new solutions

The CLI currently ships a solution template that create a new, barebones solution file.

We should provide a template that can create an empty slnx file for users to begin with. The new slnx template should use UTF-8 without a BOM. One major question: should we supplant the existing solution template to create an slnx file? Should the old solution format be accessible via a template parameter?

Migrating existing solutions

An entirely new capability to migrate a sln file to a new slnx file should be implemented as well to ease onboarding and allow for automation. This command would load the existing sln file and analyze it, translating it into an equivalent slnx file. Ideally any data that matches the default conventions of the new slnx format (for example, default build configurations like Debug and Release) could be omitted from the generated slnx file.

References

baronfel avatar May 15 '24 21:05 baronfel

Regarding the solution template, would it possible to exclude the byte order mark (BOM) now, or are there still known issues with VS or msbuild?

I know that:

  1. The Unicode Standard does not recommend its use.
  2. Even applications like Notepad now use UTF-8 without BOM by default. (source)

glen-84 avatar May 19 '24 12:05 glen-84

@glen-84 I'm inclined to make that change, but would like to discuss with @rainersigwald and the rest of the MSBuild team about any gaps there.

baronfel avatar May 19 '24 13:05 baronfel

For .sln I would leave it as it has been (I believe that's the longstanding Windows 16-bit chars with BOM).

For .slnx we should definitely avoid the BOM but would need to check with the VS folks to make sure they agree.

rainersigwald avatar May 20 '24 14:05 rainersigwald

I've reached out to the VS side to check compatibility with no-BOM.

baronfel avatar May 20 '24 14:05 baronfel

The VS team confirmed that slnx defaults to UTF-8 No-BOM, so we are clear to default the slnx template contents to no-BOM as well. I've updated the description to match this.

baronfel avatar May 20 '24 21:05 baronfel

Is there any concrete plan for this? I'm looking forward to build/test/restore support for slnx (also when referenced from a .slnf) so that we can update Sentry .NET SDK to use the new format.. Adding projects to the old format is so cumbersome when you need to keep a PR reviewable.

vaind avatar Jun 06 '24 10:06 vaind

We are still waiting for an official parser library for the new format to be made available by our partner teams. We're unable to make any progress without that library (and the stabilized format spec implied by that)

baronfel avatar Jun 06 '24 13:06 baronfel

related https://github.com/microsoft/slngen/issues/585

kasperk81 avatar Jul 12 '24 12:07 kasperk81

if the plan is still to include slnx support in sdk 9.0, is there any estimate when parsing library is going to be available on github?

kasperk81 avatar Sep 08 '24 18:09 kasperk81

Not at this time, no.

baronfel avatar Sep 08 '24 18:09 baronfel

it will have an open-source parser

even if it's not fully ready it can still be open sourced today and people can help fixing the remaining bugs in a few hours. someone did similar work starting five months ago https://github.com/winscripter/Slnx so they can collaborate with community

kasperk81 avatar Sep 12 '24 06:09 kasperk81

dotnet test also parses sln files to find test projects and fails to work with slnx, fyi.

ssg avatar Sep 19 '24 19:09 ssg

Still waiting for this. Stuff like this makes it feel like Microsoft still sees non-Visual-Studio users of .NET as second-class citizens; which is disappointing after all these years.

aradalvand avatar Sep 20 '24 09:09 aradalvand

Still waiting for this. Stuff like this makes it feel like Microsoft still sees non-Visual-Studio users of .NET as second-class citizens; which is disappointing after all these years.

It's not like slnx works very well in VS. It's an experimental preview feature. And even with VS, CI requires dotnet CLI.

ite-jankl avatar Sep 20 '24 10:09 ite-jankl

Stuff like this makes it feel like Microsoft still sees non-Visual-Studio users of .NET as second-class citizens

It's not a non-preview feature yet in VS (I believe the release notes still don't mention it at all), and the API isn't considered stable.

chucker avatar Sep 20 '24 10:09 chucker

A status update for those following this issue:

  • The new parser is now open-source, and the repository can be found at https://github.com/microsoft/vs-solutionpersistence
  • MSBuild has a PR that enables support for building slnx files at https://github.com/dotnet/msbuild/pull/10794
  • We will have to wait for the slnx parser to be onboarded to source-build-externals in this PR before we can merge the MSBuild PR.
  • The SDK codeflow for the slnx functionality is https://github.com/dotnet/sdk/pull/44283

Our current plan is to try to merge this change and get support for building slnx files from the .NET SDK in a preview release of the 9.0.200 SDK, but that is subject to some timelines that we are still navigating.

baronfel avatar Oct 10 '24 18:10 baronfel

Proposal:
dotnet new sln -n lodestone -f slnx short for dotnet new solution --name lodestone --format slnx

In .NET 9.0, the default format (-f) is still .sln. However, by .NET 10.0 or 11.0, consider making .slnx the default format to encourage developers to upgrade to Visual Studio versions that would fully support it by then. This gradual switch ensures backward compatibility while pushing adoption of the cleaner, modern solution format.

kasperk81 avatar Oct 10 '24 19:10 kasperk81

@kasperk81 From the issue description:

One major question: should we supplant the existing solution template to create an slnx file? Should the old solution format be accessible via a template parameter?

Switching the defaults for .NET 10 is a fine plan, and is what we had been internally considering anyway.

baronfel avatar Oct 10 '24 20:10 baronfel

you can't build a solution from the cli with slnx files right now with 9.0 RC2... when is this fixed? Is there a work around? It makes the slnx standard useless in all but the hello world case.

JohnGalt1717 avatar Oct 11 '24 19:10 JohnGalt1717

you can't build a solution from the cli with slnx files right now with 9.0 RC2... when is this fixed? Is there a work around? It makes the slnx standard useless in all but the hello world case.

Reading is fundamental. Not sure how you can work on software legitimately when it's spelled out plain as day above that they are going to merge it into the 9.0.200 SDK which means based on past releases you won't be able to until the February 2025 release.

buvinghausen avatar Oct 11 '24 19:10 buvinghausen

@JohnGalt1717 here’s the order of tasks:

  1. merge this: https://github.com/dotnet/source-build-externals/pull/386
  2. run dotnet nuget push so the nuget package in the badge doesn’t show 404: https://github.com/microsoft/vs-solutionpersistence/blob/main/README.md
  3. merge this: https://github.com/dotnet/msbuild/pull/10794
  4. sdk needs codeflow from the msbuild update
  5. implement it in dotnet sln in this repo
  6. lobby for 9.0 backport

if you have time prepare a pr for task 5

kasperk81 avatar Oct 11 '24 19:10 kasperk81

you can't build a solution from the cli with slnx files right now with 9.0 RC2... when is this fixed? Is there a work around? It makes the slnx standard useless in all but the hello world case.

The feature is scheduled for 9.0.2xx. 9.0 RC2 is actually 9.0.100-rc.2.

philipp-naused avatar Oct 14 '24 07:10 philipp-naused

first version with the initial support is out https://github.com/dotnet/sdk/blob/main/documentation/package-table.md (9.0.2xx column)

for now, manually write slnx

dotnet new console -o ./testapp
cd testapp
echo '<Solution><Project Path="testapp.csproj"/></Solution>' > testapp.slnx
dotnet bulid testapp.slnx

kasperk81 avatar Oct 18 '24 19:10 kasperk81

Understanding the support policy for .NET favors the even releases, once this is out in the wild with the the .NET 9 release, will the slnx support be brought back into .NET 8, or will we have to wait for 10 for LTS support with slnx in the sdk?

The slnx format being so much simpler could be very useful for me with complex CICD scenarios - thinking of dynamic slnx files on the fly to target unit tests projects based on code changes, things of that nature. As we are currently bound to the LTS release for maintainability reasons, use of this feature for our team would possibly depend on it being brought into .NET 8 (or wait until 10) as we try to target the same sdk as our target framework (especially for containerized builds and tests where we start with the sdk base image).

richshadman avatar Nov 01 '24 18:11 richshadman

The slnx format being so much simpler could be very useful for me with complex CICD scenarios - thinking of dynamic slnx files on the fly to target unit tests projects based on code changes, things of that nature.

Semi-OT, but it sounds like https://github.com/microsoft/slngen might help you.

chucker avatar Nov 01 '24 18:11 chucker

that also doesn't support slnx yet https://github.com/microsoft/slngen/issues/585

kasperk81 avatar Nov 01 '24 18:11 kasperk81

@kasperk81 I know. But the CI/CD, unit test, etc. use case sounded a bit like what slngen is trying to solve.

chucker avatar Nov 01 '24 18:11 chucker

I'm actually solving it currently with dotnet new sln and dotnet sln add - works well enough for now. however generating the solution file without being dependent on dotnet cli would give us more flexibility, and slnx is a much simpler format towards that end. Also easier to parse for a human.

richshadman avatar Nov 01 '24 20:11 richshadman

@richshadman have you considered using MSBuild projects that point to the projects you care about (maybe using Microsoft.Build.Traversal, instead of synthesizing solutions? Then you could have MSBuild conditions, and it'd probably be generally easier to debug.

rainersigwald avatar Nov 01 '24 22:11 rainersigwald

Microsoft.Build.Traversal

It did not occur to me that projects could be used this way, I mean at this point what is the purpose of slnx/sln file when you can have a "meta project" (which already supports https://github.com/microsoft/vs-solutionpersistence/issues/61 as well)?

alrz avatar Nov 01 '24 22:11 alrz