Add path to global.json SDK version lock

[rainersigwald]

Builders of large systems often want to be able to explicitly specify an SDK version and avoid installing the CLI to a machine-global location.

Currently, global.json allows the version lock, but the specified version must be installed in either a machine-global location or a nonstandard location must be specified by the environment variable DOTNET_MSBUILD_SDK_RESOLVER_SDKS_DIR.

That environment means that you must launch Visual Studio from a specific environment to get the downloaded/private SDKs.

There could be an extension to the SDK resolver to respect a path specified in the global.json. Something like

{
    "sdk": {
        "version": "1.0.0",
        "path": "tools/downloadedsdk"
  }
}

could be equivalent to setting DOTNET_MSBUILD_SDK_RESOLVER_SDKS_DIR=%GlobalJsonPath%\tools\downloadedsdk before invoking the resolver.

This would be visible by any invocation (dotnet, msbuild, VS, or MSBuild API) since it's file-based.

[rainersigwald]

@nguerrera I think this might be a cleaner way to address @jaredpar's scenario from https://github.com/Microsoft/msbuild/issues/2095.

[jaredpar]

I'm not sure this would work for Roslyn: we don't use the CLI to build. The CLI won't be able to build Roslyn anytime in the foreseeable future due to the number of desktop specific MSBuild extensions our build contains: WPF, VS SDK, SWIX, etc ...

[nguerrera]

@rainersigwald This is exactly what I had in mind when I wrote:

(Now, that actually overlaps with another feature that's evolving and we will likely land in a place where you can edit global.json to get this behavior without setting any environment variables, but the mechanism here applies more generally to arbitrary resolvers with arbitrary input.)

There wasn't an issue for this yet (thanks for starting one), just mail threads so far, with a meeting scheduled soon to hash out the details. There's an existing mechanism called "multilevel lookup" in hostfxr that adds a well-known user profile location to the sdk search. This would be an evolution of that.

I did not expect Microsoft/msbuild#2095 to be controversial, but I understand your concerns now. I wasn't intending for this and that to be mutually exclusive. I was imagining that being able to pass arbitrary data down to arbitrary resolvers would be generally useful.

@jaredpar

I'm not sure this would work for Roslyn: we don't use the CLI to build.

The resolver called by desktop msbuild will respect global.json too. That's part of it's main purpose: to pick the same msbuild targets for VS that would be used by the CLI msbuild invoked on the same project.

Do you build with CLI on Mac/Linux? If your process installs .NET Core SDK (official name of entire CLI not just dotnet/sdk) to a local location and sets this feature up via global.json, then you can have one mechanism for pinning down the precise set of things that come from the .NET Core SDK across platforms, and not require it to be globally installed anywhere. On Windows, you'd only use the tasks and targets, but if somebody wanted to use dotnet on the portable projects on Windows it would work. That could give you a simple way to reproduce issues that only happen on Mac or Linux official builds on a Windows dev box.

[nguerrera]

This could be equivalent to setting DOTNET_MSBUILD_SDK_RESOLVER_SDKS_DIR

global.json selects the full .NET Core SDK (CLI) used, not just the msbuild SDKs. We don't even have a resolver when we're invoked via dotnet as it will have taken us to a universe where we have matching msbuild SDKs for the given version.

So global.json should instead specify equivalent of DOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR (matching layout of C:\Program Files\dotnet). More granular control over the resolver than that has to go somewhere other than global.json.

[jaredpar]

Can I get more information on global.json: in particular where it lives, how it's found, etc ...?

Do you build with CLI on Mac/Linux? I

Yes and depending on the answer above it seems like a problem. The path to the downloaded SDK is going to be very different between the two systems.

{
    "sdk": {
        "version": "1.0.0",
        "path": "tools/downloadedsdk"
  }
}

In order for this to solve the problem the SDK essentially has to be within the cone of the global.json. Guessing that mean that it has to be within the cone of the repo. No other tool has that requirement for our build and it's going to take some doing to make that work.

Still overall feels clunky. MSBuild properties can today define everything about my build. Makes it dynamic, very easy to manage. Now we have JSON (again) and I have to write generators to translate between my MSBuild files and this new JSON file.

[tmat]

In order for this to solve the problem the SDK essentially has to be within the cone of the global.json. Guessing that mean that it has to be within the cone of the repo. No other tool has that requirement for our build and it's going to take some doing to make that work.

I think this will be easy to make work in our repos. The only thing that needs to be restored to a directory specified in global.json is the immediate SDK used by the projects (which is not necessarily .NET SDK). In RepoToolset repos this is RepoToolset SDK (projects use <Project Sdk="RoslynTools.RepoToolset"/>).

Our current global.json for symreader repo looks like so:

{
  "sdk": {
    "version": "2.1.100-preview-007366"
  },
  "msbuild-sdks": {
    "RoslynTools.RepoToolset": "1.0.0-beta2-62705-02"
  }
}

Let's say we could add the path like so:

{
  "sdk": {
    "version": "2.1.100-preview-007366"
  },
  "msbuild-sdks": {
    "RoslynTools.RepoToolset": { "version": "1.0.0-beta2-62705-02", "path": ".sdk" }
  }
}

and this would restore the RepoToolset to {repo-root}\.sdk directory (next to global.json)

The RepoToolset restores .NET SDK by importing it in its sdk\Sdk.props file:

<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />

at which point we can use arbitrary msbuild properties to specify the path to restore to.

[jaredpar]

@tmat that makes sense. I think that's workable.

[abelbraaksma]

This would be a very welcome addition to the rather inflexible global.json config options. Currently, not being able to point to a local path, and the VS installer constantly uninstalling previous SDK versions, makes it pretty hard to do any decent SDK-dependent development (like the F# repo project files, see dotnet/fsharp#10193, which should be fixed on 3.1.302, but the VS installer removes this in favor of 3.1.402).

Since it is trivial to create an msbuild script that downloads a specific SDK version and installs it locally, it would be very, very nice if such behavior could be baked into global.json as well:

• specify a local path, or just use token "local": true and standardize the path to be .dotnet
• if SDK is absent in that location, VS installs the SDK automatically upon fist opening the solution

All in all, that would create a much better experience then the now rather hard-to-understand "cannot load project or solution" message, which baffles newcomers and experienced users alike...

[RussKie]

This will have a significant positive impact on the developer experience for .NET repositories, such as https://github.com/dotnet/runtime/, https://github.com/dotnet/winforms, etc. Currently it is impossible to double click on a solution file to open the solutions in VS, because we build against nightly SDKs, which are located in custom locations and are forever changing.

[KirillOsenkov]

As a workaround, I keep around a .bat file with this example content:

set PATH=C:\msbuild\.dotnet;%PATH%
set DOTNET_INSTALL_DIR=C:\msbuild\.dotnet
set DOTNET_MULTILEVEL_LOOKUP=0

If you go to the repo directory (C:\msbuild in my case) and run this .bat file, then run devenv.exe mysolution.sln, it will open with these environment variables set and the SDK resolution by MSBuild will use the .dotnet that's private to the repo.

[RussKie]

That's what other dotnet repos are doing AFAIK in some shape or form. And this is not the best developer experience. It also makes it hard to run ad-hoc tests...

[KirillOsenkov]

Oh, you don't have to convince me ;) I know it's terrible and I've spilled blood, sweat and tears to even arrive at this workaround by debugging deep into the SDK resolution process.

Lots of pain and friction caused by this over the years.

[RussKie]

To include other points form offline chats and summarise:

There a number of use cases that really benefit from this (or comparable) functionality:

1. Developer experience building and contributing to .NET repos, such as https://github.com/dotnet/runtime/, https://github.com/dotnet/winforms, etc. Currently it is impossible to double click on a solution file to open the solutions in VS, because we build against nightly SDKs, which are located in custom locations and are forever changing. This make it unnecessary hard and complicated for our (especially new) contributors. Right now a lot of our tooling is cli-centric (which may be expected for non Windows platforms and CI/CD scenarios), but our Windows/Visual Studio devex is very cumbersome. Repos have custom scripts or instructions that must be run in order to open solutions in Visual Studio.
2. Local testing - it is very difficult to run customer repos against the nightlies. These builds must either be installed globally, or one has to jump through hoops and run custom scripts to bootstrap a sample.

If it is difficult to provide the download functionality, we could probably do this in staggered approach, i.e. provide the ability to resolve an SDK from a custom location.

• If the path is not specified - continue doing what we do today.
• If the path is invalid - fail.

Maybe add another property to specify an error message that can describe how to bootstrap or where to download from. E.g.:

{
  "sdk": {
    "version": "2.1.100-preview-007366"
  },
  "msbuild-sdks": {
    "RoslynTools.RepoToolset": { 
      "version": "1.0.0-beta2-62705-02",
      "path": ".sdk",
      "instructions": "Run .\restore.cmd to download the latest SDK.",
    }
  }
}

[marcpopMSFT]

@vitek-karas We explored an option offline of just supporting this within the sdk resolver to find the SDK (essentially just DOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR). Doing only that would unblock build but it would leave run (and potentially testing) still blocked as teams like WindowsDesktop have a test app that they would want to build and launch from VS against a version of the runtime listed in the global.json file.

Thoughts on finding the runtime from a path in global.json as well? I think the scenario for dotnet repos would have the runtime and SDK all in the same directory so potentially we only need one path still for both.

[RussKie]

To add to @marcpopMSFT's post.

Starting VS without settings any paths or anything:

Starting VS with env:DOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR settings:

PS C:\Development\winforms> $env:DOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR='C:\Development\winforms\.dotnet'
PS C:\Development\winforms> start .\Winforms.sln

On a second F6 it all build successfully... ¯\_(ツ)_/¯

But I was unable to launch a project from the VS:

[vitek-karas]

From runtime's perspective there's a risk of performance regressions. This would require the host to go looking for global.json (or some other similar file) performing file existence checks. The deeper the folder hierarchy, the more expensive the check. This would have to be done by EVERY application, EVERY time. Given that this feature is expected to be used almost exclusively by developers that doesn't feel like the right tradeoff.

Doing this for SDK only is in theory possible, but could open issues with compatibility between SDK and the necessary runtime/frameworks for it (since we could not rely on the dotnet.exe to be the right one anymore).

I absolutely understand the scenario and why it's important, but so far I was not able to figure out a solution which would not have a negative effect when the feature is not is use. (pay-for-play)

/cc @agocke

[marcpopMSFT]

@RussKie Thoughts on only supporting this for the SDK as originally suggested given Vitek's feedback above? It wouldn't allow for launching test apps from the IDE but would allow for building. It's unclear if tests would work for not.

[danmoseley]

@agocke thoughts? Seems there's agreement that it would be high value to solve this as it will make us and the community more productive in our repos.

[DustinCampbell]

I (think) I like this. :smile: However, pinning the SDK in global.json is already a significant problem within Visual Studio, due to mismatches between the MSBuild engine in Visual Studio and the targets/tasks loaded from the .NET SDK. I have some worry that this might exasperate that problem more.

@KathleenDollard, do you have any thoughts?

[richlander]

Is it possible to start with an ENV solution only? That would mitigate Vitek's concerns. I'd like to reduce the number of places we have to search on disk not invent more.

[agocke]

Having repo-local SDKs make sense and I think is tractable. The team talked about repo-local runtimes and that one seems much more difficult. Not only would it probably be a big perf problem, as more intermediate tools use .NET Core (I hope), it's less and less likely that you want everything to run on local runtime. For instance, would you want new VS components that run on .NET Core to suddenly pick your repo local runtime?

I think SDK is a good place to start, if we can make that work.

[richlander]

If we had a repo root story, then putting a global.json at repo root would be workable and I suspect mitigate Vitek's concerns (depending on how repo-root worked; if it was an ENV, it would work awesome).

[RussKie]

@RussKie Thoughts on only supporting this for the SDK as originally suggested given Vitek's feedback above? It wouldn't allow for launching test apps from the IDE but would allow for building. It's unclear if tests would work for not.

This wouldn't work for us. I can build from command line and don't need VS for that. I'd like a seamless devex for VS: double click sln, F5, debug.

[RussKie]

RE: perf - we can make a call, and only look for .dotnet folder in the same folder as global.json. We have many things that are convention based, we can probably add yet another thing 😊

[agocke]

@RussKie You're confusing the SDK with the runtime. global.json is an SDK configuration option. Right now there's no way to configure the muxer dotnet.exe to look somewhere else for the runtime aside from environment variables. Looking for global.json would run into the same problem -- anything you add to the muxer is overhead on every execution of a dotnet app. We've already removed a bunch of file system accesses from the host startup because they had unacceptable startup penalty. I don't consider adding any new filesystem checks to be an acceptable approach.

[Nirmal4G]

Then, why not do a context separation with global.json and other env checks?

If the file is found along with other env checks, then, the muxer should be in Dev/SDK mode; else it'll be in Runtime/Execution mode. Do what you are doing today with the exec mode or simplify further if possible since every bit of perf is important. But in Dev/SDK mode, do things that improves the developer productivity such as loading dev-specific options (look for new SDK/Runtime paths, etc…) from both the env variables and the global.json file. Almost every project, if it needs customizability, will have a global.json in the root. So, we could reduce the checks to only look for global.json to determine the context.

To add on further, you could have either --mode <dev|run|…> or --dev and --no-dev to force the muxer into a specific context.

[RussKie]

For instance, would you want new VS components that run on .NET Core to suddenly pick your repo local runtime?

This is a good question. I think the answer is "it depends", with likely "yes".

[vitek-karas]

So, we could reduce the checks to only look for global.json to determine the context.

This is the problem - currently the muxer doesn't look for this file when running an app. So doing that would be a perf hit. Note that this is not as cheap as it seems. Basically every non-dev scenario (which is a vast majority of .NET app executions) will not have global.json, but in order to figure that out the muxer would have to look at every directory from where the application is all the way up to the root of the file system. This means it's actually worse for non-dev - since in those cases there will be no global.json and so the muxer will always go all the way up to the file system root (unlike dev scenarios, which would find global.json sooner).

Currently the only reasonable solution I can think of is to require at least 1 environment variable. Something which tells the muxer to "work in the dev mode" or something similar. I don't think we can make the muxer do this work always.

Running VS components on repo-local runtime: I think the answer is "it depends", with likely "yes".

I don't agree. Note that this would apply to all kinds of things. For example today if I type code . from the repo root I get a VS Code which works. If we were to force it to use the repo local runtime, it's possible that for example the C# language service would not work. Let's say the C# language service is currently running on .NET 5 and has the default roll-forward (Minor). This means that it would not find a compatible runtime in the repo-local runtime set. I don't know if the VS Code C# language service carries its own runtime, maybe it does, but any component which doesn't could run into these problems.

Also in the cases of our .NET repos, they frequently use previews or even nightly builds of the runtime as repo-local as they need a relatively fast update cycle to get new features. Running all my dev tools on top of such runtime can easily lead to breaks (we're not perfect, and especially nightly builds are from time to time a bit breaking).

Fortunately this is a moot discussion - even if we were to look for global.json the muxer would use the location of the app to run as the starting point for the search. So things like VS, VS Code and so on would search somewhere in Program Files and not in the repo directory and would not find the file.

[Nirmal4G]

currently the muxer doesn't look for this file when running an app.

Sorry If I've got this all wrong. Doesn't it use different .NET SDK when we have a global.json with a particular sdk version mentioned? If it's not the muxer doing this, then what else?

Running all my dev tools on top of such runtime

Not the case here. As a runtime or a dev tools contributor, I'd want only certain invokes of dotnet to be run in a developer context. Like testing the repo with a just built components of runtime (e.g.: the SDKs, MSBuild or NuGet targets) or in the case of core, winforms and wpf runtime, the entire runtime.

[vitek-karas]

The muxer (dotnet.exe) does two main things:

• Runs an application (dotnet app.dll, or dotnet exec app.dll for example)
• Invokes a CLI/SDK command (dotnet build or dotnet run for example)

When it detects (by looking at the command line basically) that it's to run CLI, it switches to the CLI mode, in which case it will go look for global.json. But if it's running in the "app" mode, it will not do that. The discussion is about modifying the "app" mode to go search for global.json.