In-Process NuGet doesn't support authentication

[bjorkstromm]

In-Process NuGet don't support authentication. It should support obtaining credentials just like nuget.exe

Hints: https://github.com/NuGet/NuGet.Client/tree/dev/src/NuGet.Clients/NuGet.Credentials https://www.nuget.org/packages/NuGet.Credentials/

[bjorkstromm]

@Insire did you ever had a chance to look at this?

Conversation here

[Insire]

I did take a peak at those resources you linked and i also tried to reproduce the problem reliably. Its been a while now, so the details are somewhat hazy again, but i think i remember that i had to rely on #2014 to reproduce the issue.

So i am not sure how to go forward with this. Easiest for me, would be to wait for #2014 to be merged and then take a look again.

[bjorkstromm]

No worries! We’ll merge the PR targeting #2014 as soon as the .NET Core 2 PR is fixed and 0.26 is released.

If you don’t have time or run into issues, I can fix this issue. Just wanted to avoid duplicate work if you already had a solution for this.

[Insire]

I gladly pick this up again, once 0.26 is released. Can you summarize what the CredentialService needs to be able to do? And what, if any additional, tasks have to be completed along site the migration, apart from writing unit tests for the CredentialService?

[bjorkstromm]

@Insire once #2085 is merged, you should be able to consume NuGet.Credentials package.

Don't know how to hook it up. You'll probably need to read some source code here to see how to hook it up.

[Insire]

@mholo65 allright, i'll probably take a look at this the coming weekend. Thanks for the pointers.

[Insire]

so after investigating a little bit and starting a PR already, i am wondering what kind of workflows should I/we support?

There is authentication via NuGet.config. Then there is authentication via NuGet.config + a credential provider (what i found and started to add in the PR).

Is there any other way, i might have missed or i should focus on?

[bjorkstromm]

AFAIK, nuget.exe only supports credentials through NuGet.Config, so that would be a good start. If you have something ready, please send PR and we’ll iterate :smile:

[Insire]

Not yet. I am still piecing things together.

[bjorkstromm]

...and now I saw this https://github.com/cake-build/cake/pull/2090 :smile:

[Insire]

While i got your attention - how does the inprocess nuget client work? i was expecting to see nuget API calls, but all i found is processes being started via nuget.exe

Edit: this is what i was able to find.

[Insire]

Ok, after some digging, i conclude that the "in process client" still leverages the nuget.exe for interacting with the nuget API, its just that cake has functionality baked in for that, so one doesnt have to write cake tasks to restore packages and so on.

[bjorkstromm]

The link you provided is for nuget.exe, but the in-proc (here https://github.com/cake-build/cake/tree/develop/src/Cake.NuGet/Install) uses the NuGet.Client libraries to accomplish the same thing.

Regarding “restore packages and so on”. That are NuGet-aliases located in Cake.Common, and yes, they leverage nuget.exe.

Cake.NuGet, however, is used for pre-processor directives (#addin, #tool and #load). Here, we have two alternatives for communicating with NuGet API: 1) the old nuget.exe based and 2) the in-process (link above) based.

[Insire]

I still need to properly confirm this, but with using the latest version of cake (0.26.1) and moving the VSTS credential provider right next to the nuget exe and providing a NuGet.config next to them, i can properly access a private nuget feed when setting UseInProcessClient to true in the cake.config

I might not have to do anything for this to work.

[Insire]

@mholo65 so, the goal should be to obsolete the nuget.exe and access the nuget API directly?

[bjorkstromm]

@Insire sorry for not getting back to this sooner.

using the latest version of cake (0.26.1) and moving the VSTS credential provider right next to the nuget exe and providing a NuGet.config next to them, i can properly access a private nuget feed when setting UseInProcessClient to true in the cake.config

Is this working for NuGet aliases only? E.g. NuGetRestore? Without me knowing anything about the VSTS credential provider, I'm still don't think it would work with #addin or #tool pre-processor directive with --nuget_useinprocessclient=true.

so, the goal should be to obsolete the nuget.exe and access the nuget API directly?

Yes, for pre-processor directives. (#tool, #addin, #module, #load) NuGet aliases (NuGetRestore, etc..) would/should still shell out to nuget.exe. In order to obsolete, we should at least support getting authentication from NuGet.Config, as nuget.exe knows how to do this.

[Insire]

Is this working for NuGet aliases only? E.g. NuGetRestore? Without me knowing anything about the VSTS credential provider, I'm still don't think it would work with #addin or #tool pre-processor directive with --nuget_useinprocessclient=true.

@mholo65 the last addin is from a private VSTS nuget feed.

I pulled the source for the nuget libraries from github to get an idea whats in there and how they do things and atleast for the credential stuff, it seemed fairly straight forward. They try to load assemblies that match a certain FileName pattern and map that to an interface. After that its just business as usual, as that interface provides a way to supply the required credentials.

My current idea would be to search for a nuget config upwards the folder tree until i hit the root folder for the solution. Not sure yet how i would transform the name and password from the nuget config into something we can use yet.

I'll take another look next weekend. My weeks are rather busy at the moment.

[bjorkstromm]

@Insire oh nice! Great progress so far!

Now I see, with this PR https://github.com/cake-build/cake/pull/2090 we will locate the NuGet Extensions (CredentialProviders) which will provide the credentials. So this will at least work for VSTS.

[cmenzi]

How to handle authentication for the feeds configured in cake.config? Currently, this is only a ini format and the feeds are separated by ;. #2014

[Insire]

Right now, you can add authentication via nuget.config and if you want custom nuget credential providers (e.g. the one for VSTS). In the future, the goal is to move away from the nuget config (atleast for the in process client) and provide options to authenticate with cake only. If you got any ideas in that regard, i'm all ears.

[p10tyr]

Is this about the popup window that lest you authenticate against DevOps (in my example) the same way Visual Studio does? Do you think this will ever make it in.. or is there another way to authenticate manaully so that NuGet does not need PAT keys (which are a pain in the parse)

[TaffarelJr]

@Insire @devlead I'm trying to pull a Cake Recipe package from a private feed in GitHub Packages. I have a nuget.config file with credentials specified, but the credentials are pulled from environment variables:

    <packageSourceCredentials>
        <GitHub>
            <add key="Username" value="%GITHUB_USER%" />
            <add key="ClearTextPassword" value="%GITHUB_TOKEN%" />
        </GitHub>
    </packageSourceCredentials>

This works in NuGet, but does not seem to work in Cake. I'm guessing perhaps the variable replacement is not happening in the Cake In-Process NuGet execution, but I don't know how to tell. Diagnostic logging doesn't give any information beyond this:

+ dotnet cake --target release --verbosity diagnostic
Looking for modules...
Missing /github/home/.nuget/packages/<my_package>/<version>/<my_package>.<version>.nupkg
  GET https://api.nuget.org/v3/registration5-gz-semver2/<my_package>/index.json
  NotFound https://api.nuget.org/v3/registration5-gz-semver2/<my_package>/index.json 94ms     <-- This is expected
  GET https://nuget.pkg.github.com/<my_organization>/<my_package>/index.json                  <-- Package exists here
  NotFound https://nuget.pkg.github.com/<my_organization>/<my_package>/index.json 196ms       <-- This shouldn't fail
Error: System.AggregateException: Bootstrapping failed for '/__w/<my_app>/<my_app>/build.cake'.
build.cake, line #1: Unable to find package '<my_package>'. Existing packages must be restored before performing an install or update.

I can confirm the proper environment variables are set on the build runner. I tried --nuget_useinprocessclient=false, but the build is running on Linux, and NuGet.exe does not exist. We use dotnet nuget for everything instead.

Can you confirm if this is indeed a supported scenario in Cake the way it is in NuGet? Or is there some way to get additional logging from In-Process NuGet? Or is there some recommended way to install NuGet.exe on Linux? Or is there a custom credential provider that would suit this scenario?

[TaffarelJr]

I just hard-coded some temporary credentials in my nuget.config file, and it seems to work. So it looks like variable substitution is not a feature supported by Cake In-Process NuGet at this time.

Any idea if that could be implemented easily?

[TaffarelJr]

Actually, it appears I was mistaken. This looks to be an issue with GitHub itself. The reason it worked for me above is because I used a personal access token to test with. Running on the CI build runner, it was using a different token, and was failing (as metioned in the thread I linked). When I use a PAT, regardless of environment, the substitution appears to be taking place in Cake.

So scratch my posts, they're unrelated after all. :)