[bug]: pulsarctl 3.1 branch can't work with client credentials error

[ericsyh]

Background

I tested and found that pulsarctl 3.1 branch release can't work now which will get error like

pulsarctl topics list public/default
[✖]  client error: could not get client credentials: open : no such file or directory

But I the pulsarctl 3.0 branch works well and won't get this client credentials error.

How to reproduce

I upgraded the pulsarctl on local Mac laptop the version is

Release Version: 3.1.0.2
Git Commit Hash: cb9b6f620364a7cea19325ae2c82523d08265520
Git Branch: HEAD
UTC Build Time: 2023-08-30 09:48:42
Go Version: go version go1.18.10 linux/amd64

Then run any kind of regular commands will get the client crediential error

pulsarctl tenants list
[✖]  client error: could not get client credentials: open : no such file or directory

[ericsyh]

@tuteng @zymap @mattisonchao Could you help check this issue?

[ciiiii]

encountered same error

[peica]

change to Release Version: v2.10.5.11-5ce22c is working.

[balticore]

Same here, when using latest version and client_credentials it fails with error:

client error: authentication failed using client credentials: could not exchange client credentials: AADSTS90014: The required field 'scope' is missing from the credential. Ensure that you have all the necessary parameters for the login request.

However, when downgrading to v2.10.5.11-5ce22c it works without issues.

Not that using service account admin credentials works fine with both version, but we need to use client_credentials to scope operations to particular namespaces and topics.

[zymap]

@balticore According to the error, do you have the scope in the client credential file?

[Nikolajls]

As I'm wokring with @balticore I'll ad more details since we've encountered this again with multiple people after updating to 3.1+

This is with latest 3.3.0.5

qa_clients.json:

{
    "type": "client_credentials",
    "client_id": "My random guid",
    "client_secret": "VERYSECRETGOESHERE",
    "issuer_url": "https://login.microsoftonline.com/TenantId/v2.0",
    "scope": "api://PulsarIdentityGuid/.default"
}

pulsarctl context set "sn-qa-clients" `
	--admin-service-url "https://adminserviceurl `
	--issuer-endpoint "https://login.microsoftonline.com/TENANTID/v2.0" `
	--audience "api://PulsarIdentityGuid" `
	--scope "api://PulsarIdentityGuid/.default" `
	--key-file "file://qa_clients.json"

Oauth2 activate and a last-message id

[balticore]

As @Nikolajls mentioned yes scope is part of the parameters and exactly same command works with version v2.10.5.11-5ce22c

[RobertIndie]

@Nikolajls From my understanding, the issue that https://github.com/apache/pulsar-client-go/pull/1244 fixes is not the same as this one, right? They are not the same errors.

[Nikolajls]

@RobertIndie

The fix i proposed in that PR would make at least @balticore 's comment work( and we have multiple colleagues getting the same issue)

You're correct that the initial error from @ericsyh I've not seen, but when I debugged through Pulsarctl code all the way through pulsar admin, pulsar auth and the the code I modified in the PR i saw that would fix the "required field scope is missing" issue, that it seems multiple people are experincing.

The call chain sort of is: https://github.com/streamnative/pulsarctl/blob/ddb962b0c079c199795796872a7110d4b5234120/pkg/cmdutils/config.go#L204

https://github.com/apache/pulsar-client-go/blob/df6a15c8548271f521d9615871899bd8d8678ed1/pulsaradmin/pkg/admin/admin.go#L68

https://github.com/apache/pulsar-client-go/blob/0ac542cb90ab5e4c4319c760c1a25a3d39182f54/pulsaradmin/pkg/admin/auth/provider.go#L75

https://github.com/apache/pulsar-client-go/blob/0ac542cb90ab5e4c4319c760c1a25a3d39182f54/pulsaradmin/pkg/admin/auth/oauth2.go#L76

https://github.com/apache/pulsar-client-go/blob/0ac542cb90ab5e4c4319c760c1a25a3d39182f54/pulsaradmin/pkg/admin/auth/oauth2.go#L84C22-L84C53

My change would then make so the scope is read from the actual oauth2 config key file. https://github.com/apache/pulsar-client-go/pull/1244/files#diff-4b3210cc326df5fdf66b95b8b7a6d4721ab73fbf43b4aa05d324b046d9fa321eR70

It could also be modified to work by changing so: NewAuthenticationOAuth2WithDefaultFlow has a possiblity of parsing in the scope from the config that is used by the context, but currently it only supports the issuer and the keyfile.