add support for harbor2 OpenID Connect (OIDC) users

[Vad1mo]

I am trying out again docker-pushrm with now more realistic scenario.

My user vad1mo is Project Admin on Project c8n. This Role is needed in Harbor to be able to edit the Project Info in the UI. However it seems that this role is not sufficient to update the project info via the API.

It might be also the case that we are using Harbor2 with OIDC, I verified in the API explorer that the user is able to do operations with the username/password, where password is the one taken from harbor->user->profile and not OIDC.

Status: Downloaded newer image for chko/docker-pushrm:1
DEBU root cmd init config                         
DEBU home dir: /                                  
DEBU subcommand "pushrm" called                   
DEBU Using target: c8n.io/c8n/portal:latest       
DEBU using README file: /myvol/README.md          
DEBU server: c8n.io                               
DEBU namespace: c8n                               
DEBU repo: portal                                 
DEBU tag: latest                                  
DEBU repo provider: harbor2                       
DEBU Harbor2.GetAuthident called                  
DEBU using credentials for user vad1mo from generic env var 
DEBU Using Docker creds: vad1mo ********          
DEBU Harbor2.Pushrm called                        
DEBU push readme, response body: {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]} 
DEBU push README, status code: 401                
DEBU error pushing README, bad status code for response: 401 Unauthorized. Server responded: "UNAUTHORIZED - unauthorized" 
ERRO error pushing readme to repo server. See error message below. Run with "--debug" for more details. 

error pushing README, bad status code for response: 401 Unauthorized. Server responded: "UNAUTHORIZED - unauthorized" 

[Vad1mo]

I tried now the api explorer and the generated CURL, It works for the user vad1mo. It seems to be an issue on the docker- pushrm side rather than harbor.

I didn't see any X-Harbor-CSRF-Token in the code, it seems that docker-pushrm needs to honor the OIDC flow.

curl -X PUT "https://c8n.io/api/v2.0/projects/c8n/repositories/portal" -H "accept: application/json" -H "X-Request-Id: 8ju7378fh3f834h8" -H "authorization: Basic XXXXXXXX" -H "Content-Type: application/json" -H "X-Harbor-CSRF-Token: XXXXXX" -d "{ \"description\": \"TEST WORKS\"}"

[christian-korneck]

thanks, that’s a good catch. I’ve indeed only tested against local harbor users so far. (Just tried on demo.goharbor.io and a local instance and it worked there). Most likely we need a csrf header.

I’ll sign up on c8n.io tonight and test it.

[christian-korneck]

@Vad1mo what kind of user/pass are you using in the above curl command (in the basic auth header)? is this a local user or an oidc user? what do you use as password? the cli secret? something else? thanks

[Vad1mo]

Yes the user is OIDC but I am using the users CLI secret from the user profile.

This is also how I used it on the API Portal to login with username + CLI secret on https://c8n.io/devcenter-api-2.0

[christian-korneck]

hmm...strange, on the c8n.io api portal I don't seem to be able to make any authenticated api calls (i.e. I'm logging in with user chris and the cli secret and trying to execute the /users/current request but getting 401 unauthorized).

[christian-korneck]

ok, it now works after I have changed the cli secret to a manually generated shorter one. Also oddly I can't change it back to the old auto-generated password (getting an error in the UI). First guess is there's a problem with the length? (UI only accepts 31 characters when setting the password but the auto-generated one was 32 characters long).

[christian-korneck]

ok, I've played around a bit and I have the impression that there's a server side issue that I can't work around client side (glad if you prove me wrong 😃).

It appears to me that you can only get a valid csrf token by logging into c8n.io in the browser (via auth0). Then - and only for the session (also time limited) - you're possessing 3 cookies, of which you need two to be able to make a successful HTTP call:

• sid (needed)
• __csrf (needed)
• _gorilla_csrf (not needed? not sure what it's for?)

When you copy them from a browser session you're able to make a curl call like:

curl -H 'Cookie: sid=XXXX; __csrf=XXXX' -H 'accept: application/json' -H 'authorization: Basic XXXX' -H 'x-harbor-csrf-token: XXXX' 'https://c8n.io/api/v2.0/users/current'

where

• sid = sid (cookie value)
• x-harbor-csrf-token = __csrf (cookie value)
• authorization: Basic = base64 of <username>:<cli-secret> (needed in addition to the session cookie)

Having said that, you can get all three cookies also using basic auth, but they don't allow you to make an api call that requires auth:

(curl -c file.txt saves the cookies from the response to file and curl -b file.txt loads them from file for the request)

Make a request and we get a sid cookie:

$ curl https://c8n.io/api/v2.0/users/current -c cookies.txt
{"errors":[{"code":"UNAUTHORIZED","message":"UnAuthorize"}]}
$ cat cookies.txt
#HttpOnly_c8n.io	FALSE	/	FALSE	0	sid	XXXX

Make another request and send the stored cookie:

$ curl https://c8n.io/api/v2.0/users/current -c cookies.txt -b cookies.txt
{"errors":[{"code":"UNAUTHORIZED","message":"UnAuthorize"}]}

The server responds with the __csrf and _gorilla_csrf cookies:

$ cat cookies.txt
c8n.io	FALSE	/	TRUE	0	__csrf	XXXX 
#HttpOnly_c8n.io	FALSE	/	TRUE	1599291933	_gorilla_csrf	XXXX
#HttpOnly_c8n.io	FALSE	/	FALSE	0	sid	XXXX

But when we try to use them (in exactly the same request as with the cookies from the browser session before), they don't work:

curl -H 'Cookie: sid=XXXX' -H 'accept: application/json' -H 'authorization: Basic XXXX' -H 'x-harbor-csrf-token: XXXX' 'https://c8n.io/api/v2.0/users/current'
{"errors":[{"code":"UNAUTHORIZED","message":"UnAuthorize"}]}

So, why does it work with the API Explorer (Harbor bundled Swagger UI)? Because it runs in the browser and sends the cookies if they exist. If you try to use the API Explorer from a browser session where you haven't logged in with auth0 (or in which the cookies have expired) it doesn't work either (even if you put the username + cli secret in the swagger "login" form).

An API call with the API Explorer looks like (not as simple as the curl command that it generates):

curl -H 'Host: c8n.io' -H 'Cookie: sid=XXXX; _gorilla_csrf=XXXX; __csrf=XXXX' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:80.0) Gecko/20100101 Firefox/80.0' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'accept-language: en-US,en;q=0.5' -H 'dnt: 1' -H 'upgrade-insecure-requests: 1' -H 'cache-control: max-age=0' -H 'te: trailers' --compressed 'https://c8n.io/api/v2.0/systeminfo'

I'm not really familiar with harbor OIDC but from googling I believe to use the API with an OIDC provider you would need an OIDC token from the OIDC provider:

"You need to id token from the OIDC provider you configured." source

Here's some infos in the FAQ. And there's also an example how to get it from the Keycload OIDC provider: Looks like an API request against Keycloak's api.

So I assume that if c8n.io would somehow show users their OIDC token then users could use it with docker-pushrm. I don't think it can be used to log in with Docker, so probably a change to docker-pushrm would still be needed to enable api keys for the harbor2 provider.

I'm not sure if fetching the OIDC token is a standardized thing that is the same for all OIDC providers? (if so it might be possible to add it to docker-pushrm, i.e. by showing users an auth link that they need to open in the browser. But I assume such a webapp would need to be hosted by the owner of the realm, i.e. c8n.io).

@Vad1mo do you maybe have any idea how I could fetch the OIDC token for my c8n.io user?

[christian-korneck]

documented this as known limitation and changed issue label from bug to enhancement

[Vad1mo]

Thank you @christian-korneck for your investigation. Thats a bummer!

"Service accounts" are on the roadmap. In some issue comment it noted that this will contain some sort of API Keys.

Nevertheless, I will investigate further on how to get it working with AUTH0 for a normal user, The requested scopes openid,offline_access,profile,email should permit to get an ID Token.

[Vad1mo]

API Keys are on the Roadmap for 2.2 see https://github.com/goharbor/harbor/issues/13093

[christian-korneck]

looks like harbor v2.2.0-rc1 has been released today, with 8 new APIs for robot accounts. I’ll try it soon.

[Vad1mo]

Yes, I have seen it this morning. Recalling the last demo I have seen regarding 2.2 it might still not work to update the Docs via the Robot account.

However it will be quite easy to extend the permission, as they plan it to make it more flexible allowing to define custom permission.

What we could do is to setup a test 2.2 instance quickly and see if it is working and if not crete a PR to harbor adding this capability.

[christian-korneck]

some related links (will look into this again soon'ish) https://github.com/goharbor/harbor/issues/14145#issuecomment-781006533 https://github.com/goharbor/harbor/blob/main/src/common/rbac/const.go