Harbor 2.0.2 CSRF Token Invalid

[matthewdevenny]

Updated Harbor from 1.10.4 to 2.0.2 - using the harbor helm chart. I have Okta OIDC as my login provider. Anything that is a POST in the UI results in a CSRF token invalid message. I have SSL termination on an AWS ALB. I see this in both Safari and Chrome. No issues prior to updating to 2.0.2

[reasonerjt]

Hi, In 2.x the CSRF mechanism has been reworked. This may due to some settings in the ALB makes it drop some headers/cookies.

Does the CSRF error only happens to regular user that was authenticated via OIDC?

Does admin user also see this problem?

Could you please also capture the failed request via your browser? and the log of harbor-core?

[matthewdevenny]

Hi, In 2.x the CSRF mechanism has been reworked. This may due to some settings in the ALB makes it drop some headers/cookies.

Does the CSRF error only happens to regular user that was authenticated via OIDC?

Does admin user also see this problem?

I can't login as the admin local user anymore on the web ui - I get a 403 (I can docker login still)

{
    "errors": [
        {
            "code": "FORBIDDEN",
            "message": "CSRF token invalid"
        }
    ]
}

Could you please also capture the failed request via your browser? and the log of harbor-core?

I don't see the POST in the harbor-core logs. Here is the request captured in the browser (I removed the jwt and changed the domain name to example.com)

Summary
URL: https://harbor.example.com/api/v2.0/projects
Status: 403
Source: Network
Address: <ip>:443
Initiator: 
harbor.example.com:1:780


Request
:method: POST
:scheme: https
:authority: harbor.example.com
:path: /api/v2.0/projects
Content-Type: application/json
Pragma: no-cache
Accept: application/json
Accept-Language: en-us
Accept-Encoding: gzip, deflate, br
Cache-Control: no-cache
Host: harbor.example.com
Origin: https://harbor.example.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.2 Safari/605.1.15
Referer: https://harbor.example.com/harbor/projects
Content-Length: 70
Connection: keep-alive
Cookie: __csrf=tLJZRYSJ8TswdS5AZoZCJcl8Q0XWeq0qffdDZQqooZ2eb8kgu6UPyTxSC5s0aEKcWgOLr4uP2KYRHMR7fxyiaA==; _gorilla_csrf=MTU5NzA4MTQwM3xJa3QwTWxGYVZEaHpMM1pKVFVwNVdHSlZkVFJCZFZwT0wzbFBjR1E1V0ZkTllrOTFTRWh1VnpCQkwxVTlJZ289fNwwqBCOvViZRZcSf2HO_DkUftrgndAtuU-UqaQ1Uurz; sid=7cd520c95e6f5baac941d4d709600465; okta-jwt=<okta-jwt-redacted>; intercom-session-cfy7ppqw=; ajs_anonymous_id=%22df5b2259-4641-454a-9c0d-6d4605d2bc41%22

Response
:status: 403
Set-Cookie: __csrf=fqGWVJ/aVU3OMB8Sj7+ZDTtksuu5xIEI5AomDg9IYflUfAYxoParv8IXOsndUZm0qBt6AeQx9ISI4aEQevxiDA==; Path=/; Secure; SameSite=Strict
Content-Type: application/json; charset=utf-8
Content-Length: 65
Date: Mon, 10 Aug 2020 17:48:36 GMT
x-request-id: 0378d79b00dab72239b1a931604b1d73
Server: nginx/1.14.0

Request Data
MIME Type: application/json
Request Data: {"project_name":"foo","metadata":{"public":"true"},"storage_limit":-1}

[reasonerjt]

@boxboatmatt I don't think it is relevant to OIDC.

For the request to be successful the request has to carry the header

X-Harbor-CSRF-Token: eN+pJhySPcY/PuWz2TrgYvkrH4lqLb+/od9/yUC378mVFSr3iRoK2m/cCAdI85zspbOBd0XIGr2MWKiF0gn4cg==

And UI code will access the cookie __csrf to read the value and attach this header. Note that there is SameSite attribute in the cookie.
b/c you are using an extra LB, it's possible the SamteSite=strict makes the UI code unable to access the cookie and attach the header.

How you configured the ALB to proxy the request to nginx? I have not used ALB, but is it possible the way it proxy the request makes it look like cross-site request? If you access the nginx directly, does it work?

[matthewdevenny]

@reasonerjt The ALB is not configured to drop any header fields. And the header is not being set by the UI the majority of the time... However troubleshooting today with the developer console open to watch network traffic, I have seen the ui get in a state where it is setting the X-Harbor-CSRF-Token in the request and then it does succeed but I have no idea what is causing the UI to bounce in and out of setting that request header. When it happens the UI will work until you refresh the page or navigate to another section of the UI.

[dolv]

The same issue here. We upgraded from 1.10.1 to 2.0.2 and harbor does not work with EnableXSRF = true in app.conf. We need to disable this feature or wait until this issue is fixed. We have Keycloak as OIDC provider.

[dolv]

Admin user is not able to even log-in with EnableXSRF = true

[dolv]

More findings: if we send the X-Harbor-CSRF-Token to harbor host using curl - it appears in logs on the ingress-controller. But when using UI the X-Harbor-CSRF-Token is absent...

[reasonerjt]

@boxboatmatt

UI sets whatever in cookie __csrf to header X-Harbor-CSRF-Token please see my comment: https://github.com/goharbor/harbor/issues/12676#issuecomment-671904558 Is it possible it's due to the "SameSite" attribute?

@dolv The backend only checks the CSRF token for request coming from UI. Please refer to comments above and capture requests as was did in: https://github.com/goharbor/harbor/issues/12676#issuecomment-671501995

[matthewdevenny]

@reasonerjt harbor is installed with EXT_ENDPOINT matching the DNS hostname.. The ingress is configured to use-forwarded-headers.

Also this only happens when OIDC is enabled. On the same cluster, I just installed an identical 2nd copy of harbor at harbor2.domain and with no OIDC provider configured I can login and perform any POST/PUT... In the requests you can see X-Harbor-CSRF-Token being set.

[zdhamasha]

Running into same issue, any updates ??

[SlashTeen]

encounting the same issue, any updates?

[SlashTeen]

Hi, In 2.x the CSRF mechanism has been reworked. This may due to some settings in the ALB makes it drop some headers/cookies.

Does the CSRF error only happens to regular user that was authenticated via OIDC?

Does admin user also see this problem?

Could you please also capture the failed request via your browser? and the log of harbor-core?

encounting the same issue, any updates?

[savealive]

I've got the same issue. Harbor 2.0.2 on Kubernetes with nginx-ingress (aws classic LB, tcp backend proto, ssl termination on LB). Getting CSRF token invalid when attempting to login with OIDC for the first time and set username. Local admin user login is successful.

With enabled EnableXSRF = true in app.conf admin user is also unable to login with message 2020-09-06T02:11:36Z [DEBUG] [/server/error/error.go:58]: {"errors":[{"code":"FORBIDDEN","message":"CSRF token invalid"}]}

[ianfergu]

Same issue... any updates?

[enderv]

Was running into this as well using harbor helm charts. We finally got it working when we trimmed our xsrf key in the values. It used to be 40 characters now there is a limit of 32. If the key is over the limit core will generate its own and cause this.

[matthewdevenny]

The xsrf key is already set to 32 characters as part of the helm chart update and shortening it has no affect on my issue

[matthewdevenny]

However - I did just update to harbor 2.1.0 and this problem now seems to be resolved in this release

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[xcloud2021]

I run into this issue in 2.1.3: {"errors":[{"code":"FORBIDDEN","message":"CSRF token invalid"}]}

but, I can docker login to harbor

[forbearing]

my expose type is nodePort. modify the values.yaml file, "externalURL: https" change to "externalURL: http"

before: externalURL: https://10.240.11.10:30002 after: externalURL: http://10.240.11.10:30002

and then reinstall harbor by helm

[mattjamese]

if your external url is set as core.harbor.domain, run below command and it worked for me. helm upgrade local-harbor harbor/harbor --set externalURL=http://core.harbor.domain

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[JianfuYang]

any upate? i get same problem when i used UI API test by admin

[JianfuYang]

I run into this issue in 2.2.1: {"errors":[{"code":"FORBIDDEN","message":"CSRF token invalid"}]}

I can docker login to harbor，and I can login to harbor by WebUI. But, when I test the API by SwaggerUI, I alway got this problem。By the way, the GET Method running normal.

[YAMLcase]

if your external url is set as core.harbor.domain, run below command and it worked for me. helm upgrade local-harbor harbor/harbor --set externalURL=http://core.harbor.domain

This did not work for us. We're still getting the {"errors":[{"code":"FORBIDDEN","message":"CSRF token invalid"}]} on PUTS and POSTS

[B3DTech]

This is still an issue. Deployed via Helm, using nginx Ingress, cluster IP. When entering username/password set in the values, I get "Invalid user name or password". I can see the x-harbor-csrf-token in the header. In the browser console, I see: Strict-Transport-Security: The connection to the site is untrustworthy, so the specified header was ignored. Followed by a 401: Unauthorized.

{"errors":[{"code":"FORBIDDEN","message":"CSRF token invalid"}]}

The core pod shows:

2021-08-05T17:59:33Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /c/login

2021-08-05T17:59:33Z [DEBUG] [/server/middleware/security/unauthorized.go:28][requestID="08fe5d60443cce0ad9252eec77fc294f"]: an unauthorized security context generated for request POST /c/login

2021-08-05T17:59:33Z [DEBUG] [/core/auth/authenticator.go:147]: Current AUTH_MODE is db_auth

2021-08-05T17:59:33Z [DEBUG] [/core/auth/authenticator.go:160]: Login failed, locking admin, and sleep for 1.5s

2021-08-05T17:59:35Z [ERROR] [/core/controllers/base.go:105]: Error occurred in UserLogin: Failed to authenticate user, due to error 'Invalid credentials'

Seems like everything is working, but the username/password is wrong in the database? I updated the helm deployment with a new password, still didn't help. I did try http vs https for the externalURL, no change in behavior.

[divyanshuch1]

@B3DTech Were you able to solve this login issue?

[B3DTech]

@divyanshuch1 - yes. I had to manually change the password in the database. I followed VMware’s instructions but instead of leaving it blank, I put in the correct password I originally set.

[divyanshuch1]

@B3DTech What SQL command you used to update the password? Thanks in advance

[B3DTech]

https://kb.vmware.com/s/article/76289

Step 5, use your password, not blank.