netbirdio
Can't access dashboard - Token Invalid, Authentik
I've been looking at similar reports - and I couldn't figure out which one would be best for this one, in the end decided on new one, hopefully all the appropriate ones will be merged.
So I had a working self hosted instance of Netbird with Authentik as a IdP provider. After a while it stopped working with Token Invalid error message... which "magically" fixed itself. But now it stopped working again and I can't access the dashboard (the service itself works, the agents can connect, but I can't do any management atm).
Here's what I see in the logs:
2024-07-28T12:35:58Z DEBG management/server/idp/authentik.go:134: requesting new jwt token for authentik idp manager
2024-07-28T12:35:58Z ERRO [context: HTTP, requestID: a7655341-9864-4855-b005-3fa72ca9b82a] management/server/http/middleware/auth_middleware.go:89: Error when validating JWT claims: unable to get authentik token, statusCode 400
2024-07-28T12:35:58Z ERRO [context: HTTP, requestID: a7655341-9864-4855-b005-3fa72ca9b82a] management/server/http/util/util.go:81: got a handler error: token invalid
2024-07-28T12:35:58Z ERRO [requestID: a7655341-9864-4855-b005-3fa72ca9b82a, context: HTTP] management/server/telemetry/http_api_metrics.go:191: HTTP response a7655341-9864-4855-b005-3fa72ca9b82a: GET /api/users status 401
2024-07-28T12:35:58Z DEBG [context: HTTP, requestID: a7655341-9864-4855-b005-3fa72ca9b82a] management/server/telemetry/http_api_metrics.go:211: request GET /api/users took 305 ms and finished with status 401
2024-07-28T12:35:59Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:112: keys refreshed, new UTC expiration time: 2024-07-28 12:35:59.293866388 +0000 UTC
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/account.go:1667: overriding JWT Domain and DomainCategory claims since single account mode is enabled
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/account.go:1816: Acquired global lock in 8.327µs for user 7
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/sql_store.go:169: took 8 ms to persist an account to the store
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/account.go:1301: looking up user 7 of account cpk3ikv7g7ts73c049h0 in cache
2024-07-28T12:35:59Z DEBG management/server/account.go:1239: account cpk3ikv7g7ts73c049h0 not found in cache, reloading
2024-07-28T12:35:59Z DEBG management/server/idp/authentik.go:134: requesting new jwt token for authentik idp manager
2024-07-28T12:35:59Z ERRO [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/http/middleware/auth_middleware.go:89: Error when validating JWT claims: unable to get authentik token, statusCode 400
2024-07-28T12:35:59Z ERRO [requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418, context: HTTP] management/server/http/util/util.go:81: got a handler error: token invalid
2024-07-28T12:35:59Z ERRO [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/telemetry/http_api_metrics.go:191: HTTP response 859af32c-cfd2-4633-a3b1-2c2bba6b0418: GET /api/users status 401
2024-07-28T12:35:59Z DEBG [context: HTTP, requestID: 859af32c-cfd2-4633-a3b1-2c2bba6b0418] management/server/telemetry/http_api_metrics.go:211: request GET /api/users took 321 ms and finished with status 401
Here's sanitized management.json:
{
"Stuns": [{
"Proto": "udp",
"URI": "stun:mydomain.com:3478",
"Username": "",
"Password": ""
}],
"TURNConfig": {
"TimeBasedCredentials": false,
"CredentialsTTL": "12h0m0s",
"Secret": "secret",
"Turns": [{
"Proto": "udp",
"URI": "turn:mydomain.com:3478",
"Username": "self",
"Password": "someturnpassword"
}]
},
"Signal": {
"Proto": "http",
"URI": "mydomain.com:10000",
"Username": "",
"Password": ""
},
"Datadir": "/var/lib/netbird/",
"DataStoreEncryptionKey": "somekey",
"HttpConfig": {
"LetsEncryptDomain": "",
"CertFile": "/etc/letsencrypt/live/mydomain.com/fullchain.pem",
"CertKey": "/etc/letsencrypt/live/mydomain.com/privkey.pem",
"AuthAudience": "OauthProiderClientID",
"AuthIssuer": "https://authentik.mydomain.com/application/o/netbird/",
"AuthUserIDClaim": "",
"AuthKeysLocation": "https://authentik.mydomain.com/application/o/netbird/jwks/",
"OIDCConfigEndpoint": "https://authentik.mydomain.com/application/o/netbird/.well-known/openid-configuration",
"IdpSignKeyRefreshEnabled": true
},
"IdpManagerConfig": {
"ManagerType": "authentik",
"ClientConfig": {
"Issuer": "https://authentik.mydomain.com/application/o/netbird",
"TokenEndpoint": "https://authentik.mydomain.com/application/o/token/",
"ClientID": "OauthProiderClientID",
"ClientSecret": "",
"GrantType": "client_credentials"
},
"ExtraConfig": {
"Password": "ServiceAccountToken",
"Username": "Netbird"
},
"Auth0ClientCredentials": null,
"AzureClientCredentials": null,
"KeycloakClientCredentials": null,
"ZitadelClientCredentials": null
},
"DeviceAuthorizationFlow": {
"Provider": "hosted",
"ProviderConfig": {
"ClientID": "OauthProiderClientID",
"ClientSecret": "",
"Domain": "authentik.mydomain.com",
"Audience": "OauthProiderClientID",
"TokenEndpoint": "https://authentik.mydomain.com/application/o/token/",
"DeviceAuthEndpoint": "https://authentik.mydomain.com/application/o/device/",
"AuthorizationEndpoint": "",
"Scope": "openid",
"UseIDToken": false,
"RedirectURLs": null
}
},
"PKCEAuthorizationFlow": {
"ProviderConfig": {
"ClientID": "OauthProiderClientID",
"ClientSecret": "",
"Domain": "",
"Audience": "OauthProiderClientID",
"TokenEndpoint": "https://authentik.mydomain.com/application/o/token/",
"DeviceAuthEndpoint": "",
"AuthorizationEndpoint": "https://authentik.mydomain.com/application/o/authorize/",
"Scope": "openid profile email offline_access api",
"UseIDToken": false,
"RedirectURLs": [
"http://localhost:53000"
]
}
},
"StoreConfig": {
"Engine": "sqlite"
},
"ReverseProxy": {
"TrustedHTTPProxies": [],
"TrustedHTTPProxiesCount": 0,
"TrustedPeers": [
"0.0.0.0/0"
]
}
}
Here's sanitized openid config:
{
"issuer": "https://authentik.mydomain.com/application/o/netbird/",
"authorization_endpoint": "https://authentik.mydomain.com/application/o/authorize/",
"token_endpoint": "https://authentik.mydomain.com/application/o/token/",
"userinfo_endpoint": "https://authentik.mydomain.com/application/o/userinfo/",
"end_session_endpoint": "https://authentik.mydomain.com/application/o/netbird/end-session/",
"introspection_endpoint": "https://authentik.mydomain.com/application/o/introspect/",
"revocation_endpoint": "https://authentik.mydomain.com/application/o/revoke/",
"device_authorization_endpoint": "https://authentik.mydomain.com/application/o/device/",
"response_types_supported": [
"code",
"id_token",
"id_token token",
"code token",
"code id_token",
"code id_token token"
],
"response_modes_supported": [
"query",
"fragment",
"form_post"
],
"jwks_uri": "https://authentik.mydomain.com/application/o/netbird/jwks/",
"grant_types_supported": [
"authorization_code",
"refresh_token",
"implicit",
"client_credentials",
"password",
"urn:ietf:params:oauth:grant-type:device_code"
],
"id_token_signing_alg_values_supported": [
"RS256"
],
"subject_types_supported": [
"public"
],
"token_endpoint_auth_methods_supported": [
"client_secret_post",
"client_secret_basic"
],
"acr_values_supported": [
"goauthentik.io/providers/oauth2/default"
],
"scopes_supported": [
"email",
"profile",
"openid"
],
"request_parameter_supported": false,
"claims_supported": [
"sub",
"iss",
"aud",
"exp",
"iat",
"auth_time",
"acr",
"amr",
"nonce",
"email",
"email_verified",
"name",
"given_name",
"preferred_username",
"nickname",
"groups"
],
"claims_parameter_supported": false,
"code_challenge_methods_supported": [
"plain",
"S256"
]
}
Netbird is running inside Docker container, while Authentik in Podman one, on a separate server (with Caddy reverse proxy and Cloudflare).
I'm using Authentik for several other apps and I don't have any issues there (but there's one difference - for other apps I don't use service account setup).
On the side of Authentik I don't see any problems. Here's raw event info:
{
"user": {
"pk": 7,
"email": "myemail",
"username": "myusername"
},
"action": "authorize_application",
"app": "authentik.providers.oauth2.views.authorize",
"context": {
"flow": "someflow",
"scopes": "offline_access openid email profile",
"http_request": {
"args": {
"scope": "openid profile email offline_access api",
"state": "7Cwo6bqD1f",
"audience": "OauthProviderClientID",
"client_id": "OauthProviderClientID",
"redirect_uri": "https://mydomain.com/#callback",
"response_type": "code",
"code_challenge": "somechallenge",
"code_challenge_method": "S256"
},
"path": "/api/v3/flows/executor/default-provider-authorization-explicit-consent/",
"method": "GET",
"user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0"
},
"authorized_application": {
"pk": "somepk",
"app": "authentik_core",
"name": "Netbird",
"model_name": "application"
}
},
"client_ip": "someip",
"expires": "2025-07-28T12:51:42.272Z",
"brand": {
"pk": "somepk",
"app": "authentik_brands",
"name": "Default brand",
"model_name": "brand"
}
}
In credentials / tokens for a user that wishes to access Netbird I see:
Here are provider settings:
Any suggestions howto resolve the issue and get into the management panel are greatly appreciated. At this point I'm just blindly clicking various options as the suggestions in other topics are all over the place - it seems that I'm not the only one who has issues in pinpointing the cause / fix.
If there's some more info needed plz let me know - I'll be happy to provide it.
for the time being I've created a new provider and service account to get into the dashboard, but I fully expect the problem to reappear when token expires.
I updated to Authentik 2024.10.4 this morning and now I am getting this error. All my netbird users can't work. :-(
2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/http/util/util.go:81: got a handler error: token invalid
2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/telemetry/http_api_metrics.go:168: HTTP response 0b96d0df-5391-4083-b654-554e28c5cf10: GET /api/users status 401
I updated to Authentik 2024.10.4 this morning and now I am getting this error. All my netbird users can't work. :-(
2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/http/util/util.go:81: got a handler error: token invalid 2024-11-22T14:04:04Z ERRO [context: HTTP, requestID: 0b96d0df-5391-4083-b654-554e28c5cf10] management/server/telemetry/http_api_metrics.go:168: HTTP response 0b96d0df-5391-4083-b654-554e28c5cf10: GET /api/users status 401
Yeah, same proble here
@ne0YT sadly not. I created new provider and added it to NB. This made it work... until today. With Authentik 2024.10.4 I first couldn't launch at all - got redirection URI error. Initially the tip to change middle option for URI to regex didn't help, but later on it did... But then it brought me back to "Token Invalid" error. Super tired of this. At this point I'm starting to test other IdP . If other options wouldn't break that often or at the very least there will be a working workarounds / pointers what's wrong I'll switch (even though I like the Authentik approach to lots of user / admin facing things).
Here are the latest logs from management:
2024-11-24T14:50:53Z ERRO [context: HTTP, requestID: d0451fc8-98d0-4418-b834-8db0cb66f495] management/server/http/middleware/auth_middleware.go:89: Error when validating JWT claims: 403 Forbidden
2024-11-24T14:50:53Z ERRO [requestID: d0451fc8-98d0-4418-b834-8db0cb66f495, context: HTTP] management/server/http/util/util.go:81: got a handler error: token invalid
2024-11-24T14:50:53Z ERRO [context: HTTP, requestID: d0451fc8-98d0-4418-b834-8db0cb66f495] management/server/telemetry/http_api_metrics.go:168: HTTP response d0451fc8-98d0-4418-b834-8db0cb66f495: GET /api/users status 401
2024-11-24T14:50:53Z DEBG [context: HTTP, requestID: d0451fc8-98d0-4418-b834-8db0cb66f495] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 521 ms and finished with status 401
Please see https://github.com/netbirdio/netbird/issues/2941#issuecomment-2495692736 for a fix / workaround if you get 403 forbidden and the service account login is verified to be working.
The same problem Redirect URI error and it's resolved
https://github.com/netbirdio/netbird/issues/2941#issuecomment-2556738899
Hi Everyone, i was having the same issue until i added this on my Authentik > OAuth2 Provider (Netbird) > advanced protocol setting > Scope
authentik default OAuth Mapping: authentik API access
Hi Everyone, i was having the same issue until i added this on my Authentik > OAuth2 Provider (Netbird) > advanced protocol setting > Scope
authentik default OAuth Mapping: authentik API access
Thank you so much, this fixed it for me immediately.
i have the same error
Request failed with status code 401
Error: Token invalid
is this fixed?
Because I see zero effort on Netbirds' part to fix this error!
i have the same error
Request failed with status code 401
Error: Token invalid
is this fixed?
Because I see zero effort on Netbirds' part to fix this error!
Whilst this was over a year ago for me, the "Because I see zero effort on Netbirds' part to fix this error!" is because the problem turned out to be user-error on my behalf.
I'll go through my Reddit, GitHub, emails, and personal notes to see what the issue was, but no promises I can find what I did, nor if it's changed in any way in the time since I fixed my self-inflicted issue. Classic PEBKAC! Felt very silly for it.
Here you go (wasn't a year ago lol, but the past few months have certainly dragged for me lol!): this is what fixed the problem for me: https://github.com/netbirdio/netbird/issues/2142#issuecomment-2915471588
I submitted a pull request that was merged, which updated the docs to try and help others from hitting that same pitfall: https://github.com/netbirdio/docs/pull/359
If the above fixes your issue, and you feel the documentation is still lacking somehow (be very blunt, it's helpful), then I'll gladly submit a PR to improve the docs with the changes if it makes sense. :- )