Unable to access dashboard after upgrading management to 0.34.1 (Authentik)

[reginaldosoares]

Issue: The dashboard becomes inaccessible after upgrading the Management and Signal components to version 0.34.1.

Details:

Environment: Self-hosted instance maintained for over a year without issues. Identity Provider (IdP): Authentik.

Troubleshooting Steps Taken:

1. Rollback Attempt: Reverted to previous versions of Management and Signal; however, the dashboard remains inaccessible.
2. Configuration Review: Examined and cross-checked IdP settings and Management configurations without identifying any issues.

Possible Cause: I believe this issue may be related to the recent change introduced in the following pull request: Update account peers on login on meta change #2991

Additionally, this problem might be correlated with: Stuck on loading screen on "/peers" (Authentik) #3007

dashboard error message:

Request failed with status code 500

Error: Internal server error

management relevant logs:

2024-12-11T09:50:49Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:122: keys refreshed, new UTC expiration time: 2024-12-11 09:50:49.739044538 +0000 UTC

2024-12-11T09:50:49Z DEBG management/server/account.go:1515: account cj6m6d83pjfs73fpq20g not found in cache, reloading

2024-12-11T09:50:49Z ERRO [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2, accountID: , userID: 6] management/server/http/util/util.go:81: got a handler error: 403 Forbidden

2024-12-11T09:50:49Z ERRO [userID: 6, context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2, accountID: ] management/server/http/util/util.go:110: got unhandled error code, error: 403 Forbidden

2024-12-11T09:50:49Z ERRO [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2] management/server/telemetry/http_api_metrics.go:168: HTTP response f457f469-16c8-44cb-b265-a9d7c8e568d2: GET /api/users status 500

2024-12-11T09:50:49Z DEBG [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 546 ms and finished with status 500

[mgarces]

hello, have you tried to open your dashboard in incognito mode?

[reginaldosoares]

Hello @mgarces, yes. Attempted different browsers, other account.

[mgarces]

can you look into developer tools on the browser, and see if there is any endpoint not working?

[mgarces]

can you run the latest dashboard version too?

[reginaldosoares]

deployed on dashboard:v2.7.1

dev tools:

tracing logs of the management: relevant log:

getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:155: HTTP request 9ac3aa45-80a5-49ae-95cf-6cc82d21582b: GET /api/users

2024-12-11T10:45:26Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:122: keys refreshed, new UTC expiration time: 2024-12-11 10:45:26.737845505 +0000 UTC

2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: , userID: 6] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z TRAC [userID: 6, context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: ] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z DEBG management/server/account.go:1515: account cj6m6d83pjfs73fpq20g not found in cache, reloading

2024-12-11T10:45:26Z ERRO [userID: 6, context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: ] management/server/http/util/util.go:81: got a handler error: 403 Forbidden

2024-12-11T10:45:26Z ERRO [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: , userID: 6] management/server/http/util/util.go:110: got unhandled error code, error: 403 Forbidden

2024-12-11T10:45:26Z ERRO [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:168: HTTP response 9ac3aa45-80a5-49ae-95cf-6cc82d21582b: GET /api/users status 500

2024-12-11T10:45:26Z DEBG [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 673 ms and finished with status 500

[deffcolony]

I got a similair issue but mine says Error: Unauthenticated this happend after upgrading to latest version of netbird

can we please validate if we have the similair settings? did something change in the scope? how can we resolve this issue?

[jvanbruegge]

I have a similar issue and I found this in the authentik logs:

{"auth_via": "unauthenticated", "domain_url": "authentik.<domain>", "event": "/api/v3/core/users/?page=1", "host": "authentik.<domain>", "level": "info", "logger": "authentik.asgi", "method": "GET", "pid": 3369829, "remote": "127.0.0.1", "request_id": "fbdd3550f688401c98ac5a00af4d1a3f", "runtime": 37, "schema_name": "public", "scheme": "https", "status": 403, "timestamp": "2025-01-10T18:20:50.917033", "user": "", "user_agent": "OpenAPI-Generator/1.0.0/go"}

which would suggest that netbird is trying to get the users without using the service account

[jvanbruegge]

Never mind, the solution from here: https://github.com/netbirdio/netbird/issues/2941#issuecomment-2503971240 fixed my issue, I did need to add the API access scope

[nazarewk]

Hello @reginaldosoares,

We're currently reviewing our open issues and would like to verify if this problem still exists in the latest NetBird version.

Could you please confirm if the issue is still there?

We may close this issue temporarily if we don't hear back from you within 2 weeks, but feel free to reopen it with updated information.

Thanks for your contribution to improving the project!

[kreativmonkey]

after upgrades this problem is still present on my setup.

Authentik version 2025.4.1 and Netbird version 0.44.0.

What i've tried:

1. updated the Access Token
2. added a new provider and user for netbird.
3. Checked all settings again and again.

i don't know what to try next. devices that are connected are fine but i can not manage anything at all!

[guemidiborhane]

I'm facing the same issue on new install 0.60.7