BookStackApp
OIDC with Zitadel SaaS stops working after some time (signature could not be validated using the provided keys)
Describe the Bug
When Zitadel SaaS is used for authentication with OIDC in Bookstack, it will stop working after some time, at the latest after 24 hours, and the following error message is shown: ID token validation failed with error: Token signature could not be validated using the provided keys.
Workaround: Deleting the bookstack docker container and recreating it fixes the error for some hours.
Steps to Reproduce
- set up OIDC with Zitadel SaaS as described in #4682 by @megastary
- test successful sign in with SSO
- wait 24 hours
- retry sign in with SSO
- see error:
ID token validation failed with error: Token signature could not be validated using the provided keys
Expected Behaviour
When set up correctly, authentication with OIDC in bookstack works also after 24 hours.
Screenshots or Additional Context
Browser Details
Brave (1.66.118 Chromium: 125.0.6422.147 (Official Build) (64-bit)) on Windows 11 Version 23H2 (Build 22631.3593)
Exact BookStack Version
v24.05.1
As mentioned in #4682, BookStack does cache discovered details but only for 15 minutes.
First, it would be good to test/rule-out instance cache issues.
Can you try setting the cache to be database based.
This is done by setting CACHE_DRIVER=database in your existing .env file, or by setting CACHE_DRIVER=database to the environment for your BookStack app container. Remember to re-create the container if altering container environment options.
@ssddanbrown I have set the suggested environment variable:
user@SRV001:/opt/bookstack$ docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
bookstack_app lscr.io/linuxserver/bookstack:latest "/init" bookstack_app 7 hours ago Up 4 hours 80/tcp, 443/tcp
bookstack_db mariadb:11 "docker-entrypoint.s…" bookstack_db 7 hours ago Up 4 hours 3306/tcp
user@SRV001:/opt/bookstack$ docker exec bookstack_app printenv
PATH=/lsiopy/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=581ed5423b8b
APP_THEME=custom
OIDC_ISSUER_DISCOVER=true
CACHE_DRIVER=database
OIDC_NAME=ZITADEL
OIDC_DISPLAY_NAME_CLAIMS=name
MAIL_ENCRYPTION=tls
MAIL_PASSWORD=somepassword
PUID=1000
OIDC_GROUPS_CLAIM=custom:roles
DB_PASS=somepassword
OIDC_END_SESSION_ENDPOINT=false
MAIL_DRIVER=smtp
DB_USER=bookstack
OIDC_REMOVE_FROM_GROUPS=true
OIDC_CLIENT_ID=someid@wiki
DB_HOST=bookstack_db
APP_DEBUG=true
AUTH_AUTO_INITIATE=true
MAIL_HOST=smtp-relay.brevo.com
[email protected]
PGID=1000
APP_URL=https://wiki.some.domain
OIDC_CLIENT_SECRET=somesecret
DB_DATABASE=bookstack
AUTH_METHOD=oidc
OIDC_ADDITIONAL_SCOPES=urn:zitadel:iam:org:projects:roles
MAIL_FROM_NAME=Wiki
OIDC_USER_TO_GROUPS=true
OIDC_ISSUER=https://some-instance.zitadel.cloud
DB_PORT=3306
MAIL_PORT=587
[email protected]
PS1=$(whoami)@$(hostname):$(pwd)\$
HOME=/root
TERM=xterm
S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
S6_VERBOSITY=1
S6_STAGE2_HOOK=/init-hook
VIRTUAL_ENV=/lsiopy
LSIO_FIRST_PARTY=true
Unfortunately, the error pattern persists. Even recreating the container does not solve the problem. This means that signing in with OIDC is not possible at all.
@baua1310 Does it start working again after removing the CACHE_DRIVER option again, and a container recreate?
@ssddanbrown Yes after removing CACHE_DRIVER and recreating the container sign in with OIDC started working again.
Just wanted to +1, I have the exact same problem after implementing a fresh Bookstack instance using Zitadel as my IDP, albeit mine is self hosted not SaaS Zitadel. Followed the same path from the other issue #4682, and now after some time, not necessarily 24 hours, I get the ID token validation failed with error: Token signature could not be validated using the provided keys error.
Edit: I have not restarted / rebuilt my Bookstack container, and after waiting a while after experiencing the error, it did start working again, probably when that cache expired and it renegotiated.
To add some more possibly worthwhile info, certain OIDC token settings can be modified in Zitadel (at least in self hosted, can't speak for SaaS). @ssddanbrown would you normally expect other OIDC IDPs to have a longer access, or in this case probably more relevant ID token expiry?
And apologies for my novice showing, but initially I did not include the Refresh Token grant type, will adding that have any bearing on this ID token issue?
Ultimately I suppose it would be nice, selfishly for this scenario, for this error to then prompt Bookstack to clear that cache and retrieve it fresh, but I'm sure it's easier said than done.
@pandieme
would you normally expect other OIDC IDPs to have a longer access, or in this case probably more relevant ID token expiry? [...] I did not include the Refresh Token grant type, will adding that have any bearing on this ID token issue?
Those settings don't really matter in the context of BookStack's use. We don't hold on to tokens for refresh, I believe we just go through the original process direct with the auth provided at login time.
Ultimately I suppose it would be nice, selfishly for this scenario, for this error to then prompt Bookstack to clear that cache and retrieve it fresh
The BookStack cache should only remain for 15 minutes, that's what I can't understand and need to dive deeper into. The above tests indicate it's not being cleared (not expiring), but I feel this would have show up in other auth providers/scenarios if our cache system was fundementally not listening to cache lifetime.
Out of interest, can you describe your environment too (BookStack hosting environment& install method) just so I can guage any potential patterns?
@ssddanbrown
Out of interest, can you describe your environment too (BookStack hosting environment& install method) just so I can guage any potential patterns?
Ours is running on a Debian 12 VM, hosted on an XCP-ng pool, using Docker compose and behind a Caddy reverse proxy also running in a container on a shared Docker network.
Docker versions
Docker Compose version v2.28.1
Docker version 27.0.3, build 7d4bcd8
Bookstack
Container info
../bookstack# docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
bookstack lscr.io/linuxserver/bookstack "/init" bookstack 41 hours ago Up 41 hours 80/tcp, 443/tcp
bookstack_db lscr.io/linuxserver/mariadb "/init" bookstack_db 41 hours ago Up 41 hours 3306/tcp
../bookstack# docker exec bookstack printenv
PATH=/lsiopy/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=b1317d85c73f
DB_DATABASE=bookstackapp
PGID=1000
OIC_DUMP_USER_DETAILS=false
[email protected]
AUTH_AUTO_INITIATE=false
OIDC_ISSUER_DISCOVER=true
OIDC_GROUPS_CLAIM=groups
OIDC_CLIENT_SECRET=ourclientsecret
OIDC_ISSUER=https://idp.fqdn
MAIL_DRIVER=smtp
DB_PASS=ourpass
AUTH_METHOD=oidc
DB_USER=bookstack
MAIL_USERNAME=ouruser
TZ=Europe/London
MAIL_PORT=587
OIDC_USER_TO_GROUPS=true
OIDC_DISPLAY_NAME_CLAIMS=name
MAIL_PASSWORD=ourpass
OIDC_REMOVE_FROM_GROUPS=true
MAIL_FROM_NAME=Intranet
APP_THEME=custom
DB_PORT=3306
MAIL_ENCRYPTION=tls
APP_URL=https://our.fqdn
MAIL_HOST=smtp.fqdn
DB_HOST=bookstack_db
OIDC_NAME=SSO
OIDC_END_SESSION_ENDPOINT=false
OIDC_CLIENT_ID=ourclientid
DB_ROOT_PASS=ourpass
PUID=1000
PS1=$(whoami)@$(hostname):$(pwd)\$
HOME=/root
TERM=xterm
S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
S6_VERBOSITY=1
S6_STAGE2_HOOK=/init-hook
VIRTUAL_ENV=/lsiopy
LSIO_FIRST_PARTY=true
Docker Compose
---
services:
bookstack:
image: lscr.io/linuxserver/bookstack
container_name: bookstack
environment:
PUID: 1000
PGID: 1000
TZ: Europe/London
APP_URL: https://our.fqdn
DB_HOST: bookstack_db
DB_PORT: 3306
DB_USER: bookstack
DB_PASS: ${DB_PASS}
DB_DATABASE: bookstackapp
volumes:
- app_data:/config
restart: unless-stopped
depends_on:
- bookstack_db
networks:
- default
- caddy
env_file:
- .env
bookstack_db:
image: lscr.io/linuxserver/mariadb
container_name: bookstack_db
environment:
PUID: 1000
PGID: 1000
TZ: Europe/London
MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASS}
MYSQL_DATABASE: bookstackapp
MYSQL_USER: bookstack
MYSQL_PASSWORD: ${DB_PASS}
volumes:
- db_data:/config
restart: unless-stopped
env_file:
- .env
volumes:
app_data:
name: bookstack_app_data
db_data:
name: bookstack_db_data
networks:
caddy:
name: caddy
external: true
.env
DB_ROOT_PASS=ourpass
DB_PASS=ourpass
AUTH_METHOD=oidc
AUTH_AUTO_INITIATE=false
OIDC_NAME=SSO
OIDC_DISPLAY_NAME_CLAIMS=name
OIDC_CLIENT_ID=ourclientid
OIDC_CLIENT_SECRET=ourclientsecret
OIDC_ISSUER=https://idp.fqdn
OIDC_END_SESSION_ENDPOINT=false
OIDC_ISSUER_DISCOVER=true
APP_THEME=custom
OIDC_DUMP_USER_DETAILS=false
OIDC_USER_TO_GROUPS=true
OIDC_GROUPS_CLAIM=groups
OIDC_REMOVE_FROM_GROUPS=true
MAIL_DRIVER=smtp
MAIL_HOST=smtp.fdqn
MAIL_PORT=587
MAIL_ENCRYPTION=tls
MAIL_USERNAME=ouruser
MAIL_PASSWORD=ourpass
[email protected]
MAIL_FROM_NAME=Intranet
Custom theme function
As described in this issue #4682 we added the following file to our bookstack containers volume to Replace multiple aud values with single azp value to cater for Zitadel's returned array.
# /var/lib/docker/volumes/bookstack_app_data/_data/www/themes/custom
<?php
use BookStack\Facades\Theme;
use BookStack\Theming\ThemeEvents;
Theme::listen(ThemeEvents::OIDC_ID_TOKEN_PRE_VALIDATE, function (array $idTokenData, array $accessTokenData) {
if (is_array($idTokenData['aud']) && in_array($idTokenData['azp'], $idTokenData['aud'])) {
return array_merge($idTokenData, [
'aud' => [$idTokenData['azp']]
]);
}
});
Caddy (Reverse Proxy)
Caddyfile
our.fqdn {
reverse_proxy http://bookstack
}
Docker Compose
services:
caddy:
container_name: caddy
image: caddy:2.8.4
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- data:/data:rw
- config:/config:rw
networks:
- caddy
volumes:
data:
config:
networks:
caddy:
name: caddy
Hi, just a quick update from my side: As the problem with OIDC and Zitadel had not been solved, I switched to SAML 2.0. SAML 2.0 and Zitadel is working fine since a few months. Here is my configuration:
AUTH_METHOD=saml2
AUTH_AUTO_INITIATE=true
SAML2_NAME=ZITADEL
SAML2_EMAIL_ATTRIBUTE=Email
SAML2_EXTERNAL_ID_ATTRIBUTE=UserID
SAML2_DISPLAY_NAME_ATTRIBUTES=FullName
SAML2_IDP_ENTITYID=https://<domain>.zitadel.cloud/saml/v2/metadata
SAML2_AUTOLOAD_METADATA=true
SAML2_USER_TO_GROUPS=true
SAML2_GROUP_ATTRIBUTE=roles
SAML2_REMOVE_FROM_GROUPS=true
To map Zitadel roles to bookstack groups I added added a custom script in Zitadel called "samlRoles" inside Actions and added the script in the flow "Complement SAMLResponse" to the trigger "Pre SAMLResponse creation".
/**
* Add an custom attribute to the SAMLResponse, if it's not already present.
* Adds the roles as an additional attribute to the SAMLResponse, if it's not already present.
*
* Flow: Complement SAMLResponse, Triggers: Pre SAMLResponse creation
*
* @param ctx
* @param api
*/
function samlRoles(ctx, api) {
if (ctx.v1.user.grants == undefined || ctx.v1.user.grants.count == 0) {
return;
}
let roles = [];
ctx.v1.user.grants.grants.forEach(grant => {
grant.roles.forEach(role => {
roles.push(role)
})
})
api.v1.attributes.setCustomAttribute('roles', '', ...roles)
}
Thanks @baua1310, works like a charm! Except for the logout process, but that's no problem.
Just ran into the same issue. Zitadel is self-hosted (and configured to rewrite the roles into a roles array) in a k3s cluster, as is bookstack (via linuxserver image with the same theme-hack concerning the audience array as pointed out above). Accordingly, the behaviour is also the same: After a day or so the login (which previously has worked fine after a fresh install) stops working with "ID token validation failed with error: Token signature could not be validated using the provided keys". Interestingly, it seems to work again after approximately 15minutes (pretty much the time it took me to find this thread and read through).
The zitadel logs say: "2025/06/05 13:09:26 ERROR: Failed to extract ServerMetadata from context"
nginx access.log of bookstack:
XX.XX.X.21 - - [05/Jun/2025:13:09:23 +0000] "GET / HTTP/1.1" 302 410 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:23 +0000] "GET /login HTTP/1.1" 200 7973 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:23 +0000] "GET /dist/styles.css?version=v25.05 HTTP/1.1" 304 0 "https://url.to.bookstack.com/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:23 +0000] "GET /dist/app.js?version=v25.05 HTTP/1.1" 200 192990 "https://url.to.bookstack.com/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:23 +0000] "GET /logo.png HTTP/1.1" 304 0 "https://url.to.bookstack.com/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:23 +0000] "GET /icon-32.png HTTP/1.1" 200 746 "https://url.to.bookstack.com/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:25 +0000] "POST /oidc/login HTTP/1.1" 302 1766 "https://url.to.bookstack.com/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:29 +0000] "GET /oidc/callback?code=xavchangedkvalyJ9gKVIJg-_rkUchangedw&state=fd3db65changed45d59b2 HTTP/1.1" 302 410 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:09:30 +0000] "GET /login HTTP/1.1" 200 8073 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:28:08 +0000] "POST /oidc/login HTTP/1.1" 302 1766 "https://url.to.bookstack.com/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:28:11 +0000] "GET /oidc/callback?code=zi_XpJAQEVLchangedg&state=905changed5a02d HTTP/1.1" 302 386 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:28:12 +0000] "GET / HTTP/1.1" 200 24005 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:28:12 +0000] "GET /uploads/images/user/2025-06/thumbs-30-30/hbtnje7rz6-avatar.png HTTP/1.1" 304 0 "https://url.to.bookstack.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.21 - - [05/Jun/2025:13:28:12 +0000] "GET /uploads/images/user/2025-06/thumbs-30-30/ku0ijbpqhv-avatar.png HTTP/1.1" 304 0 "https://url.to.bookstack.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0"
XX.XX.X.42 - - [05/Jun/2025:13:32:41 +0000] "GET /plugins/.git/config HTTP/1.1" 404 7800 "-" "msnbot-media/1.1 ( http://search.msn.com/msnbot.htm)"
@ssddanbrown Have you ever had the chance to look into your suggested caching issue? (btw: I think you do a great job, whether this issue gets resolved or not!)
PS: There is one thing that I've configured different than my predecessors; Bookstack is registered within Zitadel as PKCE app. Since zitadel does not provide a client secret for such cases and bookstack needs one, I had to generate a random value by myself which I feed into the bookstack container on startup.
@ssddanbrown Have you ever had the chance to look into your suggested caching issue?
@AndrinGautschi Not yet. To be honest it's an awkward one to attempt to replicate due to needing to be done over time, and since I think there's a fair chance of this being something Zitadel specific (just based upon not having this ever reported for another OIDC system). Just need to properly dedicate the time needed.
Do you get that zitadel log error message every time this occurs? Just trying to understand how connected that is to what's happening in BookStack.
As an aside, Zitadel has recently taken millions in VC funding, and has just switched their licensing to AGPLv3+CLA. Can't speak specifically to the project or its owners or management, but this is usually an indicator that the project is going in a certain growth-focus direction which often impacts users eventually. The funding provided by the same which funded Minio, which has been in the news recently for its open-source-user unfriendly changes.
Not validated, but potentially related to https://github.com/zitadel/zitadel/discussions/2042. Might be just that Zitadel (compared to others) has no smooth key rotation (Direct change with no/minimal crossover period) leading to no available keys for a moment, and in which case we should maybe (following the spec's advice) re-fetch the keys on unfamiliar key if we've used our own cache values.
Thanks. Unfortunately, I'm very busy right now with something else. I'll come back with proper telemetry data as soon as I've found time to investigate further.