portainer
Untrusted certificate when using OAuth
Bug description When authenticating with OAuth cannot get authenticated. Using keycloak as the OAuth provider. I'm sure my certificate is ok. It's trusted by my browser.
Expected behavior Expect portainer to be able to get the token from keycloak.
Portainer Logs
[DEBUG] - OAuth authentication error: Post https://mydomain/auth/realms/myrealm/protocol/openid-connect/token: x509: certificate signed by unknown authority
http error: Unable to authenticate through OAuth (err=Unauthorized) (code=500)
Steps to reproduce the issue:
- Set up a docker using image
portainer/portainer-ce:latest - Go to Settings > Authentication > Authentication Method "OAuth"
- Fill in the urls as provided by keycloak
- Open an incognito tab and go to portainer, log in with OAuth
- Get redirected to the log in portal
- Get redirected back to portainer, authentication failed
Technical details:
- Portainer version: CE 2.0
- Docker version (managed by Portainer): portainer/portainer-ce:latest
- Platform (windows/linux): linux
- Command used to start Portainer (
docker run -p 9000:9000 portainer/portainer): docker-compose file - Browser: Chrome
Additional context
-
I tried running a docker container with a known good copy of
/etc/ssl/certs/ca-certificates.crt, same result. -
I've tried setting the token endpoint to an http url, but protainer does not support this protocol
Keycloak is not tested or supported by us, so I don't know if a self-signed cert is going to work. Try using a public, widely trusted, ssl cert ..
It's not self signed. The root is Digicert Global Root G2, tumbprint df3c24f9bfd666761b268073fe06d1cc8d4f82a4
Keycloak is at least tested by portainer though. There's a specific document for it. Though in the document http is used, which I've tried, give an error saying the protocol is not supported.
Yes, but that was without TLS.. we have not tested KeyCloak with TLS/SSL.
+1
We used Microsoft AD FS as the OAuth provider and encountered similar problems. It’s just that we do use the self-signed certificate. I searched for related documents and issues, but I didn’t find a way to provide OAuth ignore certificates or provide self-signed certificates.
2020/11/05 04:03:03 [DEBUG] - Failed retrieving access token: Post https://<AD FS Domain>/adfs/oauth2/token: x509: certificate signed by unknown authority
2020/11/05 04:03:03 [DEBUG] - OAuth authentication error: Post https://<AD FS Domain>/adfs/oauth2/token: x509: certificate signed by unknown authority
2020/11/05 04:03:03 http error: Unable to authenticate through OAuth (err=Unauthorized) (code=500)
same problem here. And I don’t think this is a keycloak Problem, it most likely that the CA ist not in the trust store.
@chrhuber @sdjnmxd @JeremyMahieu
as a workaround, you can create your own ca-certificates.crt file and mount that over the existing one. I tried this with portainer-ce:alpine, but I guess it should work for the plain version as well.
So, create a file that contains all your required CA certificates. On linux, you will find this file at /etc/ssl/certs/ca-certificates.crt. Make a copy of this file and paste the content of your CA crt file into this one.
Assuming your CA certificate is stored at /tmp/my-ca.crt, you can do the following
cat /etc/ssl/certs/ca-certificates.crt /tmp/my-ca-crt >> /opt/my-ca-certificates.crt
then, when you run your portainer, just mount this copy over the existing one:
docker run -v /opt/my-ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt ...
Have fun! PS: oauth is working in my setup using this workaround
@chrhuber @sdjnmxd @JeremyMahieu
Nevermind, this might be even easier
Just mount your CA certificate as another file to the cert store, like this (assuming your CA certificate is at /opt/my-ca.crt)
docker run -v /opt/my-ca.crt:/etc/ssl/certs/my-ca.crt ...
Seems to be working just fine and does not tamper with the existing ca-certificate file
Is there a solution for deployments with helm? i deployed portainer business using helm in our kubernetes cluster, i have issues settings up OpenID with Keycloak because its certificate is signed by our internal enterprise CA.
Hello, there is no option to trust self-signed certificates? Same bug here.
I also encountered this (using Authentik with a self-signed cert for OpenID) and solved it by customising the alpine build and injecting my CA cert using the update-ca-certificates tool.
FROM portainer/portainer-ce:alpine
RUN apk add ca-certificates --no-cache
COPY ca.pem /usr/local/share/ca-certificates/ca.pem
RUN update-ca-certificates
Not ideal but better than nothing and I dont care too much about image size
We are also experiencing a situation like this. We are trying to connect to a Keycloak instance which uses a certificate signed by our internal root CA. It would be neat if we could simply upload it via the web interface in the same way that it is seemingly already supported for Helm repositories.
What could be happening here is that something is missing the intermediate certificates. The root is known, but the intermediate cannot be checked because it's not received through the SSL handshake. And finally the leaf certificate cannot be checked becaues the intermediate is not known.
This should be a priority; adding custom CA chains to trust stores is becoming ubiquitous, following common adoption of tools for properly maintaining PKI. Things like Vault and Keycloak are used all over the place in conjunction with self-signed CAs that are not commonly added to public trust stores by default.
I don't want to trivialize the amount of work this requires, in fact the opposite; the reliance on internally-robust-but-not-publicly-trusted CAs throughout the industry is a critically important and growing use-case.
Portainer supporting OAuth2 is a convenience thing, but fully adopting open standards like OIDC and SAML, even UAM in the future are going to be the critical drivers for enterprise adoption, which is the golden goose.
Editing with additional information that may be helpful:
I'm having the same issue, trying to connect Portainer's OAuth 2.0 SSO to a private KeyCloak instance served behind a cert/key pair signed by a private CA chain. I was debating finding where the trust stores were loaded from, baking my own chain into the mix, but that's a silly workaround for a fundamentally upstream issue. Local authentication will work for now and is sufficient for the near-term, but it's a major blocker for portainer finding a major foothold in any environment under similar constraints (which I touched on above).
Portainer Business Edition v2.19.4 KeyCloak v23.0
Hunted down a handful of threads related to invalid scopes and other configuration issues, but in the end this was the issue that has prevented OAuth login in Portainer.
