x509: certificate signed by unknown authority when using v1.0.0

[nabadger]

Title

x509: certificate signed by unknown authority (1.0.0)

Summary

I'm finding that since switching from quay.io/keycloak/keycloak-gatekeeper:10.0.0 to louketo/louketo-proxy:1.0.0 I get 403 errors when attempting to auth.

Reverting to gatekeeper:10.0.0 works.

There are no other configuration changes happening here (just the image).

Environment

Version of everything that it's running in your environment:

• OS: K8s / GKE 1.14.x
• Kernel: K8s / GKE 1.14.x
• Go: K8s / GKE 1.14.x
• Server: https://github.com/dexidp/dex [ quay.io/dexidp/dex:v2.21.0 ]
• Louketo: 1.0.0

Configuration

        image: quay.io/louketo/louketo-proxy:1.0.0 
        - args:
        - --add-claims=groups
        - --client-id=grafana
        - --client-secret=grafana
        - --discovery-url=https://dex.svc.nb1.gke-dev.example.com/.well-known/openid-configuration
        - --enable-authorization-header=false
        - --enable-json-logging=true
        - --enable-logging=true
        - --enable-metrics=true
        - --enable-token-header=false
        - --enable-default-deny=true
        - --listen=:6001
        - --preserve-host=true
        - --redirection-url=https://grafana.svc.nb1.gke-dev.example.com
        - --resources=uri=/*|groups=it-all
        - --scopes=groups
        - --secure-cookie=true
        - --skip-openid-provider-tls-verify
        - --skip-upstream-tls-verify
        - --upstream-url=http://127.0.0.1:3000

I'm specifying both skip-upstream-tls-verify and skip-openid-provider-tls-verify

Expected Results

Token issued, auth'd and redirected to upstream site, i.e:

"msg":"issuing access token for user","email":"[email protected]","expires":"2020-07-03T12:22:20Z","

Actual Results

Access to grafana.svc.nb1.gke-dev.example.com was deniedY ou don't have authorization to view this page. HTTP ERROR 403

Louketo logs:

{"level":"info","ts":"2020-07-03T08:11:39.788Z","msg":"starting the service","prog":"louketo-proxy","author":"Louketo","version":"v2.3.0 (git+sha: 9eca196-dirty, built: 01-07-2020)"}
{"level":"info","ts":"2020-07-03T08:11:39.788Z","msg":"attempting to retrieve configuration discovery url","url":"https://dex.svc.nb1.gke-dev.example.com","timeout":"30s"}
{"level":"info","ts":"2020-07-03T08:11:39.803Z","msg":"successfully retrieved openid configuration from the discovery"}
{"level":"info","ts":"2020-07-03T08:11:39.804Z","msg":"enabled reverse proxy mode, upstream url","url":"http://127.0.0.1:3000"}
{"level":"info","ts":"2020-07-03T08:11:39.804Z","msg":"enabled the service metrics middleware","path":"/oauth//metrics"}
{"level":"info","ts":"2020-07-03T08:11:39.804Z","msg":"using session cookies only for access and refresh tokens"}
{"level":"info","ts":"2020-07-03T08:11:39.804Z","msg":"protecting resource","resource":"uri: /*, methods: DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT,TRACE, required: authentication only"}
{"level":"info","ts":"2020-07-03T08:11:39.805Z","msg":"Louketo proxy service starting","interface":":6001"}
{"level":"error","ts":"2020-07-03T08:12:45.192Z","msg":"no session found in request, redirecting for authorization","error":"authentication session not found"}
{"level":"info","ts":"2020-07-03T08:12:45.193Z","msg":"client request","latency":0.000375573,"status":303,"bytes":86,"client_ip":"10.12.0.13:52134","method":"GET","path":"/"}
{"level":"info","ts":"2020-07-03T08:12:45.217Z","msg":"client request","latency":0.000166577,"status":303,"bytes":312,"client_ip":"10.12.0.13:52136","method":"GET","path":"/oauth/authorize"}
{"level":"error","ts":"2020-07-03T08:12:45.896Z","msg":"no session found in request, redirecting for authorization","error":"authentication session not found"}
{"level":"info","ts":"2020-07-03T08:12:45.896Z","msg":"client request","latency":0.000084038,"status":303,"bytes":86,"client_ip":"10.12.0.13:52152","method":"GET","path":"/"}
{"level":"info","ts":"2020-07-03T08:12:46.071Z","msg":"client request","latency":0.000093519,"status":303,"bytes":312,"client_ip":"10.12.0.13:52160","method":"GET","path":"/oauth/authorize"}


{"level":"error","ts":"2020-07-03T08:13:06.425Z","msg":"unable to exchange code for access token","error":"Post \"https://dex.svc.nb1.gke-dev.example.com/token\": x509: certificate signed by unknown authority"}
{"level":"info","ts":"2020-07-03T08:13:06.425Z","msg":"client request","latency":0.100279943,"status":403,"bytes":0,"client_ip":"10.12.0.13:52736","method":"GET","path":"/oauth/callback"}

"level":"error","ts":"2020-07-03T08:13:06.425Z","msg":"unable to exchange code for access token","error":"Post "https://dex.svc.nb1.gke-dev.example.com/token": x509: certificate signed by unknown authority"}

[jangaraj]

It looks like Move from coreos/go-oidc OAuth2 library to golang.org/x/oauth2 doesn't work with skip-openid-provider-tls-verify config.

[nabadger]

@jangaraj I think it's not passing in the desired context here

https://github.com/louketo/louketo-proxy/blob/master/oauth.go#L134

The provided context is what can be used to control which http-client is used. Suspect it's using a default one.

---

Also wondering if there might have been some confusion here:

https://github.com/louketo/louketo-proxy/blob/master/oauth.go#L94

Args suggest it's passing through the oidc-client, but it's actually the config.

[jijiechen]

Hi, is there a workaround for this issue?

[jangaraj]

Workaround: make CA cert available for Louketo (it must be available in the /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem in case of louketo Docker image)

[linkvt]

I would also really appreciate a fix for this, I guess it is not really hard if you have some go knowledge and know the sources of this project which I currently not really have, there is also an example https://github.com/golang/oauth2/issues/187#issuecomment-227811477 I will take a look at this when I find some time.

Edit: just saw the pinned issue regarding sunsetting the project, totally missed that...

[jangaraj]

See https://github.com/jangaraj/louketo-proxy/commit/ddb37f62388fbd046ec6cb181ef73a9d32826903 feel free to test and create here.