🐛 Certificate signed by unknown authority in Windows binary but not in dev

[SayakMukhopadhyay]

Describe the bug I am running a home kubernetes cluster which has Traefik as an ingress controller. I am trying to setup cloudflare tunnel so that it can tunnel through my internal network and get access to the ingress controller. For that, I am testing cloudflared on Windows. This is giving me unexpected results. When I am using the downloadable binary for cloudflared on windows, doing cloudflared tunnel run is giving me the following error:

2022-06-26T10:59:38Z ERR  error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: x509: certificate signed by unknown authority" cfRay=72156f489d994be6-SIN originService=https://192.168.0.152:443
2022-06-26T10:59:39Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: x509: certificate signed by unknown authority" connIndex=0 dest=https://my.domain.com type=http

But I am unable to reproduce the error when I am running after building from the source of cloudflared. Do note, that Traefik is supposed to supply a Let's Encrypt certificate for the domain my,domain.com. Moreover, I tried using the Linux binary from my WSL instance and that too works.

To Reproduce Steps to reproduce the behavior: My tunnel config:

tunnel: 68b5730b-03b6-stuff
credentials-file: C:\Users\username\.cloudflared\68b5730b-03b6-stuff.json
warp-routing:
  enabled: true
ingress:
  - service: https://192.168.0.152:443
    originRequest:
      originServerName: my.domain.com

• Run cloudflared tunnel run
• See above error
• Compile from source in windows and run
• No error present and underlying application is successfully tunneled

Expected behavior Cloudflare Tunnel should be able to tunnel to the Traefik ingress controller and verify the certificate provided by it on Windows, both from the downloadable binary and while compiling from source.

Environment and versions

• OS: Windows 11/Ubuntu 20.04(WSL)
• Architecture: AMD64
• Version: 2022.6.3

Logs and errors

2022-06-26T10:59:38Z ERR  error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: x509: certificate signed by unknown authority" cfRay=72156f489d994be6-SIN originService=https://192.168.0.152:443
2022-06-26T10:59:39Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: x509: certificate signed by unknown authority" connIndex=0 dest=https://my.domain.com type=http

Additional context I have tried debugging the downloaded binary in order to try and find the root cause. The only difference I see between the downloaded binary and the compiled source execution is in /net/http/transport.go In the line where the breakpoint is set, the downloaded binary gets hit whereas executing from source does not. Hope that somewhat helps in figuring out where the issue is as I can't figure it out.

[sudarshan-reddy]

x509: certificate signed by unknown authority"

Have you tried installing ca-certificates on your k8s cluster?

[SayakMukhopadhyay]

The container that cloudflare is tunneling to (traefik in this case) has ca-certificates installed already. Even the nodes which run kubernetes has ca-certificates installed on them. Do note, that the Linux binary is working fine and even compiling from source on Windows is working. Its only the downloaded binary on windows which is throwing this error.