falco
falco copied to clipboard
Unable to get manifest: unable to fetch reference || tls: failed to verify certificate: x509: certificate signed by unknown authority
Describe the bug
We have private k8s cluster and we use proxy to communicate for outside the N/W. After adding the proxy under ENV for http_proxy & https_proxy in values.yaml
we are getting below error
{"level":"INFO","msg":"Resolving dependencies ...","timestamp":"2025-06-09 06:24:32"} {"level":"ERROR","msg":"unable to get manifest: unable to fetch reference "ghcr.io/falcosecurity/rules/falco-rules:3": Get "https://ghcr.io/v2/falcosecurity/rules/falco-rules/manifests/3": tls: failed to verify certificate: x509: certificate signed by unknown authority","timestamp":"2025-06-09 06:24:32"} {"level":"INFO","msg":"Resolving dependencies ...","timestamp":"2025-06-09 06:25:01"}
How to reproduce it
Add under extra and and falcoctl for falco Helm
extra:
#--Extra environment variables that will be pass onto Falco containers.
env:
- name: https_proxy
value: "http://myProxy.com:8080"
- name: http_proxy
value: "http://myProxy.com:8080"
falcoctl:
image:
# -- The image pull policy.
pullPolicy: IfNotPresent
# -- The image registry to pull from.
registry: PrivateArtifactory.com
# -- The image repository to pull from.
repository: docker/falcosecurity/falcoctl
# -- The image tag to pull.
tag: "0.11.0"
artifact:
# -- Runs "falcoctl artifact install" command as an init container. It is used to install artfacts before
# Falco starts. It provides them to Falco by using an emptyDir volume.
install:
enabled: true
#--Extra environment variables that will be pass onto falcoctl-artifact-install init container.
env:
- name: https_proxy
value: "http://myProxy.com:8080"
- name: http_proxy
value: "http://myProxy.com:8080"
Expected behaviour
{"level":"INFO","msg":"Resolving dependencies ...","timestamp":"2025-06-09 06:24:32"} {"level":"ERROR","msg":"unable to get manifest: unable to fetch reference "ghcr.io/falcosecurity/rules/falco-rules:3": Get "https://ghcr.io/v2/falcosecurity/rules/falco-rules/manifests/3": tls: failed to verify certificate: x509: certificate signed by unknown authority","timestamp":"2025-06-09 06:24:32"} {"level":"INFO","msg":"Resolving dependencies ...","timestamp":"2025-06-09 06:25:01"}
Environment
- Falco version: latest Helm
- System info: K8s (AKS)
- Cloud provider or hardware configuration:
- OS: N/A
- Kernel: N/A
- Installation method: HELM
without proxy variable, it is not been able to fetch index
{"level":"ERROR","msg":"unable to fetch index "falcosecurity" with URL "https://falcosecurity.github.io/falcoctl/index.yaml": unable to fetch index: cannot fetch index: Get "https://falcosecurity.github.io/falcoctl/index.yaml": dial tcp 185.199.109.153:443: i/o timeout","timestamp":"2025-06-09 06:00:30"}
with proxy
{"level":"INFO","msg":"Resolving dependencies ...","timestamp":"2025-06-09 06:24:32"} {"level":"ERROR","msg":"unable to get manifest: unable to fetch reference "ghcr.io/falcosecurity/rules/falco-rules:3": Get "[https://ghcr.io/v2/falcosecurity/rules/falco-rules/manifests/3](https://ghcr.io/v2/falcosecurity/rules/falco-rules/manifests/3%5C)": tls: failed to verify certificate: x509: certificate signed by unknown authority","timestamp":"2025-06-09 06:24:32"} {"level":"INFO","msg":"Resolving dependencies ...","timestamp":"2025-06-09 06:25:01"}
Hey! I found this one that can be relevant: https://github.com/falcosecurity/falco/issues/3119
Hey! I found this one that can be relevant: #3119
Getting same with FALCOCTL_ARTIFACT_NOVERIFY = true
falcoctl: artifact: install: enabled: true env: - name: https_proxy value: "http://myproxy.com:8080/" - name: http_proxy value: "http://myproxy.com:8080/" - name: FALCOCTL_ARTIFACT_NOVERIFY value: "true"
Is it still saying that
tls: failed to verify certificate: x509: certificate signed by unknown authority
?
ll saying that
Yes
{"level":"ERROR","msg":"unable to fetch index "falcosecurity" with URL "https://falcosecurity.github.io/falcoctl/index.yaml": unable to fetch index: cannot fetch index: Get "https://falcosecurity.github.io/falcoctl/index.yaml": tls: failed to verify certificate: x509: certificate signed by unknown authority","timestamp":"2025-06-11 05:39:19"}
As far as i see, it is failing here: https://github.com/falcosecurity/falcoctl/blob/6d9b57671f4c78a66af659288c0fb45fdac7d503/pkg/index/fetch/http/fetcher.go#L37
Since here we are just using net/http golang package, i assume there must be some misconfiguration in the proxy; have you tried to use uppercase for proxy env variables, eg:
- HTTP_PROXY
- HTTPS_PROXY (found this one too https://github.com/falcosecurity/falcoctl/issues/290)
As far as i see, it is failing here: https://github.com/falcosecurity/falcoctl/blob/6d9b57671f4c78a66af659288c0fb45fdac7d503/pkg/index/fetch/http/fetcher.go#L37
Since here we are just using
net/httpgolang package, i assume there must be some misconfiguration in the proxy; have you tried to use uppercase for proxy env variables, eg:
- HTTP_PROXY
- HTTPS_PROXY (found this one too falcoctl via proxy falcoctl#290)
Yes, Already tested with uppercase too
@Su-HackZ
Can you check if http://myproxy.com:8080/ (or whatever URL you used) actually resolves to your proxy from within the falcoctl's container?
Maybe is just a mistyping issue, but I see you wrote:
- name: https_proxy value: "http://myproxy.com:8080/"
Could it be:
- name: https_proxy value: "https://myproxy.com:8080/"
?
@Su-HackZ Can you check if
http://myproxy.com:8080/(or whatever URL you used) actually resolves to your proxy from within the falcoctl's container?
Checked with http://myproxy.com:8080/ getting same
env:
- name: https_proxy value: "http://myproxy.com:8080/"
- name: http_proxy value: "http://myproxy.com:8080/"
- name: FALCOCTL_ARTIFACT_NOVERIFY value: "true"
Maybe is just a mistyping issue, but I see you wrote:
- name: https_proxy value: "http://myproxy.com:8080/"
Could it be:
- name: https_proxy value: "https://myproxy.com:8080/"
?
It is not typo, for https_proxy we are using same value as http_proxy http://myproxy.com:8080
Checked with http://myproxy.com:8080/ getting same
If you meant that the proxy isn't resolving or responding as expected, the issue isn't in falcoctl.
For example, if you run from within the falcoctl container:
curl -v --proxy https://myproxy.com:8080 https://example.com/
and you get the same problem, likely the root cause is in the way the container reaches the proxy (eg. bad DNS?)
Checked with http://myproxy.com:8080/ getting same
If you meant that the proxy isn't resolving or responding as expected, the issue isn't in falcoctl.
For example, if you run from within the falcoctl container:
curl -v --proxy https://myproxy.com:8080 https://example.com/ and you get the same problem, likely the root cause is in the way the container reaches the proxy (eg. bad DNS?)
The DNS worked fine , falcoctl does not have shell the status is waiting, I did curl from another pod though, Having http_proxy as ENV variable and from the pod it was able to curl the falcoctl index
This is very strange behavior, we can see with openssl command that there is no issue with proxy too....
Checked with http://myproxy.com:8080/ getting same
If you meant that the proxy isn't resolving or responding as expected, the issue isn't in falcoctl. For example, if you run from within the falcoctl container: curl -v --proxy https://myproxy.com:8080 https://example.com/ and you get the same problem, likely the root cause is in the way the container reaches the proxy (eg. bad DNS?)
The DNS worked fine , falcoctl does not have shell the status is waiting, I did curl from another pod though, Having http_proxy as ENV variable and from the pod it was able to curl the falcoctl index
This is very strange behavior, we can see with openssl command that there is no issue with proxy too....
Do you confirm the error comes from the falcoctl-artifact-install container and not from the falcoctl-artifact-follow container?
I'm asking because there are two different env configs (I know it is a bit cumbersome):
falcoctl.artifact.install.envfalcoctl.artifact.follow.env
Both should be set if you enable both features.
Checked with http://myproxy.com:8080/ getting same
If you meant that the proxy isn't resolving or responding as expected, the issue isn't in falcoctl. For example, if you run from within the falcoctl container: curl -v --proxy https://myproxy.com:8080 https://example.com/ and you get the same problem, likely the root cause is in the way the container reaches the proxy (eg. bad DNS?)
The DNS worked fine , falcoctl does not have shell the status is waiting, I did curl from another pod though, Having http_proxy as ENV variable and from the pod it was able to curl the falcoctl index This is very strange behavior, we can see with openssl command that there is no issue with proxy too....
Do you confirm the error comes from the
falcoctl-artifact-installcontainer and not from thefalcoctl-artifact-followcontainer?I'm asking because there are two different env configs (I know it is a bit cumbersome):
falcoctl.artifact.install.envfalcoctl.artifact.follow.envBoth should be set if you enable both features.
Yes, it is coming from the falcoctl-artifact-install container not from falcoctl-artifact-follow container
both are set the same env,
@FYI, Logs from falcoctl.artifact.follow.env
Failed to load logs: container "falcoctl-artifact-follow" in pod "falco-68gpc" is waiting to start: PodInitializing Reason: BadRequest (400)
Yes, it is coming from the
falcoctl-artifact-installcontainer not fromfalcoctl-artifact-followcontainer both are set the same env,@fyi, Logs from
falcoctl.artifact.follow.envFailed to load logs: container "falcoctl-artifact-follow" in pod "falco-68gpc" is waiting to start: PodInitializing Reason: BadRequest (400)
Got it. Thank you for confirming. Unfortunately, I've no idea how to reproduce the issue at the moment. I will try to dig into it once I get some spare cycles. If you have any suggestions, please let us know!
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale
/remove-lifecycle stale