kilo
kilo copied to clipboard
squat
Graph and metrics port 1107 is publicly exposed by default
Hello, we are using a KILO enabled Kubernetes Cluster from a cloud provider. Due some investigations from other users, we identified that the Port 1107 is publicly exposed and presents sensitive data under the following URLS: Topology: http://<IP>:1107/graph Metrics: http://<IP>:1107/metrics Health: http://<IP>:1107/health
These are useful features for debugging and internal monitoring, but these information should not be publicly accessible by everyone in the internet.
Please consider to bind these http metrics to localhost and not to expose them on every interface.
https://github.com/squat/kilo/blob/3e7fe47131638045668336e7aff9bbd4849930d9/cmd/kg/main.go#L259
You can override the default with the listen flag.
Like kg - - listen localhost:1107.
If you want to query metrics from a remote host or different pod, you would need to have a proxy running in a sidecar. So using a different default value only makes sense if we add that to all the manifests.
An additional way to do this would be to continue listening on all interfaces but to require mTLS so all clients wanting to scrape have to present valid certs. In this case we'll need to add flags to allow that configuration for the Kilo HTTP server
Thank you for your comments. When KILO get deployed vanilla, the ports are open in the public and under every public IP:1107 the whole topology and graph get exposed.
Would it be a possibility to turn this option off by default ? I think there are a lot machines outside, which are exposing these sensitive information right now.
Someone of our Cloud provider explained me, that kgctl and kgctl graph will not work anymore, when the port is bound to localhost or is closed in the firewall.
Can you explain me more about this behavior ?
hi @chfxr that comment from your cloud provider is not correct.
the HTTP endpoint at :1107 is only for informational purposed and has no effect on the functionality of kgctl and and the kgctl graph command. Specifically, the functionality of the /graph endpoint has no effect on the functionality of kgctl.
TL;DR: you can completely turn off the HTTP endpoint exposed by Kilo and not see any effect in other operations.
Thank you very much for your explanation, this helps us a lot !
I forwarded this information to our Cloud provider, that kgctl and kgctl graph can run without the --listen flag.
I'm implementing protection for these endpoints using the kube-rbac-proxy project. The plan is for all requests to Kilo's endpoints to require a token that has the right to GET peers and nodes, since the data revealed by these endpoints shows information about both resources. One small hiccup is that kube-rbac-proxy does not permit specifying multiple permissions today [0]. To that end, I put together a PR [1] that implements this feature in the proxy, which will in turn allow us to modify our manifests to close this issue.
[0] https://github.com/brancz/kube-rbac-proxy/issues/154 [1] https://github.com/brancz/kube-rbac-proxy/pull/164
<3 out of curiosity, what is your cloud provider?
Its Scaleway https://www.scaleway.com/en/kubernetes-kosmos/
They fixed the issue several months ago.
So issue can be closed.