fix(*): localhost exposed application shouldn't be reachable

[lukidzi]

Summary

When a user deploys Kuma the behavior of communication with services changes. When there is no Kuma and the application running within the pod is binding to PodIP or Wildcard(0.0.0.0) other pods are able to reach the application. After Kuma is deployed it's not possible anymore. Instead, applications that bind to localhost or wildcard are exposed outside. That's a big security threat. In this PR we are introducing a different way how the inbound cluster is configured by default.

Previously we had an inbound listener that had a static cluster that pointed to the address localhost:PORT. Now it's going to point to DataplaneIP

1. Application that is binding to localhost won't be accessible anymore.
2. User can expose application binding to localhost by dataplane.networking.inbound[].serviceAddress

What can I do to make the smooth upgrade?

• check if your applications are listening on localhost and should be exposed outside - if no change to 0.0.0.0
• if you want to expose application binding to localhost set dataplane.networking.inbound[].serviceAddress to 127.0.0.1
• you can disable this behavior with kuma-cp configuration or env to kuma-cp KUMA_DEFAULTS_ENABLE_LOCALHOST_INBOUND_CLUSTERS - not recommended, we are going to remove this flag in the future

Full changelog

• [Changed the default inbound cluster IP from localhost to DataplaneIP]
• [Setting upstream_bind_address for inbound clusters that don't set have localhost address]
• [Added e2e tests that check behavior with different application bindings]
• [Changed inbound clusters naming: localhost -> inbound]
• [Made changes to applications metrics scraper to use DataplaneIP]

• [X] Link to docs PR or issue -- none
• [X] Link to UI issue or PR -- none
• [X] Is the [issue worked on linked][1]? -- Fix #4630
• [X] Unit Tests --
• [X] E2E Tests --
• [X] Manual Universal Tests --
• [X] Manual Kubernetes Tests --
• [X] Do you need to update UPGRADE.md? --
• [X] Does it need to be backported according to the backporting policy? -- none

[codecov-commenter]

Codecov Report

Merging #4750 (61329d7) into master (9e0f8c9) will increase coverage by 0.00%. The diff coverage is 44.44%.

@@           Coverage Diff           @@
##           master    #4750   +/-   ##
=======================================
  Coverage   46.45%   46.45%           
=======================================
  Files         690      690           
  Lines       47115    47164   +49     
=======================================
+ Hits        21888    21912   +24     
- Misses      23300    23322   +22     
- Partials     1927     1930    +3     

Impacted Files	Coverage Δ
pkg/core/bootstrap/bootstrap.go	0.00% <0.00%> (ø)
pkg/xds/bootstrap/components.go	0.00% <0.00%> (ø)
pkg/xds/bootstrap/handler.go	35.89% <0.00%> (-0.47%)	:arrow_down:
pkg/xds/envoy/names/resource_names.go	22.72% <ø> (ø)
app/kuma-dp/pkg/dataplane/metrics/server.go	9.75% <8.69%> (-1.57%)	:arrow_down:
pkg/util/net/ips.go	36.36% <42.85%> (+3.03%)	:arrow_up:
app/kuma-dp/cmd/run.go	66.16% <50.00%> (-0.33%)	:arrow_down:
api/mesh/v1alpha1/dataplane_helpers.go	52.30% <60.00%> (-0.04%)	:arrow_down:
pkg/xds/generator/inbound_proxy_generator.go	74.77% <66.66%> (-0.47%)	:arrow_down:
pkg/xds/bootstrap/generator.go	61.11% <84.21%> (-0.12%)	:arrow_down:
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[lobkovilya]

@lukidzi what happens if application binds to 127.0.0.6 when acts as a client? Does it bypass the envoy in that case?

[slonka]

@lukidzi - you need to pull master for the ci/run-full-matrix label to work

[lukidzi]

@lukidzi what happens if application binds to 127.0.0.6 when acts as a client? Does it bypass the envoy in that case?

It seems so, but it looks like it's the same behavior now. We've already had that rule so from iptables there is no change.