Cilium envoy crd pre beta

[jrajahalme]

Minor fixes and manifests for testing k8s ingress beta builds:

Draft support for Cilium Envoy Config CRD.

To test:

1. Download cilium-cli release v0.10.2 OR build this branch:

BINDIR=~/.local/bin make install

(You may need to modify BINDIR to something in your PATH)

2. Start Kind cluster (from your cilium-cli repo):

kind create cluster --config .github/kind-config.yaml

3. Install the beta version of Cilium with feature and kube proxy replacement flags on:

cilium install --version -service-mesh:v1.11.0-beta.1 --config enable-envoy-config=true --kube-proxy-replacement=probe
cilium hubble enable
cilium hubble port-forward&

4. Apply envoy-test.yaml:

kubectl apply -f connectivity/manifests/envoy-test.yaml

5. Run connectivity test to deploy the test services:

cilium connectivity test --test egress-l7

6. Apply L7 policy:

kubectl apply -f connectivity/manifests/client-egress-l7-http.yaml
kubectl apply -f connectivity/manifests/client-egress-only-dns.yaml

7. Observe traffic from Hubble:

hubble observe --from-pod cilium-test/client2-6dd75b74c6-68h7d -f

Note: You need to adjust the exact name of client2-xxxxx-xxxx.

8. Try out traffic from client2 (allowed by the policy) couple of times:

kubectl exec -it -n cilium-test client2-6dd75b74c6-68h7d -- curl -v echo-other-node:8080/foo

Note: You need to adjust (auto-complete) the exact name of client2-xxxxx-xxxx.

Observe:

• Without the envoy-test.yaml CRD the path /foo is not found. This L7 LB rewrites paths starting with /foo to /, which is found in the echo services.
• ClusterIP (echo-other-node) is observed in Hubble output, showing that cluster IP gets out of the source pod.
• The service is 50/50 load balanced to backend(s) of both echo-same-node and echo-other-node

Example:

Oct 13 16:30:59.023: cilium-test/client2-6dd75b74c6-68h7d:45004 <> cilium-test/echo-other-node:8080 from-endpoint FORWARDED (TCP Flags: SYN)
Oct 13 16:30:59.032: cilium-test/client2-6dd75b74c6-68h7d:45004 <> cilium-test/echo-other-node-697d5d69b7-x6qnp:8080 from-proxy FORWARDED (TCP Flags: SYN)
Oct 13 16:31:10.717: cilium-test/client2-6dd75b74c6-68h7d:45164 <> cilium-test/echo-other-node:8080 from-endpoint FORWARDED (TCP Flags: SYN)
Oct 13 16:31:10.721: cilium-test/client2-6dd75b74c6-68h7d:45164 <> cilium-test/echo-same-node-7967996674-t24mq:8080 from-proxy FORWARDED (TCP Flags: SYN)

9. Try out denied traffic from client2 (denied by the policy):

kubectl exec -it -n cilium-test client2-6dd75b74c6-68h7d -- curl -v echo-other-node:8080/bar

Note: You need to adjust (auto-complete) the exact name of client2-xxxxx-xxxx.

Observe:

• the path /bar is not allowed by the L7 policy, and will get a 403 access denied response.
• ClusterIP (echo-other-node) is observed in Hubble output, showing that cluster IP gets out of the source pod.
• Connection to the backend is established as it is allowed on the network level by the policy
• This specific HTTP request is DROPPED by the L7 LB due to the L7 policy not allowing it at the HTTP level.

Example:

Feb  8 12:13:55.172: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-other-node:8080 none REDIRECTED (TCP Flags: SYN)
Feb  8 12:13:55.172: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-other-node:8080 to-proxy FORWARDED (TCP Flags: SYN)
Feb  8 12:13:55.172: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-other-node:8080 to-proxy FORWARDED (TCP Flags: ACK)
Feb  8 12:13:55.172: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-other-node:8080 to-proxy FORWARDED (TCP Flags: ACK, PSH)
Feb  8 12:13:55.173: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-same-node-7967996674-vjkpj:8080 L3-L4 REDIRECTED (TCP Flags: SYN)
Feb  8 12:13:55.173: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-same-node-7967996674-vjkpj:8080 to-endpoint FORWARDED (TCP Flags: SYN)
Feb  8 12:13:55.173: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-same-node-7967996674-vjkpj:8080 to-endpoint FORWARDED (TCP Flags: ACK)
Feb  8 12:13:55.174: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-same-node-7967996674-vjkpj:8080 http-request DROPPED (HTTP/1.1 GET http://echo-other-node:8080/bar)
Feb  8 12:13:55.174: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-same-node-7967996674-vjkpj:8080 to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Feb  8 12:13:55.174: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-other-node:8080 to-proxy FORWARDED (TCP Flags: ACK, FIN)
Feb  8 12:13:55.174: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-same-node-7967996674-vjkpj:8080 to-endpoint FORWARDED (TCP Flags: ACK)
Feb  8 12:13:55.174: cilium-test/client2-6dd75b74c6-6bv64:43244 -> cilium-test/echo-other-node:8080 to-proxy FORWARDED (TCP Flags: ACK)