CI: review runtime tests

[tklauser]

Scope	Context	Test	Details	Equivalent K8S test
RuntimeChaos	n/a	Endpoint recovery on restart	Tests consistency of endpoint information across Cilium restarts	Similar test in k8sT/Chaos.go
		Removing leftover Cilium interfaces	Tests that leftover lxc interfaces are removed on restart	none
		Checking for file-descriptor leak	Tests that Cilium processes don't create more than 5000 open file descriptors (i.e. leak FDs)	none
		Checking that during restart no traffic is dropped using Egress + Ingress Traffic	Checks that HTTP connections between endpoints still work during Cilium restart	Similar test (using netperf instead of httpd) in k8sT/Chaos.go
		Validate that delete events on KVStore do not release in use identities		?
		Delete event on KVStore with CIDR identities		?
RuntimeCLI	Identity CLI testing	Test labelsSHA256	Tests CLI cilium identity get subcommand	Migrated in #14017
		test identity list	Tests CLI cilium identity list subcommand for an endpoint's identity and reserved identities
	stdout stderr testing	root command help should print to stdout	Tests that CLI command and subcommand help prints to stdout and not stderr
		subcommand help should print to stdout
		failed subcommand should print help to stdout
RuntimePrivilegedUnitTests	n/a	Run Tests	Run Go unit tests which need root privileges, e.g. to access BPF maps or network interfaces	none
RuntimeCassandra	n/a	Tests policy allowing all actions	Tests Cassandra L7 parser in envoy golang parser framework (proxylib)	Removed by #20765
		Tests policy disallowing Insert action
RuntimeKVStoreTest	n/a	KVStore tests under quarantine		?
RuntimePolicies	n/a	L3/L4 Checks	L3/L4 connectivity using TCP & HTTP (curl)	?
		L4Policy Checks		?
		Checks that traffic is not dropped when L4 policy is installed and deleted		?
		L7 Checks		?
		Tests Endpoint Connectivity Functions After Daemon Configuration Is Updated		?
		L3-Dependent L7 Egress		?
	CIDR L3 Policy	validates toCIDR	Tests toCIDR policy on IPv4/6 with default-deny enforcement	none
		validates fromCIDR	Tests fromCIDR policy on IPv4/6 with default-deny enforcement	none
	n/a	Extended HTTP Methods tests		?
		Tests Egress To World		?
		Test egress with L7 policy to outside cluster		?
		Tests EntityNone as a deny-all		?
	TestsEgressToHost	Tests Egress To Host		?
		Tests egress with CIDR+L4 policy		?
		Tests egress with CIDR+L4 policy to external https service		?
		Tests egress with CIDR+L7 policy		?
	Init Policy Default Drop Test	tests ingress	Covers hubble observe	?
		tests egress	Covers hubble observe	?
	Init Policy Default Drop Test With PolicyAuditMode	tests ingress	Covers hubble observe	?
		tests egress	Covers hubble observe	?
	Init Policy Test	Init Ingress Policy Test	Covers hubble observe	?
		Init Egress Policy Test	Covers hubble observe	?
	Tests for Already-Allocated Identities	Tests L4 policy is generated for endpoint with already-allocated identity	Creates a new container which has labels which have already been allocated an identity from the key-value store. Checks datapath behavior matches policy which selects this new endpoint.	?
RuntimePolicyImportTests	n/a	Invalid Policies		Removed by #20512
	Policy command	Tests getting policy by labels		?
		Tests deleting policy key		?
	n/a	checks policy trace output		?
RuntimeConntrackInVethModeTest	n/a	Conntrack-related configuration options for endpoints	Checks conntrack using Docker networking with veth.	none
RuntimeFQDNPolicies	n/a	Enforces ToFQDNs policy		?
		Validate dns-proxy monitor information		?
		Interaction with other ToCIDR rules		?
		Roundrobin DNS		?
		Can update L7 DNS policy rules		?
		CNAME follow		?
		Enforces L3 policy even when no IPs are inserted		?
		Implements matchPattern: "*"		?
		Validates DNSSEC responses		?
	toFQDNs populates toCIDRSet when poller is disabled (data from proxy)	Policy addition after DNS lookup		?
		L3-dependent L7/HTTP with toFQDN updates proxy policy		?
	n/a	DNS proxy policy works if Cilium stops		?
RuntimeKafka	n/a	Kafka Policy Ingress		Removed by #20765
		Kafka Policy Role Ingress
RuntimeLB	n/a	validates basic service management functionality		?
RuntimeMemcache (currently disabled)	Testing binary memcache	Tests basic memcache operation		Removed in #20132
		Tests policy allowing all actions
		Tests policy disallowing set action
		Tests policy allowing actions only for key
		Tests multi-get from a disallowed and allowed keys set
	Testing text memcache	Tests basic memcache operation
		Tests policy allowing all actions
		Tests policy disallowing set action
		Tests policy allowing actions only for allowed key
RuntimeMonitorTest	With Sample Containers	Cilium monitor verbose mode	Checks verbose output of cilium monitor	Implicitely through (*Kubectl).MonitorStart
		Cilium monitor event types		?
		cilium monitor check --from		?
		cilium monitor check --to		?
		cilium monitor check --related-to		?
		delivers the same information to multiple monitors		?
		checks container ids match monitor output		?
RuntimeSSHTests	n/a	Should fail when context times out	Tests SSH helpers used by runtime test code.	none, but not needed outside of runtime tests
RuntimeVerifier	n/a	runs the kernel verifier against the tree copy of the BPF datapath	Runs the kernel verifier against Cilium BPF datapath by execing into the node and runing `make -C bpf` so we can run the script directly on the node. Note: This is using code in the Cilium tree, not the Cilium container.	Done in #12658

Overview generated by running ginkgo -focus="Runtime" -noColor -dryRun -v > runtime-tests.txt: runtime-tests.txt

More details to be added...

[gandro]

Small note regarding the RuntimePolicies test suite: They are currently providing a bit of coverage for hubble observe which to my knowledge is likely not present in any equivalent K8s test. There is nothing fundamental about this, but removing the runtime tests without updating the K8s tests to insert hubble observe calls would decrease coverage for Hubble.

[tklauser]

Thanks @gandro! I've updated the table to say "Covers hubble observe" for tests that I think (or according to my grep skills) cover hubble observe.

[pchaigno]

I've updated the RuntimePolicies CIDR L3 Policy tests. I don't think we have a full equivalent in K8s* because we don't seem to test default-deny enforcement there. I've just found a bug in my (local) code thanks to the IPv6 ping with everything else blocked (because default-deny); none of the K8s* tests had a similar fail.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[ti-mo]

Hey, I'm seeing recent PRs linking to this issue, but I'm wondering if this is where we're tracking test migrations and if it's still a good idea to do so.

[tklauser]

Thanks for checking in, I wasn't aware this is still open. The list above is very outdated, so it probably makes sense to close this issue and track test migrations in separate issues.