Network test mode

[johnSchnake]

Signed-off-by: John Schnake [email protected]

What this PR does / why we need it:

Which issue(s) this PR fixes

• Fixes #

Special notes for your reviewer:

Release note:

release-note

[johnSchnake]

See #1523

I'm also wondering if we can make it a more appealing mode by taking Conformance out of it. That way it is just a networking focus entirely and may run much faster (e.g. 100 networking focused tests instead of 400 tests with 100 of them focused on networking).

We can also run it in parallel if you think that is similarly reliable (we'd just exclude serial tests then too).

[jayunit100]

yeah, take out conformance

[jayunit100]

skip Storage to so that in case any StatefulSet+Storage tests come up , we skip them here

[jayunit100]

this is great I think as a start John thanks for getting it rolling do we need a formal README.md about how to use these tests?

[johnSchnake]

My hope is that we can just lean on the new sonobuoy modes to be self documenting.

We can always do a blog to announce it and we can document the sonobuoy modes command itself to let people know that those modes exist and they can peruse them.

[johnSchnake]

So I've got a brand new sonobuoy feature to help me explore the tags/tests that a focus/skip would target.

It looks to me like you probably want more trimming:

sonobuoy e2e -f "NetworkPolicy|sig-network|StatefulSet" -s "Disruptive|Storage" -m tagCounts
[Conformance]:38
[DisabledForLargeClusters]:6
[Experimental]:2
[Feature:Example]:1
[Feature:IPv6DualStackAlphaFeature:Phase2]:6
[Feature:IPv6DualStackAlphaFeature]:21
[Feature:IngressScale]:1
[Feature:Ingress]:5
[Feature:NEG]:7
[Feature:NetworkPolicy]:29
[Feature:Networking-IPv4]:1
[Feature:Networking-IPv6]:1
[Feature:Networking-Performance]:1
[Feature:NoSNAT]:1
[Feature:PerformanceDNS]:1
[Feature:SCTP]:4
[Feature:StatefulSet]:4
[Feature:kubemci]:6
[LinuxOnly]:75
[NodeConformance]:4
[Privileged]:1
[Provider:GCE]:1
[Serial]:10
[Slow]:52
[StatefulSetBasic]:11
[k8s.io]:15
[sig-apps]:15
[sig-network]:176
[sig-storage]:4

I'm assuming we would also want to skip: Provider, Alpha, Experimental.

You could see how that goes, but there are also 52 slow tests and 10 serial. Maybe we avoid the serial tests as well and then put it in parallel mode to speed up?

[johnSchnake]

After looking at that list let me know what other tags you think to skip/focus on

[johnSchnake]

Just pinging because I know this is something you hoped to get in. I'm going to be cutting a release in the next day or two so I'll need you to be sure of the test choices.

If you want to play with the sonobuoy e2e feature (what allowed me to generate the tag list based on focus/skip values) just grab the main branch of sonobuoy and run go install

[johnSchnake]

Going to cut a release and can add this in next time; we can do point releases any time. Just let me know about the above.

[johnSchnake]

ping @jayunit100 if you figure out which focus/skip you want we can pull the trigger on this. Let me know if you want to do a quick zoom session to help iron it out.