Ensure cleanup in e2e tests

[lentzi90]

/kind feature

Describe the solution you'd like

We do not currently check if the e2e tests clean up after themselves. I think we should do at least a naive comparison before and after the e2e test suite to see if some resources were forgotten. This will help us find cases where CAPO does not properly delete resources and also makes it safer to run the e2e tests on a "real" OpenStack cloud instead of devstack.

List the resource before and save the result. List the resource after and compare. Report differences. Perhaps even fail the test if we can get this reliable enough.

Anything else you would like to add:

Listing everything in OpenStack is probably overkill. I imagine we would check something like:

• servers
• networks
• security groups
• load balancers
• volumes

Maybe also routers and ports if using pre-existing networks. I think the check must be before and after the suite (not per test) since multiple tests can run simultaneously.

[lentzi90]

/help

[k8s-ci-robot]

@lentzi90: This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mquhuy]

/assign