Setup/Teardown run as subtest using provided name

[phisco]

trafficstars

What do you want to see?

At present, when multiple Setup or Teardown steps are defined, it can be challenging to identify which specific step has failed if the proper logs were not configured initially. This results from the fact that while Assessment steps are executed in dedicated subtests, Setup and Teardown steps are not. Actually, names for the Setup and Teardown steps are totally ignored, which I don't think was the desired behaviour.

I think it could be useful to execute all steps in dedicated subtests.

Extra Labels

No response

[phisco]

Ping @vladimirvivien @harshanarayana given that I had your attention for the other issue 😬

[harshanarayana]

+1 to this. This will be a very useful change.

[harshanarayana]

@phisco Do you happen to have a possible format for these setup/teardowns in mind in terms of naming them ? If so, please share that here ?

[phisco]

@harshanarayana, do you mean for the names in the features? Teardown and Setup steps can already define a name, which in the case of the Assess steps are used as name for their dedicated sub-test, while here they are just ignored. See here for an example of how we use it.

I would expect each of these steps to be executed in its own sub-test and to be shown as .../Teardown/<NAME_OF_THE_STEP> for example here:

--- PASS: TestEnvironmentConfigDefault (30.30s)
    --- PASS: TestEnvironmentConfigDefault/TestEnvironmentConfigDefault (30.30s)
        --- PASS: TestEnvironmentConfigDefault/TestEnvironmentConfigDefault/CreateClaim (13.51s)
        --- PASS: TestEnvironmentConfigDefault/TestEnvironmentConfigDefault/MRHasAnnotation (0.51s)
        --- PASS: TestEnvironmentConfigDefault/TestEnvironmentConfigDefault/Teardown/DisableAlphaEnvironmentConfig (X.Ys)

or wthout the Teardown part:

        --- PASS: TestEnvironmentConfigDefault/TestEnvironmentConfigDefault/DisableAlphaEnvironmentConfig (X.Ys)

I'd have to check how this would play out with the current way setup teardown steps are handled.

[vladimirvivien]

@phisco I think it's a good idea to take a closer look how setup and teardown are put together. I don't see any harm in running them as subtests similar to other steps like "Assess".

[phisco]

Cool @vladimirvivien, I can give it a look over the holidays and try opening a PR for it!

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[phisco]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[cpanato]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[phisco]

I really need to find the time to work on this 😩 if anyone else wants to jump in, please do!

[vladimirvivien]

This is still relevant and needed.

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.