gateway-api icon indicating copy to clipboard operation
gateway-api copied to clipboard

Add hook points support in conformance test suite

Open shawnh2 opened this issue 1 year ago • 7 comments

What would you like to be added:

Add the hook point support for current conformance test suite.

For example, like AfterTestFail hook point, developers can attach various commands to this hook point, and programs that attach to this hook point wil only be executed if one conformance test is failed.

Why this is needed:

This will be super helpful for developers to debug the failed conformance test.

Sometimes the error trace reported by conformance test is not very useful. For example, we'd like to see the status/logs of specific resources (by attaching kubectl cmd to the hook which I described above could ease the pain), even some other configs that can be retrieved by other commands.

shawnh2 avatar Jun 04 '24 11:06 shawnh2

@mlavacca: The label(s) /label needs-triage cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/label needs-triage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Jun 04 '24 12:06 k8s-ci-robot

@mlavacca: The label(s) triage/needs cannot be applied, because the repository doesn't have them.

In response to this:

/triage needs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Jun 04 '24 12:06 k8s-ci-robot

/triage needs-triage

mlavacca avatar Jun 04 '24 13:06 mlavacca

@mlavacca: The label(s) triage/needs-triage cannot be applied, because the repository doesn't have them.

In response to this:

/triage needs-triage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Jun 04 '24 13:06 k8s-ci-robot

This sounds like a good idea, yeah. Making it easier for developers to get value from the conformance tests, whether or not they're "part of the Gateway API", sounds like a Good Thing™. 🙂

kflynn avatar Jun 04 '24 15:06 kflynn

Discussed in today's community meeting, this sounds like a great idea, thanks @shawnh2!

/triage accepted

robscott avatar Jun 04 '24 15:06 robscott

Assign myself, since I've been working on something that is quite similar to this issue as well. I will post the initial design of this here once it's done.

/assign

shawnh2 avatar Jun 06 '24 12:06 shawnh2