gateway-api icon indicating copy to clipboard operation
gateway-api copied to clipboard
verified publisher icon kubernetes-sigs

HTTPBackendRef RequestMirror Conformance Test

Open dprotaso opened this issue 1 year ago • 12 comments

What type of PR is this?

/kind cleanup /kind test /area conformance

What this PR does / why we need it

There's currently no conformance test for this feature which is 'Extended'

Additional notes

I dropped path logic in filter so we can re-use the same test for the backend

NONE

dprotaso avatar Feb 26 '24 22:02 dprotaso

/kind cleanup /kind test /area conformance

dprotaso avatar Feb 26 '24 22:02 dprotaso

adding approvers /assign @arkodg @mlavacca @sunjayBhatia

adding reviewers /cc @LiorLieberman @michaelbeaumont @Xunzhuo

dprotaso avatar Mar 07 '24 22:03 dprotaso

Conformance reviewers/approvers - can I get review please @arkodg @mlavacca @sunjayBhatia @michaelbeaumont @LiorLieberman @Xunzhuo

dprotaso avatar Apr 01 '24 20:04 dprotaso

I've added back all the test cases for the regular filter. The backend filter has a subset of these cases.

dprotaso avatar Apr 03 '24 20:04 dprotaso

has this been tested against any implementation ? afaik this isnt easily possible on envoy based implementations

arkodg avatar Apr 05 '24 11:04 arkodg

has this been tested against any implementation ?

I ran these tests against Istio. The backendRef variant fails as expected because they don't support that feature.

dprotaso avatar Apr 05 '24 13:04 dprotaso

imo lets add conformance tests if at least 1 implementation supports this case.

arkodg avatar Apr 05 '24 16:04 arkodg

imo lets add conformance tests if at least 1 implementation supports this case.

+1, I think it's pretty difficult to add conformance tests before at least one implementation supports a feature. If we were to do that, I think we'd need to introduce some concept of conformance test stability and consider untested conformance tests as a non-blocking reference point.

robscott avatar Apr 05 '24 16:04 robscott

imo lets add conformance tests if at least 1 implementation supports this case.

But then an implementation can't advertise the support this feature on the GatewayClass. They'll have to wait for a new gateway api release to do this.

+1, I think it's pretty difficult to add conformance tests before at least one implementation supports a feature. If we were to do that, I think we'd need to introduce some concept of conformance test stability and consider untested conformance tests as a non-blocking reference point.

Then why is this functionality in the API? If we have the shape of feature in our API we can write conformance for it

dprotaso avatar Apr 05 '24 16:04 dprotaso

Also the test is identical to the pre-split filter - so for this case I wouldn't be concerned with stability of the conformance test.

dprotaso avatar Apr 05 '24 16:04 dprotaso

imo lets add conformance tests if at least 1 implementation supports this case.

But then an implementation can't advertise the support this feature on the GatewayClass. They'll have to wait for a new gateway api release to do this.

+1, I think it's pretty difficult to add conformance tests before at least one implementation supports a feature. If we were to do that, I think we'd need to introduce some concept of conformance test stability and consider untested conformance tests as a non-blocking reference point.

Then why is this functionality in the API? If we have the shape of feature in our API we can write conformance for it

I +1 what @arkodg and @robscott said about waiting to have at least one implementation on board with this feature. In my opinion, conformance tests are not intended to shape the API, but instead to validate API implementations. Regardless of the nature of the test (this test could be straightforward, as it shares the same logic of the requestMirror feature), we should wait for at least one implementation to properly run the tests, otherwise, the risk of having tests which are just a bunch of untested code is pretty high (I'm not saying it's the case of this specific PR, my concern is about the approach we want to have).

mlavacca avatar Jun 11 '24 11:06 mlavacca

In a backlog grooming session, we went through this issue and discussed the need to define the status of conformance tests (experimental/GA). Shane is going to take a look into it.

xref: https://github.com/kubernetes-sigs/gateway-api/issues/3009

/assign @shaneutt

mlavacca avatar Aug 19 '24 16:08 mlavacca

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dprotaso, shaneutt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Aug 20 '24 11:08 k8s-ci-robot

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Sep 04 '24 02:09 k8s-ci-robot

@dprotaso: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-gateway-api-crds-validation-1 eb2197f993e8698c660b9dfe1db1b2bb4bcc2b02 link true /test pull-gateway-api-crds-validation-1
pull-gateway-api-crds-validation-5 eb2197f993e8698c660b9dfe1db1b2bb4bcc2b02 link true /test pull-gateway-api-crds-validation-5
pull-gateway-api-crds-validation-3 eb2197f993e8698c660b9dfe1db1b2bb4bcc2b02 link true /test pull-gateway-api-crds-validation-3
pull-gateway-api-crds-validation-4 eb2197f993e8698c660b9dfe1db1b2bb4bcc2b02 link true /test pull-gateway-api-crds-validation-4
pull-gateway-api-crds-validation-2 eb2197f993e8698c660b9dfe1db1b2bb4bcc2b02 link true /test pull-gateway-api-crds-validation-2

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

k8s-ci-robot avatar Sep 10 '24 20:09 k8s-ci-robot

ping @dprotaso

candita avatar Nov 07 '24 18:11 candita

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Feb 05 '25 18:02 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Mar 14 '25 23:03 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-triage-robot avatar Apr 13 '25 23:04 k8s-triage-robot

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Apr 13 '25 23:04 k8s-ci-robot