fix: conformance suite should not be with _test suffix

[Xunzhuo]

conformance suite defs should not be with _test suffix, this is not accessbile for inner packages.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle rotten
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[lauralorenz]

/remove-lifecycle rotten

[mikemorris]

@nojnhuh I know you recently got ~~the conformance~~ some tests running, do you think a change like this is still needed?

[nojnhuh]

@mikemorris I got the tests under e2etest/ working, but haven't looked at the conformance tests yet. Are there any docs describing how the conformance tests are designed to be used?

[mikemorris]

@nojnhuh Found the doc I remembered that had some detail on conformance plans https://docs.google.com/document/d/11mK-Xxug4GYFrhXmt6KdS5ApB2gAM3mf28HHbvMwqHs/edit (referenced in older SIG-Multicluster meeting notes).

My recollection was that e2e tests were somewhat analogous to conformance because the MCS API intentionally doesn't include a reference implementation controller (for similar reasons as Gateway API), leaving the actual mechanism of watching ServiceExport resources and creating corresponding ServiceImports for vendor implementations.

[nojnhuh]

Thanks @mikemorris! Regarding this particular change, I think it's necessary if we expect implementors to import these packages in go.mod and maintain the entrypoint to the tests themselves. Otherwise if it's like the Gateway API conformance tests where this repo will contain the entrypoint to the tests and implementors are expected to invoke that, then I don't think this change is strictly necessary.

[mikemorris]

I think it's necessary if we expect implementors to import these packages in go.mod and maintain the entrypoint to the tests themselves. Otherwise if it's like the Gateway API conformance tests where this repo will contain the entrypoint to the tests and implementors are expected to invoke that, then I don't think this change is strictly necessary.

That makes sense - I don't think it's a mutually exclusive decision, as Gateway API has conformance tests under a conformance/tests directory without _test suffixes. It's definitely encouraged (and maybe required for conformance acceptance) to run the suite standalone rather than integrated into an implementation, but I know with the current maturity of some existing implementations (and the suite itself), it may be helpful to accept this change to enable some tweaks or customizations around the framework while we work towards a more mature conformance program.

@lauralorenz @skitt do y'all have thoughts on this?

[lauralorenz]

It's been a while but I did expect the entrypoint to be maintained in this repo to be invoked by the implementors, but nothing is final, so I think this is fine. Reading back this in particular originates from some ginkgo autogeneration rules that don't really apply here anyways

/lgtm /approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lauralorenz, Xunzhuo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [lauralorenz]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[skitt]

Oops, yes, I have thoughts on this, but they align with Laura’s — I do think there’s value in providing a conformance suite which can easily be run by third-parties.