Make functional test suites exported (take 2)

[aromanovich]

What changed?

Test suite implementations are moved from tests/{xdc,ndc}/*_test.go to tests/{xdc,ndc}/*go files.

Why?

Making test suites available to third-party packages. Following up on #5205.

How did you test it?

Ran tests.

Potential risks

None.

Is hotfix candidate?

No.

[rodrigozhou]

There's an issue that was brought up by these changes in #5205. IDEs (GoLand, VSCode) can't run tests that are not in *_test.go file. Do you know of a solution that can make the tests exportable and also can make it work seamlessly with IDE?

[aromanovich]

I should've brought it up myself, but I almost always run tests from the terminal and wasn't aware of it :(

Do you know of a solution that can make the tests exportable and also can make it work seamlessly with IDE?

Unfortunately no. I borrowed this approach from persistence tests and it still seems the only one possible.

[rodrigozhou]

@alexshtin What do you think of this? I don't mind, but I don't use any IDE.

[aromanovich]

@alexshtin, could you please take a look? Most of the tests have already been affected by this issue for a couple of months (after https://github.com/temporalio/temporal/pull/5205). tests/ndc and tests/xdc are the only exceptions.

I suppose the decision is either rollback #5205 or merge this one, to keep the code consistent.

[alexshtin]

The problem is not actually a GoLand. Tests must be in *_test.go files to be executable. GoLand just has a nice feature to run single test from a test suite with one click. But it can do it only if test itself is in a *_test.go file. This is very useful when some tests are failing and you need to debug them. Same experience can be achieved by specifying custom pattern in test configuration but this requires some knowledge and far more clicks.

For example, this is how I run "Workflow Update" tests only in FunctionalSuite: Pattern is ^\QTestFunctionalSuite\E$/^\QTestUpdateWorkflow_\E.+$.

Another workaround is temporary rename specific file back to *_test.go, run/debug test, and rename back.

Unfortunately, there is no good solution. Probably some code generation can help but I wouldn't call it nice solution either.

But I agree that ability to run tests over custom server builds is more important here. So I don't mind renaming files and removing _test.go suffix. I will try code generation approach later.

One caveat though. Without _test.go suffix, test code might get to production binaries. I would like to avoid this with build tags (should be separate PR).

[alexshtin]

Why github shows all but replication.go as add/delete instead of rename? Can you fix it (and also merge conflicts) and I will approve and merge it.

[aromanovich]

Can you fix it (and also merge conflicts) and I will approve and merge it.

I've manually repeated everything from scratch again, just to be sure:

• Move ./tests/{xdc,ndc}/*_test.go to ./tests/{xdc,ndc}/*.go only removes _test suffixes;
• Move Test*TestSuite functions back to ./tests/{xdc,ndc}/*_test.go moves all func Test*TestSuite back to where they belong.

Why github shows all but replication.go as add/delete instead of rename?

replication.go doesn't define a suite, it just adds a TestReplicationMessageDLQ to NDCFunctionalTestSuite, so it's not affected. If it were up to me, I would've merged it into ndc.go.

[aromanovich]

@alexshtin, @rodrigozhou, I've rebased it again. Could you please take a look while there are no conflicts? It's usually quite a short while :)

[yycptt]

@yux0 Plz also help review this PR as it's related to the replication stack.

[aromanovich]

@alexshtin Could you please take a look?

[github-actions[bot]]

This PR was marked as stale. Please update or close it.