ct: Allow empty groups in test suites

[jchristgit]

The logic to trim empty test cases was moved from the find groups function (which will now also return groups with no testcases) to the expand groups function. Groups with no testcases will not be run (e.g. their init_per_group and end_per_group functions will not be called), but no error will be thrown as previously.

Fixes #4362.

[github-actions[bot]]

CT Test Results

    2 files    58 suites   1h 23m 30s :stopwatch: 447 tests 435 :heavy_check_mark: 12 :zzz: 0 :x: 483 runs  468 :heavy_check_mark: 15 :zzz: 0 :x:

Results for commit 5092750b.

:recycle: This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

• Complete CT logs (Download Logs)
• HTML Documentation (Download HTML Docs)
• Windows Installer

// Erlang/OTP Github Action Bot

[Maria-12648430]

Groups with no testcases will not be run (e.g. their init_per_group and end_per_group functions will not be called)

I'm not sure if this (not calling init/end_per_group) is the right thing to do. OTOH one might say what is the use of calling init/end if there are no test cases in between, OTOH I would say why is there an empty group to begin with?

But really I have no strong opinion on this, so "just saying" :wink:

[jchristgit]

I'm not sure if this (not calling init/end_per_group) is the right thing to do. OTOH one might say what is the use of calling init/end if there are no test cases in between, OTOH I would say why is there an empty group to begin with?

I chose this approach because I think that init_per_group and end_per_group are often used to initialize and cleanup something for the test cases contained within them. For instance, httpc_SUITE will start httpd servers to use for testing in there. If we do not run any test cases in the group, do we need its setup and teardown? My thought was that it would be unnecessary.

[u3s]

I'm not sure if this (not calling init/end_per_group) is the right thing to do. OTOH one might say what is the use of calling init/end if there are no test cases in between, OTOH I would say why is there an empty group to begin with?

I chose this approach because I think that init_per_group and end_per_group are often used to initialize and cleanup something for the test cases contained within them. For instance, httpc_SUITE will start httpd servers to use for testing in there. If we do not run any test cases in the group, do we need its setup and teardown? My thought was that it would be unnecessary.

Not a strong opinion, but I can thing of some counter arguments:

1. seeing group conf function without testcases in HTML - might result with operator noticing it and scratching head. hopefully figuring if group should have testcases added or should me removed from source as not relevant. operator is not always developer who knows source code
2. upon development, it might be convenient to have group conf function executed - so you can 1st develop them and then proceed with developing testcases ... of course one may write a dummy testcase for bypassing CT [lo/ma]gic ...

What do you think?