vscode-catch2-test-adapter
vscode-catch2-test-adapter copied to clipboard
matepek
Parallel execution requires per-instance unique identifiers
Checklist
- [x] Checked Support page
Is your feature request related to a problem? Please describe. When running test instances in parallel (regardless of whether it's multiple instances of the same test, or several different tests), I face an issue where tests need to know their instance ID in order to avoid resource conflicts.
Resource conflicts mean e.g. TCP/UDP port numbers, suffixes for temporary folders etc.
There are no good / fail safe strategies to do this from within a test. Not all platforms have a reliable support for temporary folders or alike, and generating unique numbers per instance is a a surprisingly hard problem.
Describe the solution you'd like
An integral ${instanceId} variable should be provided to cwd and env.
It shall default to 0 when not using parallel test execution.
It shall be in the range from 0 to parallelExecutionLimit - 1.
It may exceed advancedExecutables.parallelizationLimit - 1 - this has to be documented!
It shall re-use IDs when the test instance previously reserving the ID has terminated.
Please figure out if it's possible to also provide export this ID to tasks triggered by runTask.beforeEach and runTask.afterEach, not just cwd and env. (There is a somewhat established pattern to stash the variable block to a global scope prefixed with your plugin before invoking the task.)
Describe alternatives you've considered Using UUIDs doesn't exactly help here, as you would often reduce this to a small value range and still avoid collisions.
It's not just limited to directories, but also resources which need integral offsets such as ports. So just providing temporary CWDs or anything like that wouldn't help.
Hello,
It shall re-use IDs when the test instance previously reserving the ID has terminated.
This is either redundant or my understanding is incomplete. Please explain.
It may exceed
advancedExecutables.parallelizationLimit - 1- this has to be documented!
Can you tell me an example when it may exceeds the mentioned limit?
This is either redundant or my understanding is incomplete. Please explain.
It is redundant, just mentioned for completeness sake what the earliest possible time of reuse is.
In case there are pre/post steps to the individual executable execution, I have probably actually forgotten to include them here.
Can you tell me an example when it may exceeds the mentioned limit?
The ID space is sized as test.parallelExecutionLimit, not just test.parallelExecutionOfExecutableLimit / advancedExecutables.parallelizationLimit, and IDs are distributed over all processes.
Running multiple tests matched by different advancedExecutables in parallel may result in the assignment of IDs outside the naively assumed "legal" value range.
Would you try this. The variable is ${testMate.var.taskSlotId}.
vscode-catch2-test-adapter-4.9.0-alpha1.vsix.zip