Parameterized test classes do not respect test-targets filters by annotation

[steverichey]

Describe the bug

An annotated test method that exists in a class annotated with @RunWith(Parameterized::class) will not be included in a test run that is filtered with test-targets to include only test methods with that annotation. An example project is available here, where the method in the parameterized test class is not included in android_shards.json if test-targets specifies - annotation flank.shard.annotationbug.Annotation.

To Reproduce

1. Set up an Android project with test-targets specified in the Flank config file that specifies any annotation.
2. Mark methods in a parameterized test class with the above specified annotation.
3. Run Flank with the --dump-shards option.
4. Note that the parameterized annotated methods are not included in the shards file.

Expected behavior

Test methods in parameterized test classes should be included or not based on the same rules that apply to non-parameterized test classes.

Details (please complete the following information):

Have you tested on the latest Flank snapshot?

I'm not sure where Flank snapshots are posted. Happy to test if a link is provided.

Post the output of flank --version.

version: v23.10.1
revision: 36e37aa9c85f5cefc5680efb699d195ee48c2bc9


Total run duration: 0m  0s

Additional context

To make the issue clear (and to verify some other configuration issue didn't cause this problem) I've created a demo project: https://github.com/steverichey/flankshardannotationbug

[colinmarsch]

Have you found any workarounds for this issue? Running into the same thing and it looks like others might be as well here: https://github.com/google/TestParameterInjector/issues/27

[steverichey]

@colinmarsch We decided to remove the annotation filter, as it turned out to be unnecessary in our use case.

[SelaseKay]

Hi @steverichey, thanks for the report. I'm able to reproduce this with your sample project. I'll look into it.