aqa-tests icon indicating copy to clipboard operation
aqa-tests copied to clipboard

check duplicate exclusion cases in ProblemList files

Open sophia-guo opened this issue 11 months ago • 2 comments

If there are more exclusion entries for same testcase only the last occurrence takes effects even if each entry has different tracked issue or different os-arch.

Testcase excluded by multiple lines happened quite a few times which causes confusion and wastes efforts to triage.

We should check ProblemList files for this to ensure no multiple entries is added, especially when PR is adding the new entry.

The main script for problemlist check is https://github.com/adoptium/aqa-tests/blob/master/scripts/disabled_tests/exclude_parser.py

https://github.com/adoptium/aqa-tests/pull/5863#issuecomment-2595798908

sophia-guo avatar Jan 16 '25 14:01 sophia-guo

See duplicate item sun/security/ssl/X509TrustManagerImpl/distrust/Symantec.java https://github.com/adoptium/aqa-tests/issues/5955 generic-all

sun/security/ssl/X509TrustManagerImpl/distrust/Symantec.java https://github.com/adoptium/aqa-tests/issues/2123 generic-all

Duplicate item may refer to different issues with confusion and may be excluded on different os-arch with the first item being mistakenly ignored.

sophia-guo avatar Mar 11 '25 15:03 sophia-guo

I agree this problem is a problem, and I'm happy to help resolve it.

We should also prevent duplicates between the exclude files and:

adamfarley avatar Sep 10 '25 15:09 adamfarley