auto-pts icon indicating copy to clipboard operation
auto-pts copied to clipboard

Fix overlay application on test if more than one overlay applies to same test

Open KKopyscinski opened this issue 2 years ago • 2 comments

There is a case where one overlay contains group of tests and other it's subgroup. Then tests get overlay of "master" group and subgroup is not build. We should add solution that prioritizes longer test prefixes in applying overlay.

Example:

overlay1.conf: {
    "overlay": {
        'CONFIG_VAL': 'x',
    },
    "test_cases" : [
        'XXXX/YY/ZZ'
    ]
}

overlay2.conf: {
    "overlay": {
        'CONFIG_VAL': 'y',
    },
    "test_cases" : [
        'XXXX/YY/ZZ/BV-01-C'
    ]
}

All XXXX/YY/ZZ tests should be built with overlay1, but XXXX/YY/ZZ/BV-01-C should be build with overlay2.

KKopyscinski avatar May 30 '22 07:05 KKopyscinski

my gut feeling is that overlays should be always explicit when it comes to test names, ie it should always lists every single test is affects

sjanc avatar May 30 '22 07:05 sjanc

In current state it seems like we must do this, but maybe it's worth to add some flexibility as test groups? It would future proof overlay, as more test cases may be added later, and probably they will use same overlay as others

KKopyscinski avatar May 30 '22 07:05 KKopyscinski