experimenter icon indicating copy to clipboard operation
experimenter copied to clipboard

Integration test failure: targeting tests being tricked by targeting slug

Open jaredlockhart opened this issue 3 years ago • 1 comments

Just saw the weirdest behaviour in the targeting integration tests. We pass a couple parameters into the test, one is an application slug like DESKTOP, and the other is a targeting config slug. If the targeting config slug also includes the word desktop like something_desktop_something then it will trick pytest into running the tests with applications other than desktop which will then fail.

https://app.circleci.com/pipelines/github/mozilla/experimenter/11753/workflows/223c1fac-3cd5-4d94-a25b-6c07d0bbd489/jobs/35944

We've probably got to dig into a better way to parameterize the tests so it doesn't get tricked into running the targeting tests on applications other than desktop.

┆Issue is synchronized with this Jira Bug

jaredlockhart avatar Dec 14 '21 16:12 jaredlockhart

I think we need to have our own command line flag, and then hijack the test decision function built into pytest to use that flag as well as -k.

b4handjr avatar Dec 14 '21 17:12 b4handjr