experimenter
experimenter copied to clipboard
Integration test failure: targeting tests being tricked by targeting slug
Just saw the weirdest behaviour in the targeting integration tests. We pass a couple parameters into the test, one is an application slug like DESKTOP
, and the other is a targeting config slug. If the targeting config slug also includes the word desktop
like something_desktop_something
then it will trick pytest into running the tests with applications other than desktop which will then fail.
https://app.circleci.com/pipelines/github/mozilla/experimenter/11753/workflows/223c1fac-3cd5-4d94-a25b-6c07d0bbd489/jobs/35944
We've probably got to dig into a better way to parameterize the tests so it doesn't get tricked into running the targeting tests on applications other than desktop.
┆Issue is synchronized with this Jira Bug
I think we need to have our own command line flag, and then hijack the test decision function built into pytest to use that flag as well as -k
.