ansible icon indicating copy to clipboard operation
ansible copied to clipboard

Add test for 256-color configuration values

Open yurikhan opened this issue 3 years ago • 11 comments

SUMMARY

See #78607.

ISSUE TYPE
  • Test Pull Request
COMPONENT NAME

config

yurikhan avatar Aug 22 '22 16:08 yurikhan

hmm, shouldn't the test be failing?

bcoca avatar Aug 22 '22 20:08 bcoca

This commit https://github.com/ansible/ansible/pull/78613/commits/03654a2a499b34c385efafeb3379db4c16101c38 effectively disabled the tests.

sivel avatar Aug 22 '22 20:08 sivel

Yeah, the xfail mark means “expected to fail”. The fixer of the bug is supposed to remove the mark.

yurikhan avatar Aug 23 '22 08:08 yurikhan

just add the commit from my PR to this one and revert the xfail (that makes it easier to backport as a single unit)

bcoca avatar Aug 23 '22 16:08 bcoca

I assume by “my PR” you’re referring to #78617, but you accidentally the verb. Do you mean “add the commit from #78617 and revert the xfail”?

yurikhan avatar Aug 23 '22 16:08 yurikhan

I assume by “my PR” you’re referring to #78617, but you accidentally the verb. Do you mean “add the commit from #78617 and revert the xfail”?

I suppose so. You could first cherry-pick Brian's commit, ensure that the test starts failing (because xfail_strict produces a failure on XPASS — https://blog.ganssle.io/articles/2021/11/pytest-xfail.html). And then, do git revert as a finishing bit.

webknjaz avatar Aug 23 '22 16:08 webknjaz

This commit 03654a2 effectively disabled the tests.

It did not. xfail is not the same as skip. It does run the test (unless you set run=False) and then it makes sure that it fails (strict=True) makes it fail when it starts passing so when the problem is actually fixed, you get notified loudly.

webknjaz avatar Aug 23 '22 16:08 webknjaz

@yurikhan yes, just add my commit/copy the code here and revert the xfail change

bcoca avatar Aug 31 '22 14:08 bcoca

Seeing as I’m rebasing the branch to current devel anyway, I just dropped the commit that marked the test with xfail.

yurikhan avatar Aug 31 '22 16:08 yurikhan

/azp run

webknjaz avatar Aug 31 '22 19:08 webknjaz

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Aug 31 '22 19:08 azure-pipelines[bot]

@yurikhan Can you add a changelog fragment to document the change?

mattclay avatar Mar 22 '23 19:03 mattclay

Do test suite changes deserve a changelog entry, or should I describe the bug fixed instead?

Will look into it this coming weekend.

yurikhan avatar Mar 23 '23 06:03 yurikhan

@yurikhan Only the bug fix requires a changelog entry. Unit test changes should not be in the changelog.

mattclay avatar Mar 23 '23 16:03 mattclay

/azp run

bcoca avatar Jul 19 '23 15:07 bcoca

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Jul 19 '23 15:07 azure-pipelines[bot]