Improve integration testing

[Andersson007]

trafficstars

SUMMARY

To make things going in the collection, CI, that failed, was simplified with https://github.com/ansible-collections/community.rabbitmq/pull/78

I can't invest much time as not a maintainer of this collection and not a specialist in the area.

Would be nice:

1. Get rid of the simplification introduced with https://github.com/ansible-collections/community.rabbitmq/pull/78 replacing it with corresponding Ansible tasks. Uncomment # - stage: Docker_2_9 in .azure-pipelines/azure-pipelines.yml

2. Add ubuntu 16.04 back to .azure-pipelines/azure-pipelines.yml against to all the branches except devel

3. The CI triggers many distributions (like Fedora, CentOS, Suse, etc.) but tests actually run only on Ubuntu. I disabled them to reduce load on CI system. We could a) remove them from the test matrix or b) implement tests for them (for what? I think rabbitmq versions are actually important)

4. Would be nice to run tests against several versions of rabbitmq (starting with the oldest supported and finishing with the latest one).

[Andersson007]

cc @cognifloyd Let's track the progress in this issue

[petebuffon]

I'm currently in the process of adding ansible-collections-community-rabbitmq to both Fedora and CentOS. It would help me out if there were tests for both of these distributions.

I would be willing to help write these tests, but I'm not exactly sure where to start.

[Andersson007]

@petebuffon hello, thanks for your interest! You could start with:

• Prepare your environment using https://docs.ansible.com/ansible/devel/community/create_pr_quick_start.html
• Read the integration test quick-start guide just to imagine things better.
• Running integration tests locally against, say, fedora34. You can find the command in the docs above
• Try to make them working
• If you succeed, add fedora34 for testing against devel similarly like in https://github.com/ansible-collections/community.postgresql/blob/main/.azure-pipelines/azure-pipelines.yml
• Submit a pull request

If there are any questions, feel free to ask, thanks!

[Im0]

Might be worth considering removing shippable/posix like in https://github.com/ansible-collections/community.docker/pull/484 and perhaps some of utils/shippable could be tidied up? (Check other repo's)

[Andersson007]

Might be worth considering removing shippable/posix like in ansible-collections/community.docker#484 and perhaps some of utils/shippable could be tidied up? (Check other repo's)

Sure

[Im0]

4. Would be nice to run tests against several versions of rabbitmq (starting with the oldest supported and finishing with the latest one).

To this point. Perhaps we should test 3.9, 3.10 and 3.11 of rabbitmq. Details here: https://www.rabbitmq.com/versions.html Or, just test 3.9 and 3.11?

Something to be mindful of is the different erlang requirements for different versions: https://www.rabbitmq.com/which-erlang.html

[Im0]

@Andersson007 are most collections utilising github workflows these days? Is it strange/undesirable for community.rabbitmq to be using AZP? Or OK?

[Andersson007]

@Im0 it's OK

• GHA is used for distro-independent testing (its big downside is that currently it runs all available test targets regardless of files changed... which resp. increases the time needed for runs to pass)
• AZP is used when we need to test against different distros (though if you use only one distro, it's OK)