xk6-disruptor
xk6-disruptor copied to clipboard
Use the current branch's commit as the tag for Agent's integration tests
#322 introduces integration tests that use TestContainers to spawn a container with the disruptor agent using the latest
tag for the image.
This introduces a significant risk of not testing the correct version of the agent when making local tests. If the agent forgets to build the image locally it will be testing the image from the main branch. It is also possible that an older version of the image is present locally and be tested instead of the current modified version.
To safely address this issue it would be desirable to change how the agent's image is tagged and how it is referenced in the integration tests.
Notice that the policy for tagging the agent images must be synchronized with the disruptor.
In brief, on each release, the disruptor will reference the agent's image with a tag that matches the same release version. When build from a development branch or main, it will reference latest
. This is implemented in the version package.
In order to ensure integration tests use the current development version this policy must be changed.
A similar approach I've seen working in other projects is to use a ci-only image name/tag, that guarantees such image cannot be pulled. For example, something like ci.local/grafana/xk6-disruptor
.
Something like this might require adding some knobs to customize the agent image in runtime, though.
A similar approach I've seen working in other projects is to use a ci-only image name/tag
I think this approach works in contexts where the image is only pulled from the CI tests. For instance, the agent tests would work.
However, if we add any test that includes the disruptor, then the problem mentioned in the issue would likely happen.
A more robust solution is needed. I'm thinking about modifying the logic for defining the agent's image tag to support customization via an environment variable.
then the problem mentioned in the issue would likely happen.
You're right, I was thinking in CI scenarios where we can assume the image has not been built before.
Customizing image and/or tag through an env var makes a lot of sense to me 👍🏻