Test docker images we would build before merge

[codefromthecrypt]

I would like to enlist github actions to test the docker images we would build on PR. I know it will take some time, but if we do it right, it would likely finish before our normal integration tests do :)

It would be nice to use java to do this, vs a myriad of shell script if statements which is the usual way. The way I see it is an optional module similar to benchmarks that a github action kicks

• run the normal mvn install, but skip normal tests as we have other things that do that
• in a separate project: integration tests...
  • prepare images using docker-maven-plugin
  • run a suite of TestContainers tests that initially just starts the zipkin image in various configs and verifies HEALTHCHECK
    • not sure if TestContainers can automatically verify HEALTHCHECK or not!
  • Later, we can test other things like posting data etc

Another option is to not integrate directly the thing in maven.. rather use a shell script to do docker build then kick off the integration tests project that doesn't implicitly build the images.

Note The building of docker images here is pre-merge. This process would not effect our release process in any way.

Thoughts? cc @openzipkin/core @bsideup @saturnism

[anuraaga]

Seems like a good idea overall. Test containers are simpler than docker compose I think.

What's the "separate project"?

Also, I'd probably just use the official docker action instead of maven plugin, despite the name it seems to only push when push: true

https://github.com/marketplace/actions/build-and-push-docker-images

Includes a paragraph about setting up cache which seems helpful.

[anuraaga]

Ah also would the dockerfile build again after mvn install? Would be nice to just use those jars but not sure if our dockerfiles are equipped for that so might be a change there.

[bsideup]

I would use buildpacks to dramatically speed up the image build.

And yes, Testcontainers does support Docker's healthchecks :)

[codefromthecrypt]

First, let's remember this is about testing not releasing. so building an image on PR and testing that.

What's the "separate project"?

Sorry I meant separate maven module, similar to benchmarks, that doesn't need to execute, but is coherent (same versions etc.. linked to parent pom). Whether it is linked or not to the parent pom may change depending on how we proceed.. For example, if we break the flow such that maven doesn't control making the images later tested.. then there's I suppose not much point in having this linked.

docker action instead of maven plugin...

hmm so if I understand correctly, you mean to start the flow with docker action, then run mvn to run the docker tests, as opposed to implicitly doing that with a docker plugin.

flow option 1: use maven the whole way (when in maven)

1. docker action runs maven to..
2. run normal server build
3. uses plugin to build Dockerfile using the built jar
4. executes testcontainers tests

change needed: small change to Dockerfile to use zipkin exec jar built by maven

pros:

• use can fire and forget (execute one command and come back later)
• IDE is aware of everything cons:
• If the docker plugin becomes abandoned, this has to be revisited
• a change to the Dockerfile is needed, even if it is a small one
• unless you break the maven build into two github action steps, caching/resume would be too coarse grained
• the docker parts are implicit so less easy to see in the github action steps

flow option 2: flip between docker action and maven

1. docker action step builds Dockerfile (which internally runs maven build)
2. docker action step runs maven to execute testcontainers tests

pros:

• docker steps use official action
• don't need to use a 3rd party maven plugin for docker cons:
• IDE has no link from source built to image built (refactoring etc)
• it is strange to explain that docker build runs the same build internally, then later we run maven externally to test that image

I'm missing pros and cons on either but trying to spike some rationale. definitely I think the latter feels very weird to me, basically the inception part of docker running maven then for maven to run tests on the docker image

[anuraaga]

Ah maven plugin would let the tests build the images in IDE too - it's like gradle docker plugin 😋 That's a big advantage for the plugin approach.

[codefromthecrypt]

neat thing is we can have multiple actions anyway :) so we could test both approaches heh

[codefromthecrypt]

yep IDE is convenient, and relevant when the images are actually tested in there :)

note: I have no plans to use any sort of plugin based dockerfile generation thingies that may or may not exist as that doesn't allow "normal docker" to work later.

Ex. we still have to allow maven-in-docker for dockerhub builds to work (status quo), as I presume we did that earlier in order to not have to worry what kind of image dockerhub uses for web hooks. In fact, I should reverse that rationale as it is very subtle!

[bsideup]

BTW have you considered using Testcontainers alone to build & test the image? https://www.testcontainers.org/features/creating_images/

[codefromthecrypt]

BTW have you considered using Testcontainers alone to build & test the image? https://www.testcontainers.org/features/creating_images/

hah no. RTFM is good sometimes!

[bsideup]

Just be careful with the API because you need to explicitly add files/folders to the build context (see the example in the docs) 👍